Merge pull request 'Reference parent conference in CFP section of weekly email' (#20) from admin into main

Reviewed-on: #20

.gitignore

@@ -1,5 +1,6 @@
 /data
 fixtures
 /static
+.env
 .env.dev
-z_README_Hugh.md
+z_README_Hugh.md

README.md

@@ -5,17 +5,18 @@ A django app running on Docker. Replaces _Aus GLAM Blogs_.
 ## Deploy
 
 * `cp .env.example .env` and enter env values for your app
-* set up web server config (nginx example coming soon)
+* set up web server config (see nginx example)
 * `docker compose build`
-* `./glamr-dev migrate`
+* `./glamr-dev migrate` (you may get an error at this point - ignore)
 * `./glamr-dev createsuperuser`
+* `./glamr-dev collectstatic`
+* copy static files to where your webserver can find them, e.g.`/srv/ausglamr`
 * `docker compose up -d`
-* set up database backups (as cron jobs): `./glamr-dev backup`:
-* set up cron jobs for management commands as below
+* set up cron jobs for management commands as outlined below
 
 ## Admin
 
-Don't forget to add some Content Warnings for use by the Mastodon bot, within `/admin`.
+Don't forget to add some **Content Warnings** for use by the Mastodon bot, within `/admin`.
 
 ## CLI tool
 
@@ -59,9 +60,9 @@ These will not be triggered within the app - they should be called via cron jobs
 
 ### announce
 
-This announces the next queued announcement on Mastodon.
+This posts the next queued announcement on Mastodon.
 
-Run every 21 mins.
+Run about every 20 mins.
 
 ### check_feeds
 
@@ -83,8 +84,46 @@ Does what you think. Creates a weekly email of the latest stuff, and send to eve
 
 Run weekly.
 
-### Backups
+## Backups
 
-There is a `backup` command in `glamr-dev`. You can adjust the filepaths in your `.env` file.
+### Creating backups
 
-Run daily
+There is a `backup` command in `glamr-dev`. The backup is named after the current day of the week, so in effect there will be a maximum of 7 rotating backups. The backup command relies on two environment variables: `DOCKER_PATH` and `BACKUPS_DIR` - you need to set these in your `.env` if you plan to run backups manually, but if you are running it with a cron job (recommended) you need to set environment variables within the crontab itself. See more details below.
+
+### Restoring backups
+
+1. back up your VPS if possible (e.g. taking a snapshot)
+2. Locate the latest backup, or run the backup program: `./glamr-dev backup`
+3. copy database dump into the container: `docker cp ~/ausglamr.dump ausglamr-db-1:/tmp/`
+4. restore the dump: `docker exec -d ausglamr-db-1 pg_restore -c -e -U ausglamr -d ausglamr /tmp/ausglamr.dump`
+
+## Cron jobs
+
+The obvious way to run the management and backup commands is via cron jobs. It's important to note that environment variables generally need to be set within the crontab - you can't rely on passing them in from Ausglamr nor from a user context. If you forget to to this they will be blank and one of the bad things that will happen is cron will try and fail to run the Unix program `exec` instead of `docker exec` when trying to run backups.
+
+Below are suggested crontab settings and jobs for Ausglamr.
+
+```
+# Aus GLAMR
+# ---------
+
+# envs
+PATH=/bin:/usr/bin:/usr/local/bin:/snap/bin
+DOCKER_PATH=/usr/bin/docker
+BACKUPS_DIR=/home/ausglamr/backups
+
+# Announce thrice an hour
+1,23,47 * * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev announce
+
+# Check feeds hourly on the eleventh minute
+11 * * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev check_feeds
+
+# Queue event announcements daily at 4:01am
+1 4 * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev queue_announcements
+
+# Run backups daily - 1:01am
+1 1 * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev backup
+
+# Send email weekly at 12:05pm
+5 12 * * 1 cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev send_weekly_email
+```

ausglamr/urls.py

@@ -1,6 +1,7 @@
 """
 URL configuration for ausglamr project.
 """
+
 from django.contrib import admin
 from django.urls import path, re_path
 from django.views.generic import TemplateView
@@ -65,6 +66,8 @@ urlpatterns = [
         views.UnsubscribeEmail.as_view(),
         name="unsubscribe-email",
     ),
-    path('robots.txt', TemplateView.as_view(template_name='robots.txt',
-                                      content_type='text/plain')),
+    path(
+        "robots.txt",
+        TemplateView.as_view(template_name="robots.txt", content_type="text/plain"),
+    ),
 ]

blogs/admin.py

@@ -131,6 +131,7 @@ class Blog(admin.ModelAdmin):
         "active",
     )
     ordering = ["approved", "-suspended", "-failing"]
+    search_fields = ["title", "author_name", "url"]
     actions = [approve, unapprove, suspend, unsuspend, activate, disable]
 
 
@@ -140,6 +141,7 @@ class Article(admin.ModelAdmin):
 
     date_hierarchy = "pubdate"
     list_display = ("title", "blog_title", "pubdate")
+    search_fields = ["title", "author_name", "blog_title", "url"]
 
     def blog_title(self, obj):  # pylint: disable=no-self-use
         """get the title of the parent blog"""
@@ -151,7 +153,7 @@ class Tag(admin.ModelAdmin):
     """display settings for tags"""
 
     list_display = ("name",)
-
+    search_fields = ["name"]
 
 @admin.register(models.Event)
 class Event(admin.ModelAdmin):
@@ -166,6 +168,7 @@ class Event(admin.ModelAdmin):
         "start_date",
     )
     ordering = ["approved", "announcements"]
+    search_fields = ["name", "description", "url"]
     actions = [approve, unapprove]
 
 
@@ -176,6 +179,7 @@ class CallForPapers(admin.ModelAdmin):
     list_display = ("name", "event", "approved", "closing_date")
     list_select_related = ("event",)
     ordering = ["approved", "closing_date"]
+    search_fields = ["event__name", "event__url", "details"]
     actions = [approve, unapprove]
 
 
@@ -185,6 +189,7 @@ class Group(admin.ModelAdmin):
 
     list_display = ("name", "approved", "category", "description")
     ordering = ["approved", "name"]
+    search_fields = ["name", "description", "url"]
     actions = [approve, unapprove]
 
 
@@ -200,6 +205,7 @@ class Newsletter(admin.ModelAdmin):
         "active",
     )
     ordering = ["approved", "-failing"]
+    search_fields = ["name", "description", "url", "author_name"]
     actions = [approve, unapprove, suspend, activate, disable]
 
 
@@ -209,6 +215,7 @@ class Edition(admin.ModelAdmin):
 
     date_hierarchy = "pubdate"
     list_display = ("title", "newsletter_name", "pubdate")
+    search_fields = ["title", "description", "url", "newsletter__name", "author_name"]
 
     def newsletter_name(self, obj):  # pylint: disable=no-self-use
         """get the title of the parent newsletter"""

blogs/apps.py

@@ -1,4 +1,5 @@
 """django apps"""
+
 from django.apps import AppConfig
 
 

blogs/forms.py

@@ -141,8 +141,8 @@ class ContactForm(forms.Form):
     bot_check = forms.CharField(
         label="What is usually stored in a library?",
         max_length=10,
-        help_text="Checking that you are human"
-        )
+        help_text="Checking that you are human",
+    )
 
     def clean_bot_check(self):
         """validate the bot check"""

blogs/management/commands/check_feeds.py

@@ -117,7 +117,7 @@ class Command(BaseCommand):
                             if not opt_out:
                                 author_name = getattr(
                                     article, "author", None
-                                ) or getattr(blog, "author", None)
+                                ) or getattr(blog, "author", "")
 
                                 description = (
                                     html.strip_tags(article.summary)
@@ -125,26 +125,32 @@ class Command(BaseCommand):
                                         hasattr(article, "summary")
                                         and len(article.summary) > 0
                                     )
-                                    else html.strip_tags(article.description)
-                                    if (
-                                        hasattr(article, "description")
-                                        and len(article.summary)
+                                    else (
+                                        html.strip_tags(article.description)
+                                        if (
+                                            hasattr(article, "description")
+                                            and len(article.summary)
+                                        )
+                                        else (
+                                            html.strip_tags(article.content[0].value)
+                                            if (
+                                                hasattr(article, "content")
+                                                and len(article.content)
+                                            )
+                                            else None
+                                        )
                                     )
-                                    else html.strip_tags(article.content[0].value)[:200]
-                                    if (
-                                        hasattr(article, "content")
-                                        and len(article.content)
-                                    )
-                                    else ""
                                 )
                                 if description:
-                                    description += "..."
+                                    desc = description[:200] + "..."
+                                else:
+                                    desc = ""
 
                                 instance = models.Article.objects.create(
                                     title=article.title,
                                     author_name=author_name,
                                     url=article.link,
-                                    description=description,
+                                    description=desc,
                                     updateddate=date_to_tz_aware(
                                         article.updated_parsed
                                     ),
@@ -168,7 +174,9 @@ class Command(BaseCommand):
                                 if newish:
                                     instance.announce()
                                     blog.set_success(
-                                        updateddate=date_to_tz_aware(article.updated_parsed)
+                                        updateddate=date_to_tz_aware(
+                                            article.updated_parsed
+                                        )
                                     )
 
                 except Exception as e:
@@ -191,7 +199,7 @@ class Command(BaseCommand):
                             | Q(guid=getattr(edition, "id", edition.link))
                         ).exists():
                             author_name = getattr(edition, "author", None) or getattr(
-                                edition, "author", None
+                                edition, "author", ""
                             )
 
                             description = (
@@ -199,26 +207,32 @@ class Command(BaseCommand):
                                 if (
                                     hasattr(edition, "summary") and len(edition.summary)
                                 )
-                                else html.strip_tags(edition.description)
-                                if (
-                                    hasattr(edition, "description")
-                                    and len(edition.description)
+                                else (
+                                    html.strip_tags(edition.description)
+                                    if (
+                                        hasattr(edition, "description")
+                                        and len(edition.description)
+                                    )
+                                    else (
+                                        html.strip_tags(edition.content[0].value)
+                                        if (
+                                            hasattr(article, "content")
+                                            and len(article.content)
+                                        )
+                                        else None
+                                    )
                                 )
-                                else html.strip_tags(edition.content[0].value)[:200] + "..."
-                                if (
-                                    hasattr(article, "content")
-                                    and len(article.content)
-                                )
-                                else ""
                             )
                             if description:
-                                description += "..."
+                                desc = description[:200] + "..."
+                            else:
+                                desc = ""
 
                             instance = models.Edition.objects.create(
                                 title=edition.title,
                                 author_name=author_name,
                                 url=edition.link,
-                                description=description,
+                                description=desc,
                                 updateddate=date_to_tz_aware(edition.updated_parsed),
                                 newsletter=newsletter,
                                 pubdate=date_to_tz_aware(edition.published_parsed),

blogs/management/commands/send_weekly_email.py

@@ -133,11 +133,11 @@ class Command(BaseCommand):
         for instance in cfps:
             c_date = instance.closing_date
             title_string = (
-                f"<h4><a href='{instance.event.url}'>{instance.name}</a></h4>"
+                f"<h4><a href='{instance.event.url}'>{instance.event.name} - {instance.name}</a></h4>"
             )
             dates_string = f"<p><strong>Closes:</strong><em>{c_date:%a} {c_date.day} {c_date:%B}</em></p>"
             description_string = (
-                f"<p style='margin-bottom:24px;'>{instance.details}</p>"
+                f"<p>{instance.details}</p><p style='margin-bottom:24px;'>{instance.event.description}</p>"
             )
 
             string_list = [title_string, dates_string, description_string]

blogs/views/feeds.py

@@ -94,9 +94,7 @@ class EventFeed(Feed):
         return (
             item.description
             if hasattr(item, "description")
-            else item.details
-            if hasattr(item, "details")
-            else None
+            else item.details if hasattr(item, "details") else None
         )
 
     def item_link(self, item):
@@ -215,9 +213,7 @@ class CombinedFeed(Feed):
         return (
             item.description
             if hasattr(item, "description")
-            else item.details
-            if hasattr(item, "details")
-            else None
+            else item.details if hasattr(item, "details") else None
         )
 
     def item_link(self, item):

blogs/views/public.py

@@ -194,9 +194,9 @@ class RegisterBlog(View):
                 data["blog_info"] = blog_info
 
             else:
-                data[
-                    "error"
-                ] = "Could not auto-discover your feed info, please enter manually"
+                data["error"] = (
+                    "Could not auto-discover your feed info, please enter manually"
+                )
 
             return render(request, "blogs/confirm-register.html", data)
 

docker-compose.yml

@@ -1,5 +1,3 @@
-version: '3'
-
 services:
   db:
     image: postgres:13
@@ -25,4 +23,4 @@ services:
 volumes:
   pgdata:
 networks:
-  main:
+  main:

glamr-dev

@@ -44,6 +44,9 @@ case "$CMD" in
     collectstatic)
         runweb python manage.py collectstatic
         ;;
+    copystatic)
+		cp -r static /srv/ausglamr/
+    	;;
     createsuperuser)
         runweb python manage.py createsuperuser --no-input
         ;;
@@ -80,4 +83,4 @@ case "$CMD" in
         set +x
         echo "That is not a command"
         ;;
-esac
+esac

nginx.conf.example

@@ -0,0 +1,24 @@
+server {
+    server_name  example.com;
+
+	location = /favicon.ico { 
+		access_log off; 
+		log_not_found off; 
+	}
+
+    location /static {
+        autoindex on;
+        alias /srv/ausglamr/static;
+    }
+
+    location / {
+        root   html;
+        index  index.html index.htm;
+        proxy_pass http://127.0.0.1:8080;
+    }
+
+    error_page   500 502 503 504  /50x.html;
+    location = /50x.html {
+        root   html;
+    }
+}

requirements.txt

@@ -1,12 +1,12 @@
 beautifulsoup4==4.12.2
 gunicorn==22.0.0
-Django==4.2.11
+Django==4.2.14
 environs==9.5.0
 feedparser==6.0.10
 psycopg2==2.9.5
-requests==2.31.0
+requests==2.32.3
 
 # dev
 
 black==24.3.0
-pylint==3.0.3
+pylint==3.0.3