Compare commits

..

26 commits

Author SHA1 Message Date
Hugh Rundle cda4b61b4b Merge pull request 'Reference parent conference in CFP section of weekly email' (#20) from admin into main
Reviewed-on: #20
2024-11-09 10:02:07 +11:00
Hugh Rundle 56094b9e7e Merge branch 'main' into admin 2024-11-09 10:01:40 +11:00
Hugh Rundle 507d6d3508
close p tag properly 2024-11-09 09:59:57 +11:00
Hugh Rundle 5ce0a679fb
Add event detail to call for papers in weekly email
fixes #18
2024-11-09 09:58:40 +11:00
Hugh Rundle e814c9f981 Merge pull request 'add search in admin models' (#19) from admin into main
Reviewed-on: #19
2024-11-09 09:52:56 +11:00
Hugh Rundle da58002f2f
clean up readme 2024-11-09 09:51:49 +11:00
Hugh Rundle 60451f565a
Merge branch 'main' into admin 2024-11-09 09:50:18 +11:00
Hugh Rundle 2e0bcb1c01
Merge branch 'main' of git.suboptimal.solutions:hugh/ausglamr 2024-11-09 09:49:53 +11:00
Hugh Rundle 16561eba6f
add search to admin screens 2024-11-09 09:44:36 +11:00
Hugh Rundle 9a355ff2a0 Merge pull request 'Bump django from 4.2.11 to 4.2.14' (#16) from dependabot/pip/django-4.2.14 into main
Reviewed-on: #16
2024-07-30 20:28:34 +10:00
Hugh Rundle 1864761a47 Merge branch 'main' into dependabot/pip/django-4.2.14 2024-07-30 20:28:03 +10:00
Hugh Rundle cd42612596 Merge pull request 'update docs and glamr-dev' (#17) from docs into main
Reviewed-on: #17
2024-07-30 20:27:15 +10:00
Hugh Rundle 7ce005d2e0
update docs and glamr-dev
- improve README: adds instructions for restoring backups, and setting crontab
- remove deprecated "version" from docker-compose
- add nginx example conf
- add .env to .gitignore (!)
2024-07-30 20:23:22 +10:00
Hugh Rundle c8fc3b4c45
Merge pull request #17 from hughrun/hughrun-patch-2
Update README.md to point to new repo
2024-07-29 20:08:49 +10:00
Hugh Rundle e8bebca7b4
Update README.md to point to new repo 2024-07-29 20:08:30 +10:00
dependabot[bot] d57c44cc85
Bump django from 4.2.11 to 4.2.14
Bumps [django](https://github.com/django/django) from 4.2.11 to 4.2.14.
- [Commits](https://github.com/django/django/compare/4.2.11...4.2.14)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-10 22:08:59 +00:00
Hugh Rundle d70c1f0182
Merge pull request #15 from hughrun/fix-summaries
fix faulty description appending code
2024-06-28 18:29:12 +10:00
Hugh Rundle fd773817eb
fix faulty description appending code 2024-06-28 18:28:26 +10:00
Hugh Rundle 75de2f1c2c
Merge pull request #14 from hughrun/fix-summaries
Fix summaries
2024-06-24 12:26:24 +10:00
Hugh Rundle 88d56e5e04
formatting 2024-06-24 12:23:01 +10:00
Hugh Rundle 365b7e7d78
don't use entire post if it is in the "summary". 2024-06-24 12:22:35 +10:00
Hugh Rundle e2ab82c2de
Merge pull request #13 from hughrun/hughrun-patch-1
Update requirements.txt
2024-06-17 16:29:02 +10:00
Hugh Rundle ab6f26c046
Update requirements.txt
requests 2.32.0 was yanked.
2024-06-17 16:28:44 +10:00
Hugh Rundle 929b9fb5ef
Merge pull request #12 from hughrun/editions
check newsletter description properly
2024-06-17 16:22:24 +10:00
Hugh Rundle 9932e57b93
Merge pull request #10 from hughrun/dependabot/pip/requests-2.32.0
Bump requests from 2.31.0 to 2.32.0
2024-06-17 16:22:04 +10:00
dependabot[bot] 5bcc9f6ce5
---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-21 07:16:03 +00:00
14 changed files with 147 additions and 61 deletions

1
.gitignore vendored
View file

@ -1,5 +1,6 @@
/data /data
fixtures fixtures
/static /static
.env
.env.dev .env.dev
z_README_Hugh.md z_README_Hugh.md

View file

@ -5,17 +5,18 @@ A django app running on Docker. Replaces _Aus GLAM Blogs_.
## Deploy ## Deploy
* `cp .env.example .env` and enter env values for your app * `cp .env.example .env` and enter env values for your app
* set up web server config (nginx example coming soon) * set up web server config (see nginx example)
* `docker compose build` * `docker compose build`
* `./glamr-dev migrate` * `./glamr-dev migrate` (you may get an error at this point - ignore)
* `./glamr-dev createsuperuser` * `./glamr-dev createsuperuser`
* `./glamr-dev collectstatic`
* copy static files to where your webserver can find them, e.g.`/srv/ausglamr`
* `docker compose up -d` * `docker compose up -d`
* set up database backups (as cron jobs): `./glamr-dev backup`: * set up cron jobs for management commands as outlined below
* set up cron jobs for management commands as below
## Admin ## Admin
Don't forget to add some Content Warnings for use by the Mastodon bot, within `/admin`. Don't forget to add some **Content Warnings** for use by the Mastodon bot, within `/admin`.
## CLI tool ## CLI tool
@ -59,9 +60,9 @@ These will not be triggered within the app - they should be called via cron jobs
### announce ### announce
This announces the next queued announcement on Mastodon. This posts the next queued announcement on Mastodon.
Run every 21 mins. Run about every 20 mins.
### check_feeds ### check_feeds
@ -83,8 +84,46 @@ Does what you think. Creates a weekly email of the latest stuff, and send to eve
Run weekly. Run weekly.
### Backups ## Backups
There is a `backup` command in `glamr-dev`. You can adjust the filepaths in your `.env` file. ### Creating backups
Run daily There is a `backup` command in `glamr-dev`. The backup is named after the current day of the week, so in effect there will be a maximum of 7 rotating backups. The backup command relies on two environment variables: `DOCKER_PATH` and `BACKUPS_DIR` - you need to set these in your `.env` if you plan to run backups manually, but if you are running it with a cron job (recommended) you need to set environment variables within the crontab itself. See more details below.
### Restoring backups
1. back up your VPS if possible (e.g. taking a snapshot)
2. Locate the latest backup, or run the backup program: `./glamr-dev backup`
3. copy database dump into the container: `docker cp ~/ausglamr.dump ausglamr-db-1:/tmp/`
4. restore the dump: `docker exec -d ausglamr-db-1 pg_restore -c -e -U ausglamr -d ausglamr /tmp/ausglamr.dump`
## Cron jobs
The obvious way to run the management and backup commands is via cron jobs. It's important to note that environment variables generally need to be set within the crontab - you can't rely on passing them in from Ausglamr nor from a user context. If you forget to to this they will be blank and one of the bad things that will happen is cron will try and fail to run the Unix program `exec` instead of `docker exec` when trying to run backups.
Below are suggested crontab settings and jobs for Ausglamr.
```
# Aus GLAMR
# ---------
# envs
PATH=/bin:/usr/bin:/usr/local/bin:/snap/bin
DOCKER_PATH=/usr/bin/docker
BACKUPS_DIR=/home/ausglamr/backups
# Announce thrice an hour
1,23,47 * * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev announce
# Check feeds hourly on the eleventh minute
11 * * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev check_feeds
# Queue event announcements daily at 4:01am
1 4 * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev queue_announcements
# Run backups daily - 1:01am
1 1 * * * cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev backup
# Send email weekly at 12:05pm
5 12 * * 1 cd /home/ausglamr/ausglamr && /home/ausglamr/ausglamr/glamr-dev send_weekly_email
```

View file

@ -1,6 +1,7 @@
""" """
URL configuration for ausglamr project. URL configuration for ausglamr project.
""" """
from django.contrib import admin from django.contrib import admin
from django.urls import path, re_path from django.urls import path, re_path
from django.views.generic import TemplateView from django.views.generic import TemplateView
@ -65,6 +66,8 @@ urlpatterns = [
views.UnsubscribeEmail.as_view(), views.UnsubscribeEmail.as_view(),
name="unsubscribe-email", name="unsubscribe-email",
), ),
path('robots.txt', TemplateView.as_view(template_name='robots.txt', path(
content_type='text/plain')), "robots.txt",
TemplateView.as_view(template_name="robots.txt", content_type="text/plain"),
),
] ]

View file

@ -131,6 +131,7 @@ class Blog(admin.ModelAdmin):
"active", "active",
) )
ordering = ["approved", "-suspended", "-failing"] ordering = ["approved", "-suspended", "-failing"]
search_fields = ["title", "author_name", "url"]
actions = [approve, unapprove, suspend, unsuspend, activate, disable] actions = [approve, unapprove, suspend, unsuspend, activate, disable]
@ -140,6 +141,7 @@ class Article(admin.ModelAdmin):
date_hierarchy = "pubdate" date_hierarchy = "pubdate"
list_display = ("title", "blog_title", "pubdate") list_display = ("title", "blog_title", "pubdate")
search_fields = ["title", "author_name", "blog_title", "url"]
def blog_title(self, obj): # pylint: disable=no-self-use def blog_title(self, obj): # pylint: disable=no-self-use
"""get the title of the parent blog""" """get the title of the parent blog"""
@ -151,7 +153,7 @@ class Tag(admin.ModelAdmin):
"""display settings for tags""" """display settings for tags"""
list_display = ("name",) list_display = ("name",)
search_fields = ["name"]
@admin.register(models.Event) @admin.register(models.Event)
class Event(admin.ModelAdmin): class Event(admin.ModelAdmin):
@ -166,6 +168,7 @@ class Event(admin.ModelAdmin):
"start_date", "start_date",
) )
ordering = ["approved", "announcements"] ordering = ["approved", "announcements"]
search_fields = ["name", "description", "url"]
actions = [approve, unapprove] actions = [approve, unapprove]
@ -176,6 +179,7 @@ class CallForPapers(admin.ModelAdmin):
list_display = ("name", "event", "approved", "closing_date") list_display = ("name", "event", "approved", "closing_date")
list_select_related = ("event",) list_select_related = ("event",)
ordering = ["approved", "closing_date"] ordering = ["approved", "closing_date"]
search_fields = ["event__name", "event__url", "details"]
actions = [approve, unapprove] actions = [approve, unapprove]
@ -185,6 +189,7 @@ class Group(admin.ModelAdmin):
list_display = ("name", "approved", "category", "description") list_display = ("name", "approved", "category", "description")
ordering = ["approved", "name"] ordering = ["approved", "name"]
search_fields = ["name", "description", "url"]
actions = [approve, unapprove] actions = [approve, unapprove]
@ -200,6 +205,7 @@ class Newsletter(admin.ModelAdmin):
"active", "active",
) )
ordering = ["approved", "-failing"] ordering = ["approved", "-failing"]
search_fields = ["name", "description", "url", "author_name"]
actions = [approve, unapprove, suspend, activate, disable] actions = [approve, unapprove, suspend, activate, disable]
@ -209,6 +215,7 @@ class Edition(admin.ModelAdmin):
date_hierarchy = "pubdate" date_hierarchy = "pubdate"
list_display = ("title", "newsletter_name", "pubdate") list_display = ("title", "newsletter_name", "pubdate")
search_fields = ["title", "description", "url", "newsletter__name", "author_name"]
def newsletter_name(self, obj): # pylint: disable=no-self-use def newsletter_name(self, obj): # pylint: disable=no-self-use
"""get the title of the parent newsletter""" """get the title of the parent newsletter"""

View file

@ -1,4 +1,5 @@
"""django apps""" """django apps"""
from django.apps import AppConfig from django.apps import AppConfig

View file

@ -141,7 +141,7 @@ class ContactForm(forms.Form):
bot_check = forms.CharField( bot_check = forms.CharField(
label="What is usually stored in a library?", label="What is usually stored in a library?",
max_length=10, max_length=10,
help_text="Checking that you are human" help_text="Checking that you are human",
) )
def clean_bot_check(self): def clean_bot_check(self):

View file

@ -117,7 +117,7 @@ class Command(BaseCommand):
if not opt_out: if not opt_out:
author_name = getattr( author_name = getattr(
article, "author", None article, "author", None
) or getattr(blog, "author", None) ) or getattr(blog, "author", "")
description = ( description = (
html.strip_tags(article.summary) html.strip_tags(article.summary)
@ -125,26 +125,32 @@ class Command(BaseCommand):
hasattr(article, "summary") hasattr(article, "summary")
and len(article.summary) > 0 and len(article.summary) > 0
) )
else html.strip_tags(article.description) else (
html.strip_tags(article.description)
if ( if (
hasattr(article, "description") hasattr(article, "description")
and len(article.summary) and len(article.summary)
) )
else html.strip_tags(article.content[0].value)[:200] else (
html.strip_tags(article.content[0].value)
if ( if (
hasattr(article, "content") hasattr(article, "content")
and len(article.content) and len(article.content)
) )
else "" else None
)
)
) )
if description: if description:
description += "..." desc = description[:200] + "..."
else:
desc = ""
instance = models.Article.objects.create( instance = models.Article.objects.create(
title=article.title, title=article.title,
author_name=author_name, author_name=author_name,
url=article.link, url=article.link,
description=description, description=desc,
updateddate=date_to_tz_aware( updateddate=date_to_tz_aware(
article.updated_parsed article.updated_parsed
), ),
@ -168,7 +174,9 @@ class Command(BaseCommand):
if newish: if newish:
instance.announce() instance.announce()
blog.set_success( blog.set_success(
updateddate=date_to_tz_aware(article.updated_parsed) updateddate=date_to_tz_aware(
article.updated_parsed
)
) )
except Exception as e: except Exception as e:
@ -191,7 +199,7 @@ class Command(BaseCommand):
| Q(guid=getattr(edition, "id", edition.link)) | Q(guid=getattr(edition, "id", edition.link))
).exists(): ).exists():
author_name = getattr(edition, "author", None) or getattr( author_name = getattr(edition, "author", None) or getattr(
edition, "author", None edition, "author", ""
) )
description = ( description = (
@ -199,26 +207,32 @@ class Command(BaseCommand):
if ( if (
hasattr(edition, "summary") and len(edition.summary) hasattr(edition, "summary") and len(edition.summary)
) )
else html.strip_tags(edition.description) else (
html.strip_tags(edition.description)
if ( if (
hasattr(edition, "description") hasattr(edition, "description")
and len(edition.description) and len(edition.description)
) )
else html.strip_tags(edition.content[0].value)[:200] + "..." else (
html.strip_tags(edition.content[0].value)
if ( if (
hasattr(article, "content") hasattr(article, "content")
and len(article.content) and len(article.content)
) )
else "" else None
)
)
) )
if description: if description:
description += "..." desc = description[:200] + "..."
else:
desc = ""
instance = models.Edition.objects.create( instance = models.Edition.objects.create(
title=edition.title, title=edition.title,
author_name=author_name, author_name=author_name,
url=edition.link, url=edition.link,
description=description, description=desc,
updateddate=date_to_tz_aware(edition.updated_parsed), updateddate=date_to_tz_aware(edition.updated_parsed),
newsletter=newsletter, newsletter=newsletter,
pubdate=date_to_tz_aware(edition.published_parsed), pubdate=date_to_tz_aware(edition.published_parsed),

View file

@ -133,11 +133,11 @@ class Command(BaseCommand):
for instance in cfps: for instance in cfps:
c_date = instance.closing_date c_date = instance.closing_date
title_string = ( title_string = (
f"<h4><a href='{instance.event.url}'>{instance.name}</a></h4>" f"<h4><a href='{instance.event.url}'>{instance.event.name} - {instance.name}</a></h4>"
) )
dates_string = f"<p><strong>Closes:</strong><em>{c_date:%a} {c_date.day} {c_date:%B}</em></p>" dates_string = f"<p><strong>Closes:</strong><em>{c_date:%a} {c_date.day} {c_date:%B}</em></p>"
description_string = ( description_string = (
f"<p style='margin-bottom:24px;'>{instance.details}</p>" f"<p>{instance.details}</p><p style='margin-bottom:24px;'>{instance.event.description}</p>"
) )
string_list = [title_string, dates_string, description_string] string_list = [title_string, dates_string, description_string]

View file

@ -94,9 +94,7 @@ class EventFeed(Feed):
return ( return (
item.description item.description
if hasattr(item, "description") if hasattr(item, "description")
else item.details else item.details if hasattr(item, "details") else None
if hasattr(item, "details")
else None
) )
def item_link(self, item): def item_link(self, item):
@ -215,9 +213,7 @@ class CombinedFeed(Feed):
return ( return (
item.description item.description
if hasattr(item, "description") if hasattr(item, "description")
else item.details else item.details if hasattr(item, "details") else None
if hasattr(item, "details")
else None
) )
def item_link(self, item): def item_link(self, item):

View file

@ -194,9 +194,9 @@ class RegisterBlog(View):
data["blog_info"] = blog_info data["blog_info"] = blog_info
else: else:
data[ data["error"] = (
"error" "Could not auto-discover your feed info, please enter manually"
] = "Could not auto-discover your feed info, please enter manually" )
return render(request, "blogs/confirm-register.html", data) return render(request, "blogs/confirm-register.html", data)

View file

@ -1,5 +1,3 @@
version: '3'
services: services:
db: db:
image: postgres:13 image: postgres:13

View file

@ -44,6 +44,9 @@ case "$CMD" in
collectstatic) collectstatic)
runweb python manage.py collectstatic runweb python manage.py collectstatic
;; ;;
copystatic)
cp -r static /srv/ausglamr/
;;
createsuperuser) createsuperuser)
runweb python manage.py createsuperuser --no-input runweb python manage.py createsuperuser --no-input
;; ;;

24
nginx.conf.example Normal file
View file

@ -0,0 +1,24 @@
server {
server_name example.com;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /static {
autoindex on;
alias /srv/ausglamr/static;
}
location / {
root html;
index index.html index.htm;
proxy_pass http://127.0.0.1:8080;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}

View file

@ -1,10 +1,10 @@
beautifulsoup4==4.12.2 beautifulsoup4==4.12.2
gunicorn==22.0.0 gunicorn==22.0.0
Django==4.2.11 Django==4.2.14
environs==9.5.0 environs==9.5.0
feedparser==6.0.10 feedparser==6.0.10
psycopg2==2.9.5 psycopg2==2.9.5
requests==2.31.0 requests==2.32.3
# dev # dev