Ephes Blog

Miscellaneous things. Mostly Weeknotes and links I stumbled upon.

RSS feed icon Plain RSS

Weeknotes 2023-08-28

, Jochen

"The first principle is that you must not fool yourself — and you are the easiest person to fool." (Feynman)

Started working on a new project, which is always exciting. Refactored django-cast filters.py -> it's almost readable now 😆. Facet counts are now available via the API, so cast-vue got an update showing filters for categories and tags.

Articles

Mastodon / Twitter

Software

Podcasts

Weeknotes 2023-08-21

, Jochen

We begin in admiration and end by organizing our disappointment. --Gaston Bachelard

Attended a local Django meetup in Cologne and had to learn that the one in Cologne is unfortunately the last active Django meetup in Germany 😢. The topic was django CMS which I knew nothing about, and I was surprised how similar it is to Wagtail which I use a lot.

Added a categories/tags feature (beta) to django-cast, which I postponed because I was afraid that messing with the faceted navigation code in filters.py (based on django-filter) would be hard. It turned out to be as hard as I expected, but now it's done 😅.

On Saturday we went hiking in the Neandertal 🥾, because that's where I live. And we managed to leave it the way you want to find it: as a fascinating puzzle for archaeologists. We did this by throwing a baby carriage (without the baby) into the Düssel (the river that formed the valley).

Articles

Websites

Videos

Books

Weeknotes 2023-08-14

, Jochen
Just as most people are not going to stop eating fast food because home cooking is better, they’re not going to abandon social media because blogging is awesome. But that doesn’t mean you have to line up for a Big Mac. And it doesn’t mean your internet experience has to suck --Mike Grindle

Normal working week. Improved media file backup for django-cast (#99) and wrote a TIL blog post about how to test Django management commands with pytest. Attended a rust meetup for the first time and am now looking for a use case for rust as it seems to be really cool (I learned static typing via Java and got burned, but static typing in rust is much better - might be time to revisit that).

Articles

Weeknotes

Mastodon / Twitter

Software

Books

Podcasts

Testing Django Management Commands with PyTest

, Jochen

The other day I wanted to test some Django management commands for django-cast and found this excellent post by Adam Johnson:

How to Unit Test a Django Management Command

But I use pytest to write my Django tests, so how would this be different?

A Management Command to Back up Media Files

Ok, so I wrote this little command that creates a backup of all media files. Django 4.2 introduced a new STORAGES setting, so it's now possible to write a backup command that works without having to know exactly how the media files are stored. It only assumes there are entries for production and backup storage backends and they are configured correctly. So it doesn't matter if your files are stored on S3 and the backup should be written to the local filesystem, or vice versa. The media_backup command does not need to know any of this.

from django.core.management.base import BaseCommand

from ...utils import storage_walk_paths
from .storage_backend import get_production_and_backup_storage


class Command(BaseCommand):
    help = (
        "backup media files from production to backup storage "
        "(requires Django >= 4.2 and production and backup storage configured)"
    )

    @staticmethod
    def backup_media_files(production_storage, backup_storage):
        for num, path in enumerate(storage_walk_paths(production_storage)):
            if not backup_storage.exists(path):
                with production_storage.open(path, "rb") as in_f:
                    backup_storage.save(path, in_f)
            if num % 100 == 0:  # pragma: no cover
                print(".", end="", flush=True)

    def handle(self, *args, **options):
        self.backup_media_files(*get_production_and_backup_storage())

Read on: TIL: Testing Django Management Commands with PyTest

Using staticfiles with STORAGES in Django 4.2

, Jochen

The other day I was trying to improve a management command of django-cast to make it easier to backup media files like images and videos. There's already an existing command, but it had to assume that you stored your media files on s3 and wanted the backup to be stored on the local filesystem. It would be great if you could configure your production and backup storage and have the backup command work either way. And the new STORAGES setting added in Django 4.2 looks like a perfect fit for this. So I tried using such a configuration:

STORAGES = {
    "default": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "staticfiles": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
        "OPTIONS": {
            "location": "staticfiles",
            "base_url": "/static/",
        },
    },
    "production": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "backup": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
        "OPTIONS": {
            "location": ROOT_DIR.path("backups").path("media"),
        },
    },
}

But it didn't work. It took me longer than I would like to admit to figure out that I should have used this config:

STORAGES = {
    "default": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
    "production": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "backup": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
        "OPTIONS": {
            "location": ROOT_DIR.path("backups").path("media"),
        },
    },
}

If you want to replace django.contrib.staticfiles.storage.StaticFilesStorage with whitenoise for production, it's sufficient overwrite the backend in the production settings like this:

STORAGES["staticfiles"]["BACKEND"] = "whitenoise.storage.CompressedManifestStaticFilesStorage"