Ephes Blog

Miscellaneous things. Mostly Weeknotes and links I stumbled upon.

RSS feed icon Plain RSS
Date -

Testing Django Management Commands with PyTest

, Jochen

The other day I wanted to test some Django management commands for django-cast and found this excellent post by Adam Johnson:

How to Unit Test a Django Management Command

But I use pytest to write my Django tests, so how would this be different?

A Management Command to Back up Media Files

Ok, so I wrote this little command that creates a backup of all media files. Django 4.2 introduced a new STORAGES setting, so it's now possible to write a backup command that works without having to know exactly how the media files are stored. It only assumes there are entries for production and backup storage backends and they are configured correctly. So it doesn't matter if your files are stored on S3 and the backup should be written to the local filesystem, or vice versa. The media_backup command does not need to know any of this.

from django.core.management.base import BaseCommand

from ...utils import storage_walk_paths
from .storage_backend import get_production_and_backup_storage


class Command(BaseCommand):
    help = (
        "backup media files from production to backup storage "
        "(requires Django >= 4.2 and production and backup storage configured)"
    )

    @staticmethod
    def backup_media_files(production_storage, backup_storage):
        for num, path in enumerate(storage_walk_paths(production_storage)):
            if not backup_storage.exists(path):
                with production_storage.open(path, "rb") as in_f:
                    backup_storage.save(path, in_f)
            if num % 100 == 0:  # pragma: no cover
                print(".", end="", flush=True)

    def handle(self, *args, **options):
        self.backup_media_files(*get_production_and_backup_storage())

Read on: TIL: Testing Django Management Commands with PyTest

Using staticfiles with STORAGES in Django 4.2

, Jochen

The other day I was trying to improve a management command of django-cast to make it easier to backup media files like images and videos. There's already an existing command, but it had to assume that you stored your media files on s3 and wanted the backup to be stored on the local filesystem. It would be great if you could configure your production and backup storage and have the backup command work either way. And the new STORAGES setting added in Django 4.2 looks like a perfect fit for this. So I tried using such a configuration:

STORAGES = {
    "default": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "staticfiles": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
        "OPTIONS": {
            "location": "staticfiles",
            "base_url": "/static/",
        },
    },
    "production": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "backup": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
        "OPTIONS": {
            "location": ROOT_DIR.path("backups").path("media"),
        },
    },
}

But it didn't work. It took me longer than I would like to admit to figure out that I should have used this config:

STORAGES = {
    "default": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
    "production": {"BACKEND": "config.settings.local.CustomS3Boto3Storage"},
    "backup": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
        "OPTIONS": {
            "location": ROOT_DIR.path("backups").path("media"),
        },
    },
}

If you want to replace django.contrib.staticfiles.storage.StaticFilesStorage with whitenoise for production, it's sufficient overwrite the backend in the production settings like this:

STORAGES["staticfiles"]["BACKEND"] = "whitenoise.storage.CompressedManifestStaticFilesStorage"