Weeknotes 2021-06-28
,Interesting Articles / Podcast Episodes
- Great article about subclassing in python
- How to do documentation
- Entertaining youtube channel about technology
Serving Files With Django - DjangoCon Europe 2021 Talk
,Update 2021-07-31 - the talk was uploaded to YouTube including the Q&A-session:
DjangoCon 2021 | Serving Files with Django | Jochen Wersdörfer
Usually it's recommended to use a dedicated webserver like nginx or a CDN to serve static / user uploaded files when using Django. But after it's now (august 2020) possible to use async / await syntax in Django views, I thought it might be time to revisit this recommendation. It now should be possible to write views that are not blocking a complete worker process for the entire response serving time.
The code for returning async file responses is available via this django_fileresponse package. The process of building this package was also documented on my twitch stream (youtube playlist).
Weeknotes 2021-06-21
,For django_fileresponse I used nbdev because I liked the concept and you get a lot of infrastructure (python package, documentation) for free. But having a package with name that differs from the path from where you want to import code (django_fileresponse vs from fileresponse import x) breaks nbdev_build_docs so I had to replace this command line utility which patches the settings.ini on the fly. It was probably more effort to adapt nbdev to this than using a proper django package template, but now it works :).
Created an django example project whithin django_fileresponse and learned a little bit more about magic commands in jupyter notebooks. Serving files using gunicorn with uvicorn is working now. Had to modify the Django ASGIHandler to make it possible. I'm still not sure what's the best way to swap the default ASGIHandler with a modified version from a projects perspective (being explicit, monkeypatching..).
Serving files asynchronously from MinIO is now also possible and most of the streaming data to the client related code is shared with the class that serves files asynchronously from the filesystem.
Worked on my streaming setup too and tried to bring some structure into the mess of cables behind my desk. But it didn't improve much. Felt more like an expedition into the cable dungeon. Happy to be back alive. In preparation for the next Python Podcast episode I tried to watch most of the 2021 DjangoConEu Talks.
Interesing Articles / Podcast Episodes
Django 3.1 Async
,
With version 3.1, you can finally use asynchronous views, middlewares and tests in Django. Support for async database queries will follow later. You don’t have to change anything if you don’t want to use those new async features . All of your existing synchronous code will run without modification in Django 3.1.
Async support for Django is on it’s way for quite some time now. Since version 3.0 there’s support for ASGI included. But there was not much benefit for end users though. The only thing you could do concurrently were file uploads, since uploads don’t reach the view layer which was not async capable in Django 3.0.
When do you might want to use those new features? If you are building applications that have to deal with a high number of tasks simultaneously. Here are some examples:
- Chat services like Slack
- Gateway APIs / Proxy Services
- Games, especially MMOs like Eve Online
- Applications using Phoenix Liveview - check out Phoenix Phrenzy results for additional examples
- A reactive version of Django Admin where model changes are shown interactively
- A new api frontend for Django REST framework updating list endpoints interactively as new data comes in
- All kinds of dashboard applications showing currently active connections, requests per second updating in realtime
As Tom Christie explained in his talk Sketching out a Django redesign held at DjangoCon 2019 the core question is this: Do we want to have to switch languages to support those use cases? And while his Starlette project (gaining popularity recently in combination with the FastAPI framework) is allowing us to do all this in Python, we also might want to keep using Django.
What to Expect from this Article?
- A small example on how to use async views, middlewares and tests
- Why is async such a big deal anyway?
- The gory details of multithreading vs async, GIL and other oddities
Estimated read time: 25 minutes
There's also a podcast episode elaborating a little bit more on this topic (it's in german).
Scatter plots with density quartiles with python
,I liked the idea, but wondered whether a simple kernel density plot would have the same effect. And if not, maybe how difficult it will be to adapt the approach to python. So I created this little jupyter notebook: