Ephes Blog

Recorded and produced a new Podcast Episode about Ansible. Had to learn that streaming 48Khz 24bit FLAC audio to AirPods is too much for the Bluetooth bandwidth. At first, I thought the recording was broken, but then realized it was just a Bluetooth glitch, phew.

Did some preparation for a Python training course next week (setting up a JupyterHub instance, preparing a HedgeDoc installation).  Also, normal work as usual until we got evacuated because of a bomb disposal (a 500kg bomb from the second world war was found a few hundred meters down the street).

Progress on having a landing page where people can create podcasts/blogs:

• Did some infrastructure work to be able to use a Python installation via pyenv on production
• ...

Articles

• Falling for Kubernetes | I tend to disagree, but maybe sometime someone will prove me wrong, so I have to keep listening
• How SQLite Helps You Do ACID
• “Autistic people can't acknowledge when they are wrong” | Very familiar vibe 😬
• Putting SOLID into Perspective | SOLID, CUPID, etc. - I don't know. It's not possible to give clear advice on how things should be done. But it could be helpful to know how things tend to go wrong, just to be able to recognize them when it happens 🤓.

Courses

• Databases - Stanford School of Engineering

Twitter

• One of my all-time favourites. Stare at the red dot on the woman's nose for 30 seconds, then look at an empty wall while blinking quickly.
• wife: "we should babyproof the—" me: "there are already other orgs handling baby safety, here we mainly focus on baby capabilities research.
• #AgileMaxim 1: It is in the doing of the work that we discover the work that we must do. Doing exposes reality. | Good thread.

Software

• HedgeDoc | write and share markdown
• Excalidraw | Virtual whiteboard for sketching hand-drawn like diagrams. Collaborative and end-to-end encrypted.
• Donut on Hugging Face / Donut on GitHub | I've done things like this with tesseract getting much better results. But I'm really happy to see people starting to use transformers for problems like this. Very promising.

Podcasts

• WR1392 Strukturwandel ist teuer (WRINT Wirtschaftskunde) | Instead of raising the base rate (making investments to lower gas comsumption more difficult) or artificially lowering the gas price (providing an incentive to consume more gas) the rational economic way to handle a gas shortage would be to tax gas consumption. Of course this is not possible politically, but..
• #309 – John Carmack: Doom, Quake, VR, AGI, Programming, Video Games, and Rockets (Lex Fridman Podcast) | Good episode
• Pydantic v2 - The Plan (Talk Python To Me) | pydantic-core is completely written in rust, leading to smaller binaries compared to cython
• WR1390 China (Wer redet ist nicht tot) | Interesting, media coverage about Chinas social scoring is mainly creating a myth, etc..
• Jochen Distelmeyer (Reflektor) | Trying to listen to more music instead of podcasts - maybe I can start by just listening to podcasts about music, no?

Out of Context Images

Back to work. Picked up my jogging habit after having to stop during covid / vacation and it worked out fine. Made plans to get more into music again (atm just got more hardware).

Progress on having a landing page where people can create podcasts/blogs:

• #32 fixed by only having one place to turn on the test client
• #33 moved registry app urls into urls.py of registry app
• #34 tests for account and social account adapters only open for registration with appropriate setting present
• #35 test coverage is now at 100%
• #36 fixed some mypy issues
• #37 replace poetry with pip-tools
• #38 fixed showing multiple running deployments at the same time for tests
• #39 tried to speed up ansbile (mostly without success)
• #40 handling of exceptions raised during starting a deployment

Articles

• Heuristics for Effective Software Development Organizations: A continuously evolving list.*
• Why i love still php and javascript after 20+ years | I know I have to work on my attitude, but for me programming is a lot of fun and I couldn't care less whether the result is useful for anybody. Sometimes I have to rewatch DjangoCon 2021 | Programming for pleasure | Daniele Procida to remind me I have to work on this.

Videos

• Basic dot Py | Demonstration of some features of textual
• Agile & Scrum Don't Work | Allen Holub In The Engineering Room Ep. 9 | Learned a new acronym: SAFE stands for shitty agile for enterprises 😁

Twitter

• Interesting twitter thread on the usage of the enum module
• Python 3.10's match/case can use custom __eq__ for matching. Here's how to use regex cases: Many more good tips at @raymondh 's
• Took myself on a Saturday Night date and blew my mind with Mersenne Primes. A good evening all round 💃
• W.I.P. collage tool for stable diffusion

Software

• Django 4.1 released
• pytest-mock | Finally getting rid of those ugly stacked decorators or nested with-statements 🥳

Podcasts

• Rachel Potvin // VP Engineering at GitHub - Episode #55 (alphalist)
• Gebrochene Fußnoten ...von Umbrüchen, Wandlungen und plötzlichen Lebensereignissen (Was denkst du denn?)
• Catherine Opie (Design Matters)
• The Benefits Of Python And Django For Going From Zero To MVP At Speed - Episode 373 (Podcast.__init__)
• Django 4.1 Preview - Jeff Triplett (Ep 115 Replay) | Good overview over the Django 4.1 features - release was last week.

Out of Context Images

For the last two weeks, I was on vacation, so no work. Seems my job is to listen to podcasts because there were significantly fewer episodes I listened to. Need to check with my boss 😄.

Had to learn the hard way that shared albums in apple photos are completely broken and will cripple your images (max width or height set to 2048px + conversion from heic to low quality jpeg). And even if you stop following a shared album, those broken images still sit in your library and you have to remove them manually one by one. Great fun if you just imported a few hundred photos from a shared album. I don't understand tech companies'  obsession with destroying their user's photos. Signal is bad, Whatsapp is worse, and now even you Apple? A working method for Apple devices is to generate an iCloud share link (which takes lots of time) from which others then can import the original photos.

Progress on having a landing page where people can create podcasts/blogs:

• Finished Add Email Backend for Registration Mails. This feature uses mailgun which is a pretty crappy service, if you've found a better one, please let me know.
• Add WordPress Ansible Service #28 moved the ansible folder over from podlove_test
• Cleaned up the register link and removed some unused code #30
• Use always _base.html in favor of base.html #31
• Added styled allauth templates #27

Articles

• Weeknotes: Joining the board of the Python Software Foundation | I Hope Johannes picks up his weeklog again so I'll be able to link his instead 😏.

Videos

• What Is a Reasonable Percentage for Code Coverage and Why Is It 100%? [Jürgen Gmach - PyOhio 2022]
• What's next for Vue? Evan You explains... | I've given up on following vue for quite some time now and even stepped back as an organiser of Vue.js Cologne. I've done one big vue project and lots of small htmx / Django projects in the last year and I feel much more productive using the latter. Learning Vue was always a hedge against being forced to use Javascript for web development, but the risk lowered substantially over the last year. But occasionally I still like to look at Vue content 😀.

Twitter

• Shirt: Today Satan
• Shirt: Jar Jar Binks goes Patrick Bateman

Software

• flake8 5.0.0 release 🍾

Podcasts

• Oral History - Die Geschichte des Heavy Metal (Hörsaal) 
• Sebastian Raschka: AI Education and Research (The Gradient Podcast) | Good episode, it has been quite some time I heard someone talk about ordinal regression (used it for learning to rank more than a decade ago).
• Python Language Summit 2022 (Talk Python To Me)

Out of Context Images

Found a workaround for the fastdeploy / SQLAlchemy / asyncpg / Linux deadlock bug. I now instantiate a new SQLAlchemy engine on every request. That's probably not very efficient, since it would be better to use the engine's built-in ability to hold a connection pool, but having working code is a good start.

Progress on having a landing page where people can create podcasts/blogs:

• Finished Add remove domain / deployed fqdn #14
• Finished Switch between cast / wordpress #2 for deployments 🍾 - it was a complete surprise for me that this is even possible. But since I have now a working deployment system where all responsibilities are nicely separated, it's possible no not only to deploy Django, but anything including weird stuff like Wordpress without having to worry the rest of my system might suffer.
• Finished Fix pytest warnings + some cleanup stuff and coverage

Articles

• An SPA Alternative
• How Did REST Come To Mean The Opposite of REST?
• How To Test Django Models (With Examples)
• Building the blog while flying it Series' Articles

Videos

• Two Sigma Presents Pandas at a Crossroads the Past Present and Future with Jeff Reback

Twitter

• In Python, partials are an easy way to reuse function code. 🐍 🤩
• Don't move elements in a list while looping over it! You will get surprising (wrong!) results: 😳 🤨
• Since #Python 3.7, modules can define __getattr__ to provide dynamic attributes on modules. Why use two-argument arithmetic when you can do it with one!? 😎 🥳 😍

Software

• django-readers | A lightweight function-oriented toolkit for better organisation of business logic and efficient selection and projection of data in Django projects.
• django-sesame has a new tutorial | I'm searching for a solution to be able to log in users for a newly deployed site without having to store a password. This could be one.

Podcasts

• UKW100 Brexit: Kopf unter der Decke (Unsere kleine Welt) | Yup, brexit is still a trainwreck.

Out of Context Images

Fought with strange bugs last week, also some work and much outdoor stuff.

For example when using Jupyter Notebooks in a Django environment, the PYTHONPATH was always set to some old project and therefore imports from my current project didn't work. And it was simply not possible to get the PYTHONPATH right. I could not find the old projects name in any config or virtualenv. Weird. But then I had a closer look at the kernels using jupyter kernelspec list:
 

$ jupyter kernelspec list Available kernels: django_extensions /Users/jochen/Library/Jupyter/kernels/django_extensions python3 /Users/jochen/.virtualenvs/foo/share/jupyter/kernels/python3

Ok ~/Library/Jupyter/kernels is weird. And there the name of the different project appeared in the kernel.json. Whoa. Somehow this old django_extensions kernel has overwritten my django_extensions kernel from the virtualenv. After deleting the old one, my kernelspec looked like this:

$ jupyter kernelspec list Available kernels: django_extensions /Users/jochen/.virtualenvs/foo/share/jupyter/kernels/django_extensions python3 /Users/jochen/.virtualenvs/foo/share/jupyter/kernels/python3

This was pretty hard to debug.

And then there was this strange fastdeploy bug forcing me to have a closer look at SQLAlchemy. I still don't know what the root cause of the problem is. I switched to using SQLAlchemy in combination with asyncpg a frew months ago and there were this "The garbage collector is trying to clean up connection <asyncpg connection>" in the logs right from the beginning. I didn't pay attention, because it all seemed to work fine. But during my deployment tests with cast_registry I finally saw some real errors. The tests caused a lot (well, not really) of read queries to the database while the deployment process is posting steps to fastdeploy causing writes. And now I saw 401 responses to some requests from cast_registry, because the access token could not be verified or 404 because some object was not found. Looking at the logs I found some messages from asyncpg like "asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress". Hmm, seems like two coroutines are trying to use the same database connection which should not happen. After some debugging I found out the reason for this was me using "uow=SqlAlchemyUnitOfWork()" holding the database connection as a default argument in a bootstrap function, which causes uow to be instanciated at import time and never be replaced by a new uow, ouch. But after fixing that, fastdeploy is now ending up in a deadlock after some time. But only on Linux.

And while I'm sure it's all my fault and I'm just holding SQLAlchemy wrong and being and early adopter using it in combination with asyncpg doesn't help - I just cannot wrap my head around the SQLAlchemy interface. Why are there engines, connections and sessions? When do I have to instanciate/open/close them?  Why do I have to close a connection, but dispose an engine? I often close a session, then a connection and then dispose the engine. If you should always use this combination, why isn't there a highlevel function doing just that? If you don't, why is there no documentation showing why and how? 😤

Progress on having a landing page where people can create podcasts/blogs:

And then I started an experiment trying to help someone getting started with Django and writing a series of blogposts about it.

• Merged the old work in progress on the "Deployed Services Configuration"-issue to be able to work on this database bug without conflicts getting out of hand 🫣

Out of Context Images

Articles

• Pydantic V2 Plan
• Interactively compare Hubble and Webb images
• Forms in Django 4.0+

Videos

• The Amplituhedron - Nima Arkani-Hamed | Hmm, didn't know about this one..

Twitter

• With the newest version of Git 2.37.0, you can run just "git push" to push new branches. No more "--set-upstream origin".
• ChainMap is in collections in the #Python stdlib: it's a list of dictionaries, searched in order for items. Handy for nested contexts like programming language scopes. Or pizza specials! 🍕🥤😎🍕
• I was recently looking for materials to learn causal inference. These lectures given by @david_sontag are really amazing.
• A Python pattern that I like to use is attaching exceptions to callables that use them. To me this is syntactical sugar on larger projects. #twitter4devs
• This is probably well-known in some circles but not everywhere. The most important skill for Research Scientists in AI (at least at @OpenAI ) is software engineering. Background in ML research is sometimes useful, but you can usually get away with a few landmark paper. | Software engenieering is the most useful skill in a lot of fields right now..

Software

• Community: Component Libraries | List of web component libraries for date pickers etc..
• htmx 1.8.0 Release
• Typer Release 0.6.1 | Integration with rich

Podcasts

• #799 (Fünfphasig) (Bits und so)
• #293 – Donald Hoffman: Reality is an Illusion – How Evolution Hid the Truth (Lex Fridman Podcast)
• WR1384 Zeitenwende (Zum Thema) | Great episode, would like to hear a lot more stuff like this
• Kommunismus - Ein schillerndes Gespenst (Sein und Streit) | Interesting topic, but the episode was rather boring (at least for me)
• Chip Kidd—award-winning designer, musician, author, and all around rock star—joins for his fifth time to talk about his recent projects and so much more (Design Matters)