Ephes Blog

i still think my favourite thing that's ever happened to me on the internet is the time a guy said "people change their minds when you show them facts" and I said "actually studies show that's not true" and linked TWO sources and he said "yeah well I still think it works" --Catherine na Nollag

Our washer/dryer broke down. My wife tried to repair it, but it turned out to be completely dead. During our repair attempts, we had to shut off the water supply to the kitchen, which basically made the kitchen unusable too. On the plus side, I did manage to get some productive doomscrolling done while anxiously waiting to see if the new machine would drain properly. Word of advice: try not to deal with this kind of hassle right before Christmas.

Articles

• AI and ML
  • Gemini 2.0 Flash: An outstanding multi-modal LLM with a sci-fi streaming mode
  • AI’s Transformative Impact On Web Design: Supercharging Productivity Across The Industry | Pretty upbeat article with lots of sources
• Frontend
  • Web Almanac 2024 published
  • The Lowdown on Dropdowns in HTML & CSS | How to do standard UI stuff without Javascript
  • View Transitions API: Single Page Apps Without a Framework
  • New CSS that can actually be used in 2024
  • Your CSS reset should be layered
• 3 shell scripts to improve your writing, or "My Ph.D. advisor rewrote himself in bash | good advice

Fediverse

• Doing a (brief) presentation Monday on long term software development (>10 years). Things you regret in year 8. I'd love to hear things you found out and wished you'd done differently. Or conversely, what worked out really well. Will turn this into a public presentation & blog post. To kick it off: PLEASE KEEP YOUR CODE AS SIMPLE AS POSSIBLE. | Good 🧵
• OpenAI Sora is out | Runway works in EU

Stuff

• The Holotypic Occlupanid Research Group : A Database of Synthetic Taxonomy
• Fair fashion basics
• Introducing Limbo: A complete rewrite of SQLite in Rust | Interesting
• An official logo for CSS | CSS has a new logo

Out of Context Images

The thing most people don't remember about Prometheus is that humans already had fire, but then Zeus locked it up behind a paywall, because Zeus was a petty little bitch. Prometheus stole it *back*. In this essay about DRM I will --jwz

I was just going through my weeknotes from last year and decided to take it a bit easier on myself with the weekly updates 😇. Not much else has changed though - I'm still juggling too many things at once. Despite being busy, I did manage to write a TIL post about including extra data in Python packages, and I published a quick hot take on why I think Big Data might be on its way out (which actually started as a comment in a discussion that I had Claude help me reshape into a blog post). Still working on the latest podcast episode, but on a brighter note, we got to take a lovely trip to Maastricht.

Stuff

• Constraints are Good: Python's Metadata Dilemma | Yep, but at least uv seems to be a step in the right direction
• Web apps have client and server state (plus realtime and LiveView) | Maybe you can skip those complexities by skipping realtime and complex client state for most use cases? 😇
• dear santa, I'm writing to tell you I've been naughty and it was worth it. | webcomic
• Python 3.13.1 Release
• Subclassing, Composition, Python, and You — Hynek Schlawack | Very good talk

Out of Context Images

Modern laptops are significantly more powerful than many realize, often capable of handling data processing tasks that traditionally were relegated to computing clusters. This perspective often surprises people, which in turn surprises me, because it's not a new development. While certain tasks undeniably require immense computational power, more often than not, we are limited by input/output (I/O) speeds rather than raw processing power. The solid-state drives (SSDs) in today's laptops, such as those in MacBooks, are incredibly fast, making them highly efficient for data-intensive tasks.

Consider a common scenario: you have several hundred gigabytes of Parquet files and need to perform aggregations or simple computations that could be handled with a tool like polars/pandas/numpy. In a cluster environment, these files are typically distributed across multiple machines using a system like Hadoop Distributed File System (HDFS) or stored in S3 buckets. Tools like Impala, Hive, or Spark are then used to execute SQL-like queries, distributing the workload across numerous nodes. However, these nodes often spend a significant amount of time waiting for I/O operations from S3 or HDFS, leading to inefficiencies.

Hannes Mühleisen, the developer of DuckDB, addressed this issue in a talk, explaining why they chose not to create a distributed version of DuckDB. He pointed out that distributing the load across multiple machines rarely offers a performance benefit that justifies the added complexity. In their tests, a single server node running DuckDB could match the performance of a 60-node Spark cluster—a trade-off that hardly seems worthwhile.

This sentiment is echoed by one of the original BigQuery developers in a blog post, stating that most companies believe they have "big data" when, in reality, their data can be efficiently processed on a single machine.

The evolution of big data infrastructure meetings tells the story:

• 2014: "We need a whole rack for this!"
• 2019: "Maybe just one really beefy server?"
• 2024: "Have you tried using your MacBook?"
• Next year: "Just run it on your smart watch, bro"

My favorite quote regarding Big data comes from Gary Bernhardt:

Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000.

The Takeaway

Before investing in complex and costly infrastructure, it's crucial to assess the actual requirements of your data processing tasks. Modern laptops are powerful tools that, when fully utilized, can handle substantial workloads efficiently. By rethinking our approach and leveraging the hardware we already have, we can achieve better performance without unnecessary complexity.

Today, I learned how to include external files in a Python package using uv and the hatchling build backend. My goal was to add a directory containing sample fixture data to the package. Here’s the resulting pyproject.toml file:

This turned out to be surprisingly tricky to get right, and the toml file doesn’t make it immediately clear which paths are the source and which are the target in the sdist and wheel. So, here’s an overview of the directory structure:

When I die please mix my ashes with concrete, then turn me into a brick and throw me at an Amazon Warehouse --Erik Uden

Yep, it’s been a stressful week, just as expected. Somehow, we managed to record a podcast episode, but now I need to find the time to get it published.

Fediverse

• tl;dr: If you do a lot with timezones or other heavy dt-manipulation, you probably want Luxon or similar. Otherwise, plain JS is just fine these days. | Good to know!
• As of Chrome 131 you have more options to style `<details>` and `<summary>`.
• Dasz soltu wōl gestaten, swanne er sînen knappen von der mære herab abcontrefeiet, sintemālen so er im steickbeugel stehet. Solt aber dasz imāgo mit helff von fuci rotoris erzeuget seyn, sō dults nimmer, denn dîselben seind des teuffels.

Software

• Model Context Protocol | Open protocol for LLMs to talk to services with tools and extra context by Anthropic

Videos

• Hidden gems of Django 5.x with Sarah Boyce
• The Rising Sea - Matthew Drury (PyCascades 2024)