My thoughts on Python packaging

Karl Knechtel

Today I'm offering a sort of "side story" to my main series on Python packaging. The main thrust of the series has been that everything is broken or historically has been broken; but I've also been trying to fight some common misconceptions and defend some things that people don't seem to like but which are actually quite reasonable in context.

But I've been doing this in the shadow of uv existing, and uv's momentum has been unstoppable this year. Of course there were many adopters in 2024 as well, but we're now seeing more and more evidence in PyPI stats, surveys, CI pipelines etc. that people are switching away from pip (and other tools) to uv.

Which is unsurprising, given how positive the coverage has been overall. There have been many popular posts about it on Hacker News this year (I wouldn't mind adding to the pile!) and everyone is touting its features and praising their experience with it.

So before the year is out, I really wanted to write down some thoughts about uv's success so far and the impact it's had on me — which as it turns out is mostly not about actually using it. This will mostly be gathering things that I've already said repeatedly in the aforementioned threads, but I think there's value to that.

Python packaging: Why we can't have nice things
Part 3: Premature Compilation

Pip 25.0 has been out for a bit over a month now; and we now also have an official blog post about the release, as well as a 25.0.1 patch for a regression.

Pip 25.0 has what I consider a very serious security vulnerability. In the Python ecosystem, it's normal and expected that third-party packages provide their own, arbitrary "setup" code for installation (for example, to run C compilers in project-specific ways, when the code uses a C extension). But Pip will run such code in many more situations than you might naively expect. I think it's obvious that running arbitrary code when you aren't expecting it and prepared for it is a much bigger problem. The user should have a chance to decide whether to trust the code, first.

I believe that warnings are more important than baiting people to read the post, so here's the PSA up front:

Never use Pip to download, test, "dry-run" etc. an untrusted source distribution (sdist). It will try to build the package, potentially running arbitrary code (as building an sdist always entails). Instead, use the PyPI website directly, or the API it provides.
Never use sudo to run Pip (nor run it with administrative privileges on Windows). Aside from the potential problems caused by conflicting with the system package manager, Pip will not drop privileges when it runs as root and attempts to build an sdist - which again, potentially runs arbitrary code.
If you expect wheels to be available for the packages you want to install with Pip, strongly consider adding --only-binary=:all: to the Pip command to ensure that only wheels are used. If you really need to use sdists, it's wise to inspect them first, which by definition isn't possible with a fully automated installation.
If you release Python packages, please try to provide wheels for them, even if - no, especially if your package includes only Python code and doesn't require explicitly "compiling" anything. An sdist is much slower to install than a wheel even in these cases, and making a wheel available allows your users to demand wheels from Pip - raising the overall baseline for trust and safety in the Python ecosystem.

Okay, I did clickbait a bit. This security issue isn't some new discovery. In fact, it has plagued Pip for its entire history.

Please enjoy my detailed analysis below.

Python Packaging: Why we can't have nice things
Part 2: Stupid Pipx Tricks

Pip has a lot of problems (that I'll be discussing in future posts in this series), but the good news is that you don't have to resort to heavyweight third-party tools to improve your experience with Python packaging. Pipx (now under the Python Packaging Authority (PyPA) umbrella) is a focused wrapper around Pip that handles the major pain points without trying to take over your entire workflow.

In this post I'll talk about Pipx's major use cases, its limitations, and how to get more mileage out of it with a few simple tweaks.

Python packaging: Why we can't have nice things
Part 1: The Old Refrain

This post is a start of a series I've planned about how packaging currently works in Python, what's wrong with it, and how to cope with the problems. But before I get into the meat of it, I want to talk about common complaints that don't resonate with me.