Posts about python

Karl Knechtel

Pip 25.0 has been out for a bit over a month now; and we now also have an official blog post about the release, as well as a 25.0.1 patch for a regression.

Pip 25.0 has what I consider a very serious security vulnerability. In the Python ecosystem, it's normal and expected that third-party packages provide their own, arbitrary "setup" code for installation (for example, to run C compilers in project-specific ways, when the code uses a C extension). But Pip will run such code in many more situations than you might naively expect. I think it's obvious that running arbitrary code when you aren't expecting it and prepared for it is a much bigger problem. The user should have a chance to decide whether to trust the code, first.

I believe that warnings are more important than baiting people to read the post, so here's the PSA up front:

Never use Pip to download, test, "dry-run" etc. an untrusted source distribution (sdist). It will try to build the package, potentially running arbitrary code (as building an sdist always entails). Instead, use the PyPI website directly, or the API it provides.
Never use sudo to run Pip (nor run it with administrative privileges on Windows). Aside from the potential problems caused by conflicting with the system package manager, Pip will not drop privileges when it runs as root and attempts to build an sdist - which again, potentially runs arbitrary code.
If you expect wheels to be available for the packages you want to install with Pip, strongly consider adding --only-binary=:all: to the Pip command to ensure that only wheels are used. If you really need to use sdists, it's wise to inspect them first, which by definition isn't possible with a fully automated installation.
If you release Python packages, please try to provide wheels for them, even if - no, especially if your package includes only Python code and doesn't require explicitly "compiling" anything. An sdist is much slower to install than a wheel even in these cases, and making a wheel available allows your users to demand wheels from Pip - raising the overall baseline for trust and safety in the Python ecosystem.

Okay, I did clickbait a bit. This security issue isn't some new discovery. In fact, it has plagued Pip for its entire history.

Please enjoy my detailed analysis below.

A Brief Annotation

I've been quite busy working on both the next article in my packaging series and on the overall appearance of the blog (I wasn't able to keep that confined to the weekend, apparently).

So, today, just a quick note, on the occasion of the 4th anniversary of the creation of PEP 649 "Deferred Evaluation Of Annotations Using Descriptors".

Yes, that's a mouthful, but in short: starting in Python 3.14, if you use annotations, you'll be able to defer the evaluation of the annotation code. (The feature was supposed to be added for 3.13, but didn't make it in.) That means you don't have to rely on strings for forward references in your type annotations, but you can still make full use of annotations at runtime (you'll have a proper object for the annotation itself, rather than just a string). Thank you to core Python developer Mr. Larry Hastings for putting a tremendous amount of effort into refining this proposal.

Now, I don't personally use type annotations very much - I don't use a type checker at all; I only write annotations as a form of documentation. But as it happened, when I was new to the Python Discourse forum, I came across Mr. Hastings' post, puzzling over the best way to name what will soon become the __annotate__ attribute of annotated objects.

The name __annotate__ was my suggestion.

I was not credited in the PEP, and I was ignored when I tried to point this out. So it falls to me to draw attention to my contribution.

This is all very niche stuff, of course - but I'm happy to have been able to leave this mark on the Python language itself, as opposed to just making useful things with it.

Karl Knechtel

January 7, 2025

Comments

Python Packaging: Why we can't have nice things
Part 2: Stupid Pipx Tricks

Pip has a lot of problems (that I'll be discussing in future posts in this series), but the good news is that you don't have to resort to heavyweight third-party tools to improve your experience with Python packaging. Pipx (now under the Python Packaging Authority (PyPA) umbrella) is a focused wrapper around Pip that handles the major pain points without trying to take over your entire workflow.

In this post I'll talk about Pipx's major use cases, its limitations, and how to get more mileage out of it with a few simple tweaks.

New year, new blog

You're not imagining things - the blog has a whole new look, in large part as a result of switching to the Nikola site generator.

`fixup! added list`
The rest of the TODOwl

Happy new year to all.

Today's post is about a folder on my desktop named dev. It's where I've kept (for many years, well into my Windows-using days, even into the era when I used SVN rather than Git) all my working copies for my own projects (and forks of others'), mostly Python code of course. (I'm not sure how I organized things at the time, but there are projects in there dating back to 2006.)