Python 2 EOL 2020

I think we should drop Python 2 whenever it is EOL (which sounds like January). Ubuntu have the desire/resources to e.g. backport security patches after that date which we do not.

1 Like

I am glad you did that, it helps indirectly for Linuxbrew. And it allowed to create this first list.

One question: how do you know that a specific formula depends on Python 2. If it depends on system Python 2, you will not notice.

And regarding node (and others): the process will probably be painful.

A “mee too” for drop support Brew Python2, and allow packages to risk it with Mac Python2 if they need it.

1 Like

I would go for dropping python2 support as well and removing dependencing formulas. Probably homebrew needed more build resources to test this thoroughly though. Having a kind of Darwin container/jail system would help as well, just remove both homebrew and system python2 inside the jail and build along.

1 Like

It will be funny to see how the Node.js community will react when brew install node fails because even the soon-to-be-released Node 12 requires legacy Python.

1 Like

This is an issue for the node community, but I think we should try to take this into account. We are in April now. We could write on their issue tracker during the summer, telling them that they have 6 months to fix it.

Right now nothing is decided, I want to gather more opinions. The only thing I see here is that most of the people would like to get rid of Python 2 support. Not sure we should discriminate between Brewed Python 2 support and Apple Python 2 support.

I think dropping 2 is going to cause a word of pain. A well behaved shell today has python resolving to 2 and python3 resolving to 3. So many systems depend on that and that’s at the user/cli level. Digging a little deeper there are numerous dependencies sprinkled everywhere. I’m having flashbacks to the attempt to kill perl 4 as 5 became a thing. It never really transitioned, instead the language fragmented. This is, to be fair, a python issue, not a brew issue. I just suspect brew is going to be a lot more usable with ongoing 2 support even as the community attempts (and I predict fails) to EOL it. All this is true for Linux but even more so for Mac OS given the less coupled nature to various open source projects.

1 Like

I think we should take a cue from Apple when the next macOS beta comes out. If they are still shipping python2 it would be nice to allow formulae to use system python2.

One big question I have (mostly due to my incomplete understanding of brewed python packages and the internals of homebrew/brew) is:

Will vendored modules/packages still work with system python2? Will we need to migrate/revision bump formulae with vendored python2 dependencies to reinstall them with system python?

I have a distant memory of the painful situation of figuring out where system python pip installs stuff, and dealing with bad permissions… I’ve been using brewed python for so long that I have no idea whether any of this is still applicable.

2 Likes

Yes. It will be painful. But it should not be painful for the brew maintainers. This is why we are having this discussion right now. It is a Python issue, and how the thing was handled by the Python maintainers/community.

Removing Python 2 from brew will probably break some people’s workflows. But it will not break the underlying OS. At least we are on the safe side here.

We can not take the responsibility to ship a know unmaintained and deprecated library. Especially one as critical as Python, which may live on with unmatched security issues. This is why we need to drop it.

Even if they release the next mac OS with Python 3 this year, we still support the two previous mac OS versions. Apple is 3 years too late for us. And we need to find a solution that fits well for the 3 last versions of mac OS. Not 1 solution for the latest mac OS, and another solution for the 2 previous ones.

It depends on what we choose here. If we say that Apple’s system Python 2 is fine, why not.

I posted a comment on the node issue tracker: https://github.com/nodejs/node-gyp/issues/1337#issuecomment-486087002

Just to let them know of the situation. Because this is one of the biggest packages in brew.

We should take their situation into account, but not block the whole deprecation process only because node is not ready.

I still want to hear some more ideas/opinions before summarising the options we have here.

1 Like

Yes I agree we need one solution. If the next release of macOS still ships with Python2, however, then that is an indication of some level of continued support from apple. So in that case I’m more inclined to allow formulae to have dependencies on system python2.

If, however, they ship with Python3, then I would be inclined to drop all Python2 support at Python2 EOL.

1 Like

From following macOS releases for 10 years my best guesses would be (in descending probability of happening):

  1. macOS 10.15 ships just Python 2
  2. macOS 10.15 ships both Python 2 and Python 3
  3. macOS 10.15 ships Python 3
1 Like

Looks like the discussion has started at openSuse too: https://lists.opensuse.org/opensuse-factory/2019-04/msg00229.html

I just noticed that mercurial 5 has been released with beta support for python3. I’ve made a mercurial-python3 formula in a tap for developers who want to test it (they say 1% of tests fail, so they aren’t ready to distribute it with python3 support):

1 Like

I would just like to point out that, for the record, the Homebrew “python” formula (as in, python.rb) has in fact installed Python 3 for some time now – Homebrew users have been getting Python 3 when doing brew install python for the last year and change, I want to say?

While the python.rb formula has not installed a /usr/local/bin/python symlink to the Python 3 executable, it has installed such a symlink into /usr/local/opt/python/libexec/bin, alongside symlinks to python-config, pip, and other related executables. The caveats method of the Python formula helpfully points this out; personally, I have that libexec directory at the head of my PATH, as I would imagine so do many users of the Python formula.

I still have python@2 installed, though – mainly to satisfy one or another of the listed formulae whose software problematically targets only Python 2. I would agree with @fxcoudert in that triage is the essential first step for dealing with these.

For example, if I scan the list, I know that:

  • numpy.rb, opencv@3.rb and protobuf.rb can trivially drop Python 2;
  • py2cairo.rb, pygobject.rb and qscintilla2.rb specify old variants of packages that have updated formulae already in homebrew-core;
  • supervisor.rb installs a codebase that is probably not going to show Python 3 any love anytime soon, and when it does, it will support being pip installed as it always has;
  • handbrake.rb, ino.rb, and fontforge.rb will likely need upstream patches… possibly with simple changes, maybe with complications – technical and/or social;
  • Some of these, like cvs2svn.rb, are probably due for a general review of their utility and statistically-supported popularity; others, such as ipython@5.rb, could stand to have the utility of their formulae weighed against the versions provided by e.g. PyPI via pip, Anaconda via conda, and other package-management systems whose mission may hew more closely to the mission of the software in question.

I would also cast my vote for a grace period for python@2 – I still occasionally launch its interpreter to examine one or another of its quirky edge-cases, for one reason or another. If it hangs around until 2021, OK sure fine – Python 3 is only going to become more and more excellent during that timeframe, easing the burden of porting the most stubborn of holdouts.

1 Like

N.B. I take it back about fontforge.rb – I just looked over that formula and a) its depends_on clause specifies [:build, :test] (meaning Python isn’t a strict requirement for like 99% percent of FontForge installers, who’ll just use the off-the-shelf bottle) and b) it is already using Python 3: all of the commands specified in the formula explicitly name the python3 executable.

The 10.15 final release may not even have python bundled according to the beta release notes.

macOS 10.15 Beta Release Notes

But, I would think at a minimum Python 3 would be included. I’m all for dropping/updating/replacing any packages dependent upon Python 2.

Regardless of what the release notes may imply this will be my 10th macOS upgrade working on Homebrew and I guarantee you that Python 3 and 2 will ship with macOS 10.15.0.

I wouldn’t know since I don’t typically deal with the betas of the OS. But, it would be nice if they’re going to ship one or both versions that it at least come with more updated versions that aren’t 1-2+ years old. Since they were moving to a new partitioning scheme within APFS, dropping 32-bit support, app notarizations, etc., then it seemed reasonable this may occur. Especially, since these could be installed by any user who requires them and most that require them will know how to install them. I believe there was something mentioned I read somewhere along the lines of, if an app requires these, then they would need to include it in the bundle. However, I can’t recall if that was actual context or I mixed it up with other material. So, you may be correct and have more intimate knowledge about if it is. I can only go off of beta notes. Perhaps, they do but it’s more of a forewarning to them not being included in future releases after 10.15.x.

With https://github.com/Homebrew/homebrew-core/pull/42494, numpy and a few other formulae will move to Python 3 only.

I think we should focus on moving as much as possible to python 3 from the above list in the next months. We will then see what remains at the end.

1 Like