Homebrewed python3 causing segfaults

(Brandon Kuczenski) #1

Hello, new OSX and homebrew user here- I am trying to setup python / pip / virtualenv on Mac OS the way I’ve always used it for linux. Various things I read on the Internet suggested that homebrew was the best way to get up-to-date python on mac os. We followed the instructions in this webpage:

Then we tried to run some code that works on my linux machine. When it runs on mac, it causes a “segmentation fault: 11”. I’ve done some detective work and it turns out that this happens in the python XML package, lxml.

From web searches, I’ve seen numerous reports of segfaults in python on mac, and they are all caused by issues with static linking of libraries at build-time, leading to conflicts . e.g.:


The solutions all seem to be specific to the linked problems, though, and I don’t know how to avoid the problem in my case. It sounds like something that most people don’t run into. Being a very novice developer on Mac OS, I don’t know how to proceed- can anyone here recommend a solution?

(Sean Molenaar) #2

So, are you running your code in pyenv or on the python that you installed with homebrew?

(Brandon Kuczenski) #3

With the homebrew python, verified first because it’s python3, and second by checking which python. Then we installed and configured virtualenvwrapper and used mkvirtualenv to create and activate a virtual environment - again with python3.

Once within the virtual environment we used pip and python as normal to install needed packages. Then we ran the code and got the segfault. It doesn’t happen immediately, but it is repeatable, and the stack trace points to code within lxml.

One course of action would be to file a bug with lxml, but I know the problem doesn’t happen on any of my linux computers.

(Sean Molenaar) #4

Do you have some sample code to reproduce this?
And what do brew doctor and brew config say?

(Brandon Kuczenski) #5

Thanks for your queries and your patience… Some updates:

  • Did some cleanup in /usr/local and now brew doctor says only:

    Warning: A newer Command Line Tools release is available.
    Update them from Software Update in the App Store.

    Error: No such file or directory @ dir_s_mkdir - /Users/Kyle/Library/Caches/Homebrew/.d20181105-5046-1bcnd1x

  • brew config:

    HOMEBREW_VERSION: 1.7.7-23-g17154ab
    ORIGIN: https://github.com/Homebrew/brew
    HEAD: 17154abf466f35f781fc977baf4908f1aaf194f8
    Last commit: 4 weeks ago
    Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
    Core tap HEAD: e301e64db4be327da4e220916361ab5ed6eb5863
    Core tap last commit: 3 weeks ago
    HOMEBREW_PREFIX: /usr/local
    CPU: dual-core 64-bit penryn
    Homebrew Ruby: 2.3.7 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/bin/ruby
    Clang: 9.1 build 902
    Git: 2.19.1 => /usr/local/bin/git
    Curl: 7.54.0 => /usr/bin/curl
    Java: 1.8.0_73
    macOS: 10.13.6-x86_64
    Xcode: N/A

  • I tried to reduce the crash to a minimal example, but it is only reproducible when I import my main package, even though I am not using any of the functions of that package to reproduce the bug. I created a stripped-down “fake” version and when I import that instead, the crash does not occur.

  • The code that causes the segfault is in the function render_text_block in this file: https://github.com/bkuczenski/lca-tools/blob/master/antelope_catalog/providers/xml_widgets.py

(Sean Molenaar) #6

Without code to reproduce it I’m going to have to assume the bug is in your code, or in one of the libraries you use. There isn’t really an indication that homebrew is at fault here other than that it does work somewhere else.