Installing python module with formula

Hi, I’m working on a brew formula for my groups project here: https://github.com/bioinfologics/sdg

It is a C++ project that provides a Cpp library, some binaries, and also provides a python module, wrapping the C++ library. The make install process tries to install the python wrapper module to the site-packages folder of the python 3 used - cmake figures out where site-packagesis, and it successfully finds /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/.

The standard build and install process without brew is:

git pull https://github.com/bioinfologics/sdg
cd sdg
mkdir build
cd build
cmake -DBUILD_PYTHON_INTERFACE=ON -DCMAKE_C_COMPILER=gcc-7 -DCMAKE_CXX_COMPILER=g++-7 ./
make
make install

Which works for me with OSX and python 3 provided by brew.

So I made a homebrew formula to make this trivial for people to do, but it fails when it gets to the install step:

==> make install
[  1%] Built target pysdg_swig_compilation
[  2%] Built target CatchMain
[ 70%] Built target sdg
[ 77%] Built target sdg_static
[ 82%] Built target sdg-dbg
[ 87%] Built target sdg-mapper
[ 87%] Built target sdg-kmercounter
[ 87%] Built target sdg-datastore
[ 93%] Built target pysdg
[ 96%] Built target sdg-workspace
[100%] Built target runBasicTests
Install the project...
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:44 (file):
  file cannot create directory:
  /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pysdg.
  Maybe need administrative privileges.


make: *** [install] Error 1

==> Formula
Path: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/sdg.rb
==> Configuration
HOMEBREW_VERSION: 2.1.8-38-g3dab112
ORIGIN: https://github.com/Homebrew/brew
HEAD: 3dab11263c7d3ae568bd585932b393ef526fc5b2
Last commit: 25 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 84988df6fcf76a1203dd245d6dba76afde9b1b1d
Core tap last commit: 29 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
CPU: octa-core 64-bit skylake
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1001
Git: 2.22.0 => /usr/local/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 9.0.1, 1.8.0_152
macOS: 10.14.5-x86_64
CLT: 10.2.1.0.1.1554506761
Xcode: 10.3
XQuartz: 2.7.11 => /opt/X11
==> ENV
HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
MAKEFLAGS: -j8
CMAKE_PREFIX_PATH: /usr/local/opt/openssl:/usr/local/opt/readline:/usr/local/opt/sqlite:/usr/local
CMAKE_INCLUDE_PATH: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/libxml2:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
CMAKE_FRAMEWORK_PATH: /usr/local/opt/python/Frameworks
PKG_CONFIG_PATH: /usr/local/opt/openssl/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig:/usr/local/opt/sqlite/lib/pkgconfig:/usr/local/opt/xz/lib/pkgconfig:/usr/local/opt/python/lib/pkgconfig:/usr/local/opt/isl/lib/pkgconfig:/usr/local/opt/mpfr/lib/pkgconfig:/usr/local/opt/mysql/lib/pkgconfig:/usr/local/opt/pcre/lib/pkgconfig
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.14
HOMEBREW_GIT: git
HOMEBREW_SDKROOT: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
ACLOCAL_PATH: /usr/local/share/aclocal
PATH: /usr/local/Homebrew/Library/Homebrew/shims/mac/super:/usr/local/opt/cmake/bin:/usr/local/opt/gdbm/bin:/usr/local/opt/openssl/bin:/usr/local/opt/sqlite/bin:/usr/local/opt/xz/bin:/usr/local/opt/python/bin:/usr/local/opt/gcc@7/bin:/usr/local/opt/mysql/bin:/usr/local/opt/sphinx/bin:/usr/local/opt/pcre/bin:/usr/local/opt/swig/bin:/usr/bin:/bin:/usr/sbin:/sbin

Error: sdg 1.0_rc3 did not build
Logs:
     /Users/bward/Library/Logs/Homebrew/sdg/00.options.out
     /Users/bward/Library/Logs/Homebrew/sdg/01.cmake.cc
     /Users/bward/Library/Logs/Homebrew/sdg/01.cmake
     /Users/bward/Library/Logs/Homebrew/sdg/02.make.cc
     /Users/bward/Library/Logs/Homebrew/sdg/CMakeCache.txt
     /Users/bward/Library/Logs/Homebrew/sdg/03.make
     /Users/bward/Library/Logs/Homebrew/sdg/02.make
READ THIS: https://docs.brew.sh/Troubleshooting

It seems that when I run make install, everything’s fine, but when it occurs when the formula runs, some different permissions are used or some such thing.

This is my first stab and making a homebrew formula, is doing sudo make install fine (I assume probably not)? If not, what should I do instead?

Why no sudo: https://docs.brew.sh/FAQ#why-does-homebrew-say-sudo-is-bad

https://docs.brew.sh/Python-for-Formula-Authors for more info about python.

For some debugging info; /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ doesn’t exist on my machine so I think it might not be as correct as you think. (maybe you sudo pip installed something?)

Thanks!

That path on my system is actually a symbolic link, to /usr/local/lib/python3.7/site-packages.

Rather than have CMake and make install try to copy the python module over. I’m thinking I should give the module a setup.py, and then once make has built the module with swig and so on. I should do something like either pip3 install . and let the my brewed python3 distribution my formula depends on install it where it needs.

Did you try a python virtual environment to make/build within? You could give ‘venv’ a shot which is included with Python 3.3+. Another option is ‘virtualenv’. These would let you manage packages and modules which aren’t a part of the library. Plus, you can always wipe the environment and start over with much greater ease than using your system or brew installed python directories. You wouldn’t need to do symlinks either for the virtual environment. Here’s some documentation which may be useful.

https://docs.python.org/3/tutorial/venv.html