Best practice for Homebrew on Travis? (brew update is +5min to build time)

What’s the best practice of using Homebrew on Travis?

Specifically, what to do about the slow brew update? It might work well on a desktop system, but on a CI pipeline it means as soon as you get use a single homebrew package, your build time goes gets the one-time +5 minutes penalty to it - that’s lot when a build itself could just take 2mins.

Anyone have a fix?

Not really sure what you want from it but you could only upgrade the package you need (as it’ll upgrade all it’s dependencies)

Five years ago we (me and my teamworkers) were seriously concerned about 1GB+ source repo being git cloned every time a build started. Do you really think that 2 extra minutes is an overhead?

Also, can you clarify - how do you use brew on CI? Do you test your own formulas on it?

I think OP is referring to the updating/pull of either brew itself (the ruby stuff) and/or the updating of repositories. This does add time, but ensures that the latest versions are being installed. Without this, errors or false successes based on an older version of a package could happen, causing things to fail on a actual system of either a dev or user, say building from source.

Nope, I just get a few libraries I need for my C++ build. Actual installation of them takes half a long, see my builds: https://travis-ci.org/Mudlet/Mudlet/jobs/557225266:

CI/travis.before_install.sh (which is just brew update): 300+s
CI/travis.install.sh: 150s

I want to cut down on my 20min build times and cutting out 5min from it is important!

That all makes sense - but it takes way too long. Is there a way to cache that? I’d be happy to only check for it once a day or so.

I looked around but caching the repository status seems very complicated, to say at least - hoping people can help here.

Caching is possible but it’s part of Travis CI itself and not Homebrew. Homebrew doesn’t take that long to check for regular updates and can be configured to only check once a day but that state won’t resume on Travis which uses the same (old) base image for every build.

That’s right, this makes sense. So what can I cache myself then, what paths?

The .git directories for all repositories under /usr/local/Homebrew.

Having used Travis-CI for a long time to do macOS builds, my advice would be to migrate your builds to Azure Pipelines. I really got tired of constantly seeing my builds fail due to Homebrew packages failing to download. This does not happen on Azure in my experience…

If you would like to see an example: https://github.com/tschoonj/xraylib/blob/master/azure-pipelines.yml or https://github.com/tschoonj/xmimsim/blob/xmi_batch/azure-pipelines.yml

Homebrew packages failing to download is not a problem: you can easily cache the http downloads by caching $HOME/Library/Caches/Homebrew and that works. So that’s a non-issue for us.

Thanks! I take it only .git directories and nothing else in there, right?

Perfect! From 5 minutes down to 2 seconds.

Travis configuration for everyone else reading:

cache:
  directories:
    - $HOME/Library/Caches/Homebrew
    - /usr/local/Homebrew
before_cache:
  - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then brew cleanup; fi
# Credit https://discourse.brew.sh/t/best-practice-for-homebrew-on-travis-brew-update-is-5min-to-build-time/5215/9
# Cache only .git files under "/usr/local/Homebrew" so "brew update" does not take 5min every build
  - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then find /usr/local/Homebrew \! -regex ".+\.git.+" -delete; fi