Brew search is slow. Here's a bash function for local-only search

I find brew search to be painfully slow. I know that the network requests are partially to blame, but when I do brew search --debug obscure_thing it takes ~1.5s or so before the debug info shows its first curl request.

99.9% of the time I just want to search my local formulas.

To do this pure local search I ended up rolling my own little bash function. Check it out:

function hbs() {
  bold=$(tput bold)
  normal=$(tput sgr0)
  if [[ ! $1 ]]; then
    echo "Input missing";
    return
  fi
  QUERY=$(echo $1 | tr '[:upper:]' '[:lower:]')

  ## replace that path with `$(brew --repository)/Library/Taps` if you don't mind an extra 30ms
  RESULTS=$(find /usr/local/Homebrew/Library/Taps -iname "*${QUERY}*" -iname "*.rb")
  if [[ ! $RESULTS ]]; then
    echo "No results found";
    return
  fi
  echo "${bold}====> Results:$normal"

  while read line; do
    fname="${line##*/}";
    printf '%-35s %-20s \n' ${fname%.*} $line;
  done <<< "$RESULTS";
  echo -e "\n\c";

  while read line; do
      if [[ "$line" == *"/Casks/"* ]]; then
        cmd_extra="cask";
      else
        cmd_extra="";
      fi
    fname="${line##*/}";
    echo "========${bold} ${fname%.*} $normal========"; brew $cmd_extra info $line; 
  done <<< "$RESULTS";
}

With this I can run hbs python and get the first results in less than half a second! Running brew info on each result of course takes awhile, but usually just the listing is enough.

Would a brew search --local flag be interesting to people? Would the devs support that? :slight_smile:

1 Like

Interesting take on an alternative search algorithm, though I recommend turning it to a proper external command, and leveraging on the standard Homebrew environment variables rather than hardcoding paths: https://docs.brew.sh/External-Commands

I personally have a brew-lsearch script in my PATH that’s simply:

#!/usr/bin/env bash
HOMEBREW_NO_GITHUB_API=1 exec brew search "$@"

so brew lsearch does everything brew search can, just offline.

1 Like

Your algorithm is completely different from standard Homebrew (start digging at /usr/local/Homebrew/Library/Homebrew/cmd/search.rb), so I have my doubts.

If you make a pull request to modify the existing search it would at least be considered.

1 Like

Oh wow @gromgit didn’t realize is was so easy to add an external custom command to homebrew like!

Though you would think that HOMEBREW_NO_GITHUB_API=1 would make brew search faster, but it doesn’t almost nothing for me. Here’s the run time comparison.

$ hyperfine "HOMEBREW_NO_GITHUB_API=1 brew search thisdoesnotexist"  "brew search thisdoesnotexist"
Benchmark #1: HOMEBREW_NO_GITHUB_API=1 brew search thisdoesnotexist
  Time (mean ± σ):      5.216 s ±  0.901 s    [User: 3.636 s, System: 1.403 s]
  Range (min … max):    4.863 s …  7.772 s    10 runs

Benchmark #2: brew search thisdoesnotexist
  Time (mean ± σ):      5.431 s ±  0.080 s    [User: 3.688 s, System: 1.434 s]
  Range (min … max):    5.351 s …  5.601 s    10 runs

Summary
  'HOMEBREW_NO_GITHUB_API=1 brew search thisdoesnotexist' ran
    1.04 ± 0.18 times faster than 'brew search thisdoesnotexist'    

This makes a bit of sense since the actual curl request to github happens quite fast. When I test it independently it only takes ~200-300ms for me, which is about the difference between these two run times.

Just by observing file system activity I can tell that brew search seems to cause homebrew to open up every Cask file on disk. My guess is that’s because it’s using the name of the cask as defined in the cask ruby file for the searching, and not just the name of the file on disk?

Oddly, if brew search ONLY did a Github API requests, that would end up being way faster as well. Here’s the curl request that homebrew uses for searching makes, and it only takes ~300ms for me.

$ time curl "https://api.github.com/search/code?q=user:Homebrew+path:Formula+path:Casks+path:.+filename:python+extension:rb&per_page=100" | jq '.items[].name'
"python-dbus.rb"
"gst-python.rb"
"app-engine-python.rb"
"minimal-python.rb"
"python-markdown.rb"
"boost-python.rb"
"reorder-python-imports.rb"
"python@3.8.rb"
"python@3.7.rb"
...
curl   0.02s user 0.02s system 14% cpu 0.340 total
jq '.items[].name'  0.04s user 0.00s system 20% cpu 0.339 total

So maybe a --github-only flag would make more sense. Assuming there’s no reasonable way to speedup the local search.

That flag’s really intended for when GitHub access is slow or nonexistent. I have a MacBook Air that I often use untethered, and…

$ hyperfine "HOMEBREW_NO_GITHUB_API=1 brew search thisdoesnotexist" "brew search thisdoesnotexist"
Benchmark #1: HOMEBREW_NO_GITHUB_API=1 brew search thisdoesnotexist
  Time (mean ± σ):      4.202 s ±  0.156 s    [User: 2.928 s, System: 1.234 s]
  Range (min … max):    4.007 s …  4.430 s    10 runs
 
Benchmark #2: brew search thisdoesnotexist
  Time (mean ± σ):     11.270 s ±  0.164 s    [User: 2.967 s, System: 1.258 s]
  Range (min … max):   11.062 s … 11.560 s    10 runs
 
Summary
  'HOMEBREW_NO_GITHUB_API=1 brew search thisdoesnotexist' ran
    2.68 ± 0.11 times faster than 'brew search thisdoesnotexist'