Brew edit changes not used when installing

When I try to install ffmpeg using changes to the flags and dependencies that should enable building with jack support, after installation, I still have no jack support. I’ve followed directions that suggest using brew edit, then changing what is needed to be changed, but I can see the flags are still set to “disable” and running it shows it doesn’t have jack support. I have built ffmpeg from source with support and can confirm this works as expected.

Here is what I changed my ffmpeg Formula (located in: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/ffmpeg.rb) to:

class Ffmpeg < Formula
  desc "Play, record, convert, and stream audio and video"
  homepage "https://ffmpeg.org/"
  url "https://ffmpeg.org/releases/ffmpeg-4.2.2.tar.xz"
  sha256 "cb754255ab0ee2ea5f66f8850e1bd6ad5cac1cd855d0a2f4990fb8c668b0d29c"
  revision 2
  head "https://github.com/FFmpeg/FFmpeg.git"

  bottle do
    sha256 "34e8b4424611acc2f90e27b4e1318fc3972b036231a171faa4e017a9b98b9d1b" => :catalina
    sha256 "80582f6eac8470182df842a072e074de3624ec3f5c091aa9151c178745a06011" => :mojave
    sha256 "afb1c2a2c38fa4d39dbd178cf5258bc3b81e805196196d24ab3676f134914cab" => :high_sierra
  end

  depends_on "nasm" => :build
  depends_on "pkg-config" => :build
  depends_on "texi2html" => :build
  depends_on "aom"
  depends_on "fontconfig"
  depends_on "freetype"
  depends_on "frei0r"
  depends_on "gnutls"
  depends_on "lame"
  depends_on "jack"
  depends_on "libass"
  depends_on "libbluray"
  depends_on "libsoxr"
  depends_on "libvidstab"
  depends_on "libvorbis"
  depends_on "libvpx"
  depends_on "opencore-amr"
  depends_on "openjpeg"
  depends_on "opus"
  depends_on "rtmpdump"
  depends_on "rubberband"
  depends_on "sdl2"
  depends_on "snappy"
  depends_on "speex"
  depends_on "tesseract"
  depends_on "theora"
  depends_on "webp"
  depends_on "x264"
  depends_on "x265"
  depends_on "xvid"
  depends_on "xz"

  uses_from_macos "bzip2"
  uses_from_macos "zlib"

  def install
    # Work around Xcode 11 clang bug
    # https://bitbucket.org/multicoreware/x265/issues/514/wrong-code-generated-on-macos-1015
    ENV.append_to_cflags "-fno-stack-check" if DevelopmentTools.clang_build_version >= 1010

    args = %W[
      --prefix=#{prefix}
      --enable-shared
      --enable-pthreads
      --enable-version3
      --enable-avresample
      --cc=#{ENV.cc}
      --host-cflags=#{ENV.cflags}
      --host-ldflags=#{ENV.ldflags}
      --enable-ffplay
      --enable-gnutls
      --enable-gpl
      --enable-libaom
      --enable-libbluray
      --enable-libmp3lame
      --enable-libopus
      --enable-librubberband
      --enable-libsnappy
      --enable-libtesseract
      --enable-libtheora
      --enable-libvidstab
      --enable-libvorbis
      --enable-libvpx
      --enable-libwebp
      --enable-libx264
      --enable-libx265
      --enable-libxvid
      --enable-lzma
      --enable-libfontconfig
      --enable-libfreetype
      --enable-frei0r
      --enable-libass
      --enable-libopencore-amrnb
      --enable-libopencore-amrwb
      --enable-libopenjpeg
      --enable-librtmp
      --enable-libspeex
      --enable-libsoxr
      --enable-videotoolbox
      --enable-libjack
      --enable-indev=jack
    ]

    system "./configure", *args
    system "make", "install"

    # Build and install additional FFmpeg tools
    system "make", "alltools"
    bin.install Dir["tools/*"].select { |f| File.executable? f }

    # Fix for Non-executables that were installed to bin/
    mv bin/"python", pkgshare/"python", :force => true
  end

  test do
    # Create an example mp4 file
    mp4out = testpath/"video.mp4"
    system bin/"ffmpeg", "-filter_complex", "testsrc=rate=1:duration=1", mp4out
    assert_predicate mp4out, :exist?
  end
end

I had uninstalled ffmpeg prior to this, there was no other version on my machine. I ran brew install ffmpeg after making these changes.

brew install ffmpeg will install from the binaries that homebrew provides so won’t have your changes. Seeing how https://github.com/homebrew-ffmpeg/homebrew-ffmpeg/issues/2 is still open Jack support probably isn’t as easy as just adding the flags though. I’d see if you can help that tap enable Jack support instead of fiddling with the core formula.

Yeah, actually those flags do make it possible to install, and it also includes the dependency for jack as expected.
Most of the information I read on SO and elsewhere about altering flags for installs in brew describe the technique I followed. Why is this information incorrect? If the install process uses pre-compiled binaries anyway, why bother having a whole config available to use and edit? Additionally, is it possible to force brew to compile and install using the updated commands rather than use the binaries?

Most of the information I read on SO and elsewhere about altering flags for installs in brew describe the technique I followed.

This is the way to alter an install, however this is really meant if you want to submit your changes to homebrew core. If you want to have a version seperate from homebrew-core the best way is to maintain your own tap (or check the tap I linked). This is also partially the reason why you have these files. You can make your changes and submit them through git, as people are used to.

Additionally, is it possible to force brew to compile and install using the updated commands rather than use the binaries?

https://docs.brew.sh/Manpage#install-options-formula --build-from-source