Llvm-config 10.0.1 advertise libxml2.tbd as system-libs

At crystal-lang.org we’ve been linking llvm libraries shipped in by llvm formula for a while. We’ve been using llvm-config --libs --system-libs --ldflags to extract which libraries should be linked all togetherr.

Since llvm 10.0.1 we are unable to use that command since the xml2 library is wrongly advertised. I’m using Mojave 10.14.6 btw.

A libxml2.tbd is now advertised instead of xml2 as it was before.

When running $ "/usr/local/opt/llvm/bin/llvm-config" --libs --system-libs --ldflags 2> /dev/null with llvm 10.0.1

-L/usr/local/Cellar/llvm/10.0.1/lib ...(stripped)... -lz -lcurses -lm -llibxml2.tbd
$ /usr/local/opt/llvm/bin/llvm-config --libs --system-libs --ldflags 2> /dev/null
-L/usr/local/Cellar/llvm/10.0.1/lib -Wl,-search_paths_first -Wl,-headerpad_max_install_names
-lLLVMXRay -lLLVMWindowsManifest -lLLVMTableGen -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMOrcJIT -lLLVMOrcError -lLLVMJITLink -lLLVMObjectYAML -lLLVMMCA -lLLVMLTO -lLLVMObjCARCOpts -lLLVMExtensions -lPolly -lPollyISL -lLLVMPasses -lLLVMLineEditor -lLLVMLibDriver -lLLVMInterpreter -lLLVMFuzzMutate -lLLVMFrontendOpenMP -lLLVMMCJIT -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMDWARFLinker -lLLVMDlltoolDriver -lLLVMOption -lLLVMDebugInfoGSYM -lLLVMCoverage -lLLVMCoroutines -lLLVMXCoreDisassembler -lLLVMXCoreCodeGen -lLLVMXCoreDesc -lLLVMXCoreInfo -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Utils -lLLVMX86Info -lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyInfo -lLLVMSystemZDisassembler -lLLVMSystemZCodeGen -lLLVMSystemZAsmParser -lLLVMSystemZDesc -lLLVMSystemZInfo -lLLVMSparcDisassembler -lLLVMSparcCodeGen -lLLVMSparcAsmParser -lLLVMSparcDesc -lLLVMSparcInfo -lLLVMRISCVDisassembler -lLLVMRISCVCodeGen -lLLVMRISCVAsmParser -lLLVMRISCVDesc -lLLVMRISCVUtils -lLLVMRISCVInfo -lLLVMPowerPCDisassembler -lLLVMPowerPCCodeGen -lLLVMPowerPCAsmParser -lLLVMPowerPCDesc -lLLVMPowerPCInfo -lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMMSP430Disassembler -lLLVMMSP430CodeGen -lLLVMMSP430AsmParser -lLLVMMSP430Desc -lLLVMMSP430Info -lLLVMMipsDisassembler -lLLVMMipsCodeGen -lLLVMMipsAsmParser -lLLVMMipsDesc -lLLVMMipsInfo -lLLVMLanaiDisassembler -lLLVMLanaiCodeGen -lLLVMLanaiAsmParser -lLLVMLanaiDesc -lLLVMLanaiInfo -lLLVMHexagonDisassembler -lLLVMHexagonCodeGen -lLLVMHexagonAsmParser -lLLVMHexagonDesc -lLLVMHexagonInfo -lLLVMBPFDisassembler -lLLVMBPFCodeGen -lLLVMBPFAsmParser -lLLVMBPFDesc -lLLVMBPFInfo -lLLVMARMDisassembler -lLLVMARMCodeGen -lLLVMARMAsmParser -lLLVMARMDesc -lLLVMARMUtils -lLLVMARMInfo -lLLVMAMDGPUDisassembler -lLLVMAMDGPUCodeGen -lLLVMMIRParser -lLLVMipo -lLLVMInstrumentation -lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMAMDGPUAsmParser -lLLVMAMDGPUDesc -lLLVMAMDGPUUtils -lLLVMAMDGPUInfo -lLLVMAArch64Disassembler -lLLVMMCDisassembler -lLLVMAArch64CodeGen -lLLVMCFGuard -lLLVMGlobalISel -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMDebugInfoDWARF -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMTextAPI -lLLVMBitReader -lLLVMCore -lLLVMRemarks -lLLVMBitstreamReader -lLLVMAArch64AsmParser -lLLVMMCParser -lLLVMAArch64Desc -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMDebugInfoMSF -lLLVMBinaryFormat -lLLVMAArch64Utils -lLLVMAArch64Info -lLLVMSupport -lLLVMDemangle
-lz -lcurses -lm -llibxml2.tbd
  

When running $ /usr/local/opt/llvm/bin/llvm-config --libs --system-libs --ldflags 2> /dev/null with llvm 10.0.0_3 or before

-L/usr/local/Cellar/llvm/10.0.0_3/lib ...(stripped)... -lz -lcurses -lm -lxml2
$ /usr/local/opt/llvm/bin/llvm-config --libs --system-libs --ldflags 2> /dev/null
-L/usr/local/Cellar/llvm/10.0.0_3/lib -Wl,-search_paths_first -Wl,-headerpad_max_install_names
-lLLVMXRay -lLLVMWindowsManifest -lLLVMTableGen -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMOrcJIT -lLLVMOrcError -lLLVMJITLink -lLLVMObjectYAML -lLLVMMCA -lLLVMLTO -lLLVMPasses -lLLVMObjCARCOpts -lLLVMLineEditor -lLLVMLibDriver -lLLVMInterpreter -lLLVMFuzzMutate -lLLVMFrontendOpenMP -lLLVMMCJIT -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMDWARFLinker -lLLVMDlltoolDriver -lLLVMOption -lLLVMDebugInfoGSYM -lLLVMCoverage -lLLVMCoroutines -lLLVMXCoreDisassembler -lLLVMXCoreCodeGen -lLLVMXCoreDesc -lLLVMXCoreInfo -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Utils -lLLVMX86Info -lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyInfo -lLLVMSystemZDisassembler -lLLVMSystemZCodeGen -lLLVMSystemZAsmParser -lLLVMSystemZDesc -lLLVMSystemZInfo -lLLVMSparcDisassembler -lLLVMSparcCodeGen -lLLVMSparcAsmParser -lLLVMSparcDesc -lLLVMSparcInfo -lLLVMRISCVDisassembler -lLLVMRISCVCodeGen -lLLVMRISCVAsmParser -lLLVMRISCVDesc -lLLVMRISCVUtils -lLLVMRISCVInfo -lLLVMPowerPCDisassembler -lLLVMPowerPCCodeGen -lLLVMPowerPCAsmParser -lLLVMPowerPCDesc -lLLVMPowerPCInfo -lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMMSP430Disassembler -lLLVMMSP430CodeGen -lLLVMMSP430AsmParser -lLLVMMSP430Desc -lLLVMMSP430Info -lLLVMMipsDisassembler -lLLVMMipsCodeGen -lLLVMMipsAsmParser -lLLVMMipsDesc -lLLVMMipsInfo -lLLVMLanaiDisassembler -lLLVMLanaiCodeGen -lLLVMLanaiAsmParser -lLLVMLanaiDesc -lLLVMLanaiInfo -lLLVMHexagonDisassembler -lLLVMHexagonCodeGen -lLLVMHexagonAsmParser -lLLVMHexagonDesc -lLLVMHexagonInfo -lLLVMBPFDisassembler -lLLVMBPFCodeGen -lLLVMBPFAsmParser -lLLVMBPFDesc -lLLVMBPFInfo -lLLVMARMDisassembler -lLLVMARMCodeGen -lLLVMARMAsmParser -lLLVMARMDesc -lLLVMARMUtils -lLLVMARMInfo -lLLVMAMDGPUDisassembler -lLLVMAMDGPUCodeGen -lLLVMMIRParser -lLLVMipo -lLLVMInstrumentation -lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMAMDGPUAsmParser -lLLVMAMDGPUDesc -lLLVMAMDGPUUtils -lLLVMAMDGPUInfo -lLLVMAArch64Disassembler -lLLVMMCDisassembler -lLLVMAArch64CodeGen -lLLVMCFGuard -lLLVMGlobalISel -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMDebugInfoDWARF -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMTextAPI -lLLVMBitReader -lLLVMCore -lLLVMRemarks -lLLVMBitstreamReader -lLLVMAArch64AsmParser -lLLVMMCParser -lLLVMAArch64Desc -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMDebugInfoMSF -lLLVMBinaryFormat -lLLVMAArch64Utils -lLLVMAArch64Info -lLLVMSupport -lLLVMDemangle
-lz -lcurses -lm -lxml2
  

  • I couldn’t find documentation regarding how to tweak does settings in llvm and put them in the formula.
  • I also couldn’t find where the name libxml2.tbd comes from.
  • I’ve found no changes mentioning xml between llvm 10.0.1 and 10.0.1

While I’m blindly tweaking the formula, maybe someone could have a clue of what might be going on here.

cc: @SMillerDev @Bo98

1 Like

This is an LLVM bug and affects LLVM builds outside of Homebrew too.

I’ve not looked too much into it yet. There’s been several unresolved llvm-config macOS bug reports since LLVM 3.9 (e.g. https://bugs.llvm.org/show_bug.cgi?id=40252), but perhaps we’re being forced now to patch the bugs ourselves.

There has however been libxml2 CMake changes (for Big Sur support), so perhaps llvm-config broke from it. I’ll dig deeper.

1 Like

Just ran into the same bug. FWIW, using -lxml2 still works, but libxml2.tbd does exists as
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/lib/libxml2.tbd (see contents below). Is there a way to patch the formula or downgrade LLVM?

--- !tapi-tbd-v3
archs:           [ x86_64 ]
uuids:           [ 'x86_64: 24147A90-E3EB-3926-BFB0-5F0FC9F706E2', 'x86_64: 24147A90-E3EB-3926-BFB0-5F0FC9F706E2' ]
platform:        zippered
install-name:    '/usr/lib/libxml2.2.dylib'
current-version: 10.9
compatibility-version: 10
exports:
  - archs:           [ x86_64 ]
    symbols:         [ _UTF8ToHtml, _UTF8Toisolat1, ___docbDefaultSAXHandler, ___htmlDefaultSAXHandler,
...

As a workaround @Blaisorblade you can do

$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/6616d50fb0b24dbe30f5e975210bdad63257f517/Formula/llvm.rb
$ brew pin llvm

That should leave you with the 10.0.0_3 bottle.

1 Like

Thanks, that helps — I had to brew uninstall llvm first. Various alternatives failed (some details below), some with exceptions that might be issues, but I’ll leave that call to others.

$ brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/6616d50fb0b24dbe30f5e975210bdad63257f517/Formula/llvm.rb
Updating Homebrew...
==> Auto-updated Homebrew!
Updated Homebrew from ded2927751 to 236cbbc53e.
Updated 5 taps (homebrew/cask-versions, homebrew/core, homebrew/cask, homebrew/bundle and homebrew/cask-fonts).
==> Updated Formulae
Updated 117 formulae.
[...]

Warning: Calling Installation of llvm from a GitHub commit URL is deprecated! Use 'brew extract llvm' to stable tap on GitHub instead.
######################################################################## 100.0%
Warning: llvm 10.0.0_3 is already installed and up-to-date
To reinstall 10.0.0_3, run `brew reinstall llvm`
$ brew upgrade https://raw.githubusercontent.com/Homebrew/homebrew-core/6616d50fb0b24dbe30f5e975210bdad63257f517/Formula/llvm.rb
Warning: Calling Installation of llvm from a GitHub commit URL is deprecated! Use 'brew extract llvm' to stable tap on GitHub instead.
######################################################################## 100.0%
Error: uninitialized constant Cask::CaskLoader::FromURILoader::Cache
Please report this issue:
  https://docs.brew.sh/Troubleshooting
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:95:in `initialize'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:196:in `new'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:196:in `block in for'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:195:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:195:in `for'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:184:in `load'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:178:in `cask_exists_with_ref'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:59:in `block in to_resolved_formulae_to_casks'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:56:in `each'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:56:in `to_resolved_formulae_to_casks'
/usr/local/Homebrew/Library/Homebrew/cli/args.rb:72:in `resolved_formulae_casks'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:76:in `upgrade'
/usr/local/Homebrew/Library/Homebrew/brew.rb:119:in `<main>'
$ brew reinstall https://raw.githubusercontent.com/Homebrew/homebrew-core/6616d50fb0b24dbe30f5e975210bdad63257f517/Formula/llvm.rb
Warning: Calling Installation of llvm from a GitHub commit URL is deprecated! Use 'brew extract llvm' to stable tap on GitHub instead.
######################################################################## 100.0%
Error: uninitialized constant Cask::CaskLoader::FromURILoader::Cache
Please report this issue:
  https://docs.brew.sh/Troubleshooting
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:95:in `initialize'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:196:in `new'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:196:in `block in for'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:195:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:195:in `for'
/usr/local/Homebrew/Library/Homebrew/cask/cask_loader.rb:184:in `load'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:178:in `cask_exists_with_ref'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:59:in `block in to_resolved_formulae_to_casks'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:56:in `each'
/usr/local/Homebrew/Library/Homebrew/cli/named_args.rb:56:in `to_resolved_formulae_to_casks'
/usr/local/Homebrew/Library/Homebrew/cli/args.rb:72:in `resolved_formulae_casks'
/usr/local/Homebrew/Library/Homebrew/cmd/reinstall.rb:64:in `reinstall'
/usr/local/Homebrew/Library/Homebrew/brew.rb:119:in `<main>'

Installing formulae from random urls is a bad idea. You’re better off using brew extract as the output mentions.

FWIW, I posted the logs only because the error asked for a report, but I didn’t have time to troubleshoot it properly.

What’s wrong with a permalink? I’ve looked into the docs of brew extract, googled about it, and found https://stackoverflow.com/q/62785290/53974, but none of that motivates the extra steps — especially since this is a temporary solution.

Thanks again to @bcardiff for providing a simple and complete solution :-).

In this case it comes from the homebrew-core repo, so likely it’s fine. The basic concept of installing a random formula from some URL is potentially very dangerous though (especially since there’s a lot of recommendations in blogs to “install this URL for an OpenSSL version”). The other problem you already noticed, which is that installing 2 versions of software with the same name will break things.

For this temporary solution it’s probably fine, just be aware of the risks.

Thanks to https://github.com/Homebrew/homebrew-core/pull/59953 I was able to find https://reviews.llvm.org/D86134 which fix the issue upstream.

I don’t see any backport of those to llvm10.

Applying those changes:

probably requires others like:

Not sure if it’s better to get those patches in brew directly meanwhile or a backport from llvm should be waited for.

If the patchset is small enough and it fixes a major issue I think it would be fine to patch it.

I was able to reach this patch for llvm-project.

To apply it to homebrew-core I needed to split it in the resources structure of the formula https://github.com/bcardiff/homebrew-core/commit/1fda278866512376109fa953599cc4e65021df67 . I left them in a private gist at https://gist.github.com/bcardiff/82c843ca8d745887289c3c3397461157 . I am not sure if that’s the way to go.

The output of $ llvm-config --libs --system-libs --ldflags looks better.

-L/usr/local/Cellar/llvm/10.0.1/lib -Wl,-search_paths_first -Wl,-headerpad_max_install_names
-lLLVMXRay ...(stripped) -lLLVMDemangle
-lm -lz -lcurses -lxml2

But when I attempt to link to them I get some errors that I am not sure are because of building llvm from source or if the patch breaks something else

Undefined symbols for architecture x86_64:
  "llvm::DisableABIBreakingChecks", referenced from:
      llvm::VerifyDisableABIBreakingChecks in llvm_ext.o
      llvm::VerifyDisableABIBreakingChecks in libLLVMAArch64AsmParser.a(AArch64AsmParser.cpp.o)
      llvm::VerifyDisableABIBreakingChecks in libLLVMAArch64CodeGen.a(AArch64AsmPrinter.cpp.o)
      llvm::VerifyDisableABIBreakingChecks in libLLVMAArch64CodeGen.a(AArch64TargetMachine.cpp.o)
      llvm::VerifyDisableABIBreakingChecks in libLLVMAArch64Info.a(AArch64TargetInfo.cpp.o)
      llvm::VerifyDisableABIBreakingChecks in libLLVMAArch64Desc.a(AArch64MCTargetDesc.cpp.o)
      llvm::VerifyDisableABIBreakingChecks in libLLVMARMAsmParser.a(ARMAsmParser.cpp.o)
      ...
  "typeinfo for llvm::AsmPrinter", referenced from:
      typeinfo for (anonymous namespace)::AArch64AsmPrinter in libLLVMAArch64CodeGen.a(AArch64AsmPrinter.cpp.o)
      typeinfo for llvm::ARMAsmPrinter in libLLVMARMCodeGen.a(ARMAsmPrinter.cpp.o)
  "typeinfo for llvm::CallbackVH", referenced from:
      typeinfo for llvm::ValueMapCallbackVH<llvm::Value*, llvm::SCEVWrapPredicate::IncrementWrapFlags, llvm::ValueMapConfig<llvm::Value*, llvm::sys::SmartMutex<false> > > in libLLVMARM
...
(stripped)
...

Just add .patch to the original commit url and use that. Or, if you need to modify the patch, submit it to the formula-patches repo of homebrew.

PRs submitted

The llvm::DisableABIBreakingChecks error was due to some misconfiguration in my environment.