Brew failing to handle localisation

I have:

|> printenv | grep ^L
LINUX_VERSION=Debian
LINUXBREW_HOME=/home/linuxbrew/.linuxbrew
LANGUAGE=en_GB:en
LC_ADDRESS=en_GB.UTF-8
LC_NAME=en_GB.UTF-8
LC_MONETARY=en_GB.UTF-8
LOGNAME=russel
LC_PAPER=en_GB.UTF-8
LANG=en_GB.UTF-8
LESSCLOSE=/usr/bin/lesspipe %s %s
LC_IDENTIFICATION=en_GB.UTF-8
LESSOPEN=| /usr/bin/lesspipe %s
LIBRARY_PATH=.:/home/users/russel/Built/lib
LC_TELEPHONE=en_GB.UTF-8
LC_MESSAGES=en_GB.UTF-8
LC_MEASUREMENT=en_GB.UTF-8
LD_LIBRARY_PATH=.:/home/users/russel/Built/lib
LC_CTYPE=en_GB.UTF-8
LC_TIME=en_GB.UTF-8
LC_ALL=en_GB.UTF-8
LC_COLLATE=en_GB.UTF-8
LC_NUMERIC=en_GB.UTF-8

which seems to be fine generally. However brew insists on stating:

|> brew list
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 4: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 4: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
==> Downloading https://linuxbrew.bintray.com/bottles-portable-ruby/portable-ruby-2.6.3_2.x86_64_linux.bottle.tar.gz
Already downloaded: /home/users/russel/.cache/Homebrew/portable-ruby-2.6.3_2.x86_64_linux.bottle.tar.gz
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = "en_US.UTF-8",
	LANG = (unset)
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
==> Pouring portable-ruby-2.6.3_2.x86_64_linux.bottle.tar.gz
alsa-lib	  chapel  fontconfig  gpatch  kotlin  libice		libsm	libxcb	  libxfixes  libxrender  ncurses      patchelf	      python@3.8  unzip       xz
bzip2		  cups	  freetype    gradle  libbsd  libpng		libx11	libxdmcp  libxi      libxt	 openjdk      pip-completion  readline	  util-linux  zip
cargo-completion  expat   gdbm	      groovy  libffi  libpthread-stubs	libxau	libxext   libxrandr  libxtst	 openssl@1.1  pkg-config      sqlite	  xorgproto   zlib

it seems to completely ignore the fact that I have correctly set exactly the settings it says are unset.

|> echo $LANGUAGE $LC_ALL $LANG
en_GB:en en_GB.UTF-8 en_GB.UTF-8

What is the brew bash script system doing to manage to lose this information?

What’s the output of locale charmap?

|> locale charmap
UTF-8

OK, that’s what Homebrew expects. How about the output of brew config and brew doctor?

|> brew config
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 4: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
HOMEBREW_VERSION: 2.4.3
ORIGIN: https://github.com/Homebrew/brew
HEAD: c69d97aaa5ba07e8c2af3708c8093ca0875cfb06
Last commit: 3 days ago
Core tap ORIGIN: https://github.com/Homebrew/linuxbrew-core
Core tap HEAD: 13d04d630d0b9625692d72e89cae5a5c392c8d19
Core tap last commit: 5 hours ago
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_DISPLAY: :0
HOMEBREW_EDITOR: emacsclient --tty --alternate-editor emacs
HOMEBREW_MAKE_JOBS: 8
CPU: octa-core 64-bit haswell
Homebrew Ruby: 2.6.3 => /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/bin/ruby
Clang: 9.0 build (parse error)
Git: 2.27.0 => /usr/bin/git
Curl: 7.68.0 => /usr/bin/curl
Kernel: Linux 5.7.0-1-amd64 x86_64 GNU/Linux
OS: Debian GNU/Linux bullseye/sid (sid)
Host glibc: 2.30
/usr/bin/gcc: 9.3.0
glibc: N/A
gcc: N/A
xorg: N/A
|> brew doctor
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 4: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: /usr/bin occurs before /home/linuxbrew/.linuxbrew/bin
This means that system-provided programs will be used instead of those
provided by Homebrew. The following tools exist at both paths:

Consider setting your PATH so that /home/linuxbrew/.linuxbrew/bin
occurs before /usr/bin. Here is a one-liner:
  echo 'export PATH="/home/linuxbrew/.linuxbrew/bin:$PATH"' >> /home/users/russel/.bash_profile
  xzless
  javap
  renice
  fc-query
  pngfix
  sqlite3
  javadoc
  ionice
  png-fix-itxt
  jjs
  lzegrep
  lzfgrep
  linux64
  lzmore
  ul
  linux32
  fallocate
  last
  lzcat
  fc-list
  jmap
  lscpu
  xzdiff
  ipcrm
  lastb
  unlzma
  keytool
  xzcmp
  script
  c_rehash
  chrt
  lzdiff
  hexdump
  javac
  taskset
  jinfo
  rev
  rmiregistry
  ncursesw6-config
  fc-match
  fc-cache
  zipnote
  mcookie
  clear
  funzip
  toe
  fc-pattern
  reset
  lsns
  utmpdump
  uuidgen
  jstatd
  jshell
  zipgrep
  unzip
  jcmd
  zipsplit
  unzipsfx
  libpng16-config
  scriptreplay
  rmic
  jarsigner
  pkg-config
  jconsole
  openssl
  logger
  wall
  fc-validate
  look
  zip
  jlink
  choom
  column
  infotocap
  xz
  ipptool
  serialver
  lslocks
  jpackage
  unxz
  jaotc
  ipcmk
  jmod
  jar
  jfr
  jhsdb
  fc-scan
  fc-cat
  lzcmp
  libpng-config
  lsmem
  setarch
  lzgrep
  xzfgrep
  mesg
  lzless
  zipcloak
  zipinfo
  setsid
  flock
  lslogins
  jdeprscan
  jdb
  captoinfo
  rmid
  jstack
  lsipc
  ncurses6-config
  ipcs
  i386
  whereis
  xzmore
  lzmainfo
  scriptlive
  jps
  prlimit
  tset
  x86_64
  patch
  getopt
  col
  namei
  jstat
  xzgrep
  jimage
  nsenter
  xzcat
  jrunscript
  fc-conflist
  xzegrep
  infocmp
  colrm
  setterm
  eject
  uuidparse
  unshare
  fincore
  jdeps
  tic
  lzma
  tput
  tabs
  java

Curiouser and curiouser. Your locale charmap output should not have caused brew.sh to set LC_ALL at all, and I’ve never seen bash warn about the bog-standard en_US.UTF-8 locale. Also, there’s nothing obviously wrong with your installation.

If you can confirm that you’ve made no changes under /home/linuxbrew/.linuxbrew that you can’t afford to use, try a brew update-reset and see if that fixes things.

As far as I am aware I have done nothing to /home/linuxbrew/.linuxbrew, I just let brew manage everything under /home/linuxbrew.

|> brew update-reset
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 4: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
==> Fetching /home/linuxbrew/.linuxbrew/Homebrew...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

==> Resetting /home/linuxbrew/.linuxbrew/Homebrew...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Branch 'master' set up to track remote branch 'master' from 'origin'.
Switched to and reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/homebrew/homebrew-core...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

==> Resetting /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/homebrew/homebrew-core...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/linuxbrew/homebrew-xorg...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 4 (delta 2), reused 4 (delta 2), pack-reused 0
Unpacking objects: 100% (4/4), 371 bytes | 26.00 KiB/s, done.
From https://github.com/Linuxbrew/homebrew-xorg
 * [new branch]      migrate-to-core -> origin/migrate-to-core
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

==> Resetting /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/linuxbrew/homebrew-xorg...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

and then:

|> brew list
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh: line 4: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
alsa-lib	  cups	      gdbm    kotlin  libpng		libxau	  libxfixes   libxt    openssl@1.1     python@3.8  util-linux  zlib
bzip2		  expat       gpatch  libbsd  libpthread-stubs	libxcb	  libxi       libxtst  patchelf        readline    xorgproto
cargo-completion  fontconfig  gradle  libffi  libsm		libxdmcp  libxrandr   ncurses  pip-completion  sqlite	   xz
chapel		  freetype    groovy  libice  libx11		libxext   libxrender  openjdk  pkg-config      unzip	   zip

OK, your post-reset brew list now doesn’t re-download portable Ruby, which was an oddity in your original run.

To return to your original question:

Homebrew sanitizes its own runtime environment (not your normal shell environment) to minimize unpredictability. In particular, it drops all user-customized locale environment variables, so that non-ASCII characters like the brew emoji aren’t scrambled by some weird character mapping.

In that light, what’s the output of these commands:

LANG= locale charmap
localectl status
HOMEBREW_NO_ENV_FILTERING=1 brew list

The attempted reinstall of Ruby seems random, it comes and goes.

|> LANG= locale charmap
UTF-8
|> localectl status
   System Locale: LANG=en_GB.UTF-8
                  LANGUAGE=en_GB:en
       VC Keymap: n/a
      X11 Layout: gb
       X11 Model: pc105
|> HOMEBREW_NO_ENV_FILTERING=1 brew list
alsa-lib	  chapel  fontconfig  gpatch  kotlin  libice		libsm	libxcb	  libxfixes  libxrender  ncurses      patchelf	      python@3.8  unzip       xz
bzip2		  cups	  freetype    gradle  libbsd  libpng		libx11	libxdmcp  libxi      libxt	 openjdk      pip-completion  readline	  util-linux  zip
cargo-completion  expat   gdbm	      groovy  libffi  libpthread-stubs	libxau	libxext   libxrandr  libxtst	 openssl@1.1  pkg-config      sqlite	  xorgproto   zlib

That’s actually more troubling than bash throwing a setlocale warning on en_US.UTF-8. If it still happens after the brew update-reset you just did, and it’s still for portable-ruby-2.6.3_2 (i.e. not an upgrade), definitely call for help again.

Back to the locale stuff. At this point, it seems like the en_US.UTF-8 locale is not installed on your system. Since it’s Homebrew’s standard locale, please do the following:

  1. Edit /etc/locale.gen as root, and uncomment the line that starts with en_US.UTF-8. (If no such line exists,
  2. Run locale-gen as root. You should see en_US.UTF-8... done at some point in its output.
  3. Run brew list again and see if bash still complains.

Wilco re the attempted re-install of ruby.

Re the locale, I am surprised by the need to have to install en_us.UTF-8, my locale is en_gb.UTF-8, I am not in the USA, I am in the UK. Isn’t the point that applications should deal with the locale of the user rather than user having to suffer the application’s imposed locale?

I can’t speak for the rationale. I can only surmise that the Homebrew maintainers ran into enough problems with supporting everyone’s locales, that they actually bothered to write (and maintain) the code to nail it down.

Hummm… or alternatively ran away from being able to cope with other people’s locales and instead impose USA English on everyone? Anyway having followed your instructions the locale message have gone. On the other hand:

|> brew list
==> Downloading https://linuxbrew.bintray.com/bottles-portable-ruby/portable-ruby-2.6.3_2.x86_64_linux.bottle.tar.gz
Already downloaded: /home/users/russel/.cache/Homebrew/portable-ruby-2.6.3_2.x86_64_linux.bottle.tar.gz
==> Pouring portable-ruby-2.6.3_2.x86_64_linux.bottle.tar.gz
alsa-lib	  expat       gradle  libice		libxau	   libxi       ncurses	       pkg-config  util-linux
bzip2		  fontconfig  groovy  libpng		libxcb	   libxrandr   openjdk	       python@3.8  xorgproto
cargo-completion  freetype    kotlin  libpthread-stubs	libxdmcp   libxrender  openssl@1.1     readline    xz
chapel		  gdbm	      libbsd  libsm		libxext    libxt       patchelf        sqlite	   zip
cups		  gpatch      libffi  libx11		libxfixes  libxtst     pip-completion  unzip	   zlib

let’s hope this reinstall never happens again now the locale en_US.UTF-8 is installed.

This issue is a known bug, unfortunately with no resolution currently. The workaround as you’ve discovered is to install the en_US.UTF-8 locale. A PR to fix this issue would be welcome.

OK. Is there a bug report for this that I can monitor?

I am not sure I can do anything in the next few weeks, but maybe later in the year I could have a look at it.

No current issue. Your original bug report from 2018 is at https://github.com/Linuxbrew/brew/issues/568

I’d forgotten about that, I guess because it was closed as an issue. I guess then this is a “Wont Fix” for the Linux brew port of Homebrew people?

We’re waiting for a PR from an interested member of the community of Homebrew contributors.

Turns out that any UTF-8 locale may be used, so en_GB.UTF-8 should work just fine.
See https://github.com/Homebrew/brew/blob/master/Library/Homebrew/brew.sh#L1-L5
What’s the output of locale charmap? It should be UTF-8.

|> locale charmap
UTF-8

Which is why I consider this an error in the brew script system. I should not have to install en_US.UTF-8 because that test should not fail. Of course this brew.sh is not the initiating Bash script. /home/linuxbrew/.linuxbrew/bin/brew is:

> which brew
/home/linuxbrew/.linuxbrew/bin/brew

So running bash -x /home/linuxbrew/.linuxbrew/bin/brew we see that brew.sh is called thus:

exec /usr/bin/env -i HOME=/home/users/russel SHELL=/bin/bash PATH=/usr/bin:/bin:/usr/sbin:/sbin TERM=xterm-256color DISPLAY=:0 LOGNAME=russel USER=russel SSH_AUTH_SOCK=/run/user/1000/keyring/ssh HOMEBREW_BREW_FILE=/home/linuxbrew/.linuxbrew/bin/brew HOMEBREW_DISPLAY=:0 'HOMEBREW_EDITOR=emacsclient --tty --alternate-editor emacs' HOMEBREW_LIBRARY=/home/linuxbrew/.linuxbrew/Homebrew/Library HOMEBREW_PATH=.:/home/users/russel/bin.lynet:/home/users/russel/Built/bin:/home/users/russel/bin.Linux.x86_64:/home/users/russel/bin:/home/users/russel/.local/bin:/home/users/russel/.cargo/bin:/home/Checkouts/Git/Cask/bin:/bin:/home/linuxbrew/.linuxbrew/opt/groovy/libexec/bin:/home/users/russel/.nvm/versions/node/v12.0.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/sbin:/sbin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin HOMEBREW_PREFIX=/home/linuxbrew/.linuxbrew HOMEBREW_REPOSITORY=/home/linuxbrew/.linuxbrew/Homebrew /bin/bash /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.sh

so if I try:

|> /usr/bin/env -i HOME=/home/users/russel SHELL=/bin/bash PATH=/usr/bin:/bin:/usr/sbin:/sbin TERM=xterm-256color DISPLAY=:0 LOGNAME=russel USER=russel SSH_AUTH_SOCK=/run/user/1000/keyring/ssh HOMEBREW_BREW_FILE=/home/linuxbrew/.linuxbrew/bin/brew HOMEBREW_DISPLAY=:0 'HOMEBREW_EDITOR=emacsclient --tty --alternate-editor emacs' HOMEBREW_LIBRARY=/home/linuxbrew/.linuxbrew/Homebrew/Library HOMEBREW_PATH=.:/home/users/russel/bin.lynet:/home/users/russel/Built/bin:/home/users/russel/bin.Linux.x86_64:/home/users/russel/bin:/home/users/russel/.local/bin:/home/users/russel/.cargo/bin:/home/Checkouts/Git/Cask/bin:/bin:/home/linuxbrew/.linuxbrew/opt/groovy/libexec/bin:/home/users/russel/.nvm/versions/node/v12.0.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/sbin:/sbin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin HOMEBREW_PREFIX=/home/linuxbrew/.linuxbrew HOMEBREW_REPOSITORY=/home/linuxbrew/.linuxbrew/Homebrew locale charmap
ANSI_X3.4-1968

we see the core of the problem: brew.sh is not running in the user’s environment, it is running in an empty environment.