Deep dive

Making a Homebrew bottle run somewhere Homebrew never put it

Close to 200 of hop's recipes are Homebrew's own prebuilt binaries — "bottles" — fetched straight from ghcr.io/homebrew/core. This page explains why that isn't as simple as downloading and unzipping, and the real bugs it took to get all the way there.

In plain terms A prebuilt binary usually has some file paths baked into it at compile time — "my shared library lives at exactly /this/one/place." Homebrew's own binaries are built assuming they'll live at Homebrew's own fixed install location. hop installs things somewhere else entirely, in its own per-package store, so every one of those baked-in paths has to be found and rewritten before the binary will actually run — not just extracted.

The problem, concretely

Homebrew's official Linux install location is always /home/linuxbrew/.linuxbrew; on a Mac it's /opt/homebrew. Because that location is fixed, Homebrew's build infrastructure can bake absolute references to it directly into a binary — "when you need libcurl, look at exactly this path" — and never worry about it being wrong, because every real Homebrew install puts libcurl at that exact path.

hop's store is deliberately not a fixed, shared location. Every package gets its own directory, named by a content hash that depends on exactly what's in it: curl-8.22.0-b126ad9cb96d. That's what makes two different builds of the same version coexist safely and what makes an unmodified store path safe to keep forever — but it also means the path a Homebrew binary expects to find its dependencies at simply doesn't exist under hop. Downloading and extracting the bottle isn't enough; every one of those baked-in references has to be located and rewritten to point at wherever hop actually put the dependency.

Placeholder tokens, not real paths

Fortunately, Homebrew's own bottles aren't quite as fixed as they look. Rather than baking in the literal install path at build time, Homebrew bakes in a placeholder token — a string like @@HOMEBREW_PREFIX@@/opt/curl/lib — and rewrites those tokens to the real path as one of the very last steps of its own install process. hop does the equivalent rewrite itself, substituting its own store path instead.

There isn't just one token shape. Over the course of getting this right against real bottles, hop's relocation code ended up handling several distinct forms, each confirmed against an actual bottle that used it:

  • @@HOMEBREW_PREFIX@@/opt/<formula>/... — the ordinary case, a reference to another formula's own directory.
  • @@HOMEBREW_CELLAR@@/<formula>/<version>/... and the mixed @@HOMEBREW_PREFIX@@/Cellar/<formula>/<version>/... form — both name a formula's install directory directly rather than through Homebrew's opt symlink farm.
  • A bare @@HOMEBREW_PREFIX@@/lib/<file>, naming no formula at all — Homebrew's own convention for a keg-linked library placed directly at the prefix root. gdk-pixbuf-csource failed with a bare "Symbol not found" on macOS until this case was handled: hop tries the formula's own store path first, then every dependency, and uses whichever one the file actually exists at.
  • @@HOMEBREW_PERL@@ — Homebrew doesn't vendor its own Perl on either platform, so formulas with a Perl-based script (autoconf, exiftool) reference the ambient system interpreter instead, resolved via a plain PATH lookup rather than hop's own store.

The very first version of this matching missed one detail entirely: a formula name that contains an @, like python@3.14 or openssl@3. Without @ in the character class the regex simply stopped matching one character early, truncating the token and silently leaving it unresolved — confirmed for real on python@3.14's own pip3.14 script, which shipped a literal, unrewritten #!@@HOMEBREW_CELLAR@@/python@3.14/... shebang and refused to execute at all.

Not just text — RPATH and LC_RPATH

Placeholder tokens don't only show up as readable text in shell scripts. A compiled binary's own metadata carries the same kind of reference, in a form specific to each platform's executable format:

  • On Linux, an ELF binary's RPATH/RUNPATH — a colon-separated list of directories the dynamic linker searches for shared libraries — is rewritten with patchelf.
  • On macOS, a Mach-O binary's LC_LOAD_DYLIB and LC_ID_DYLIB load commands (what library a binary needs, and what a library calls itself) are rewritten with install_name_tool.

A third Mach-O load command, LC_RPATH, went unpatched for a real stretch of this project's life — the original code only ever parsed otool -L/-D output (load commands and IDs), never otool -l's program-header view, where LC_RPATH actually lives. This broke exactly the formulas that reference a sibling library via the relative form @rpath/libX.dylib rather than a full path — fbthrift, watchman, and jasper among them — with a placeholder LC_RPATH entry that nothing was ever rewriting.

One rewrite per file, not several

The most consequential bug in this whole mechanism wasn't a missing case — it was assuming that patching a file's RPATH, then separately patching its interpreter, then separately patching it again to add a dependency's directory, was equivalent to computing the right answer once and writing it. It isn't.

The bug that exposed this: many Homebrew Linux bottles don't declare an RPATH entry for every dependency consistently across every file within the same formula. glib's libglib-2.0.so.0 correctly references zlib-ng-compat's directory in its RPATH; libgio-2.0.so.0, in the exact same formula, needs the same library and doesn't declare it. Real Homebrew papers over this with ldconfig's single, shared, prefix-wide library cache — every installed formula's lib directory is registered there once, so a file missing its own explicit RPATH entry still finds what it needs. hop's isolated, per-package store has no equivalent, so the fix adds, unconditionally, every dependency's lib directory to a file's RPATH regardless of whether that specific file's own declared entries already covered it.

The first version of that fix applied it as a separate, third patchelf invocation, after the placeholder-resolution pass and the interpreter fix's own pass. Rebuilt and tested against glib alone, this produced an outright segmentation fault — not a missing-library error, a crash — in a binary whose ELF header still looked completely well-formed under inspection. Three sequential rewrites of the same file's dynamic section, it turned out, is enough to corrupt something patchelf itself doesn't reliably preserve across repeated edits. The fix: compute the complete desired RPATH and interpreter for a file up front, as pure data, and apply both together in exactly one patchelf call.

Confirmed, not assumed: every claim on this page was checked against a real, running binary — most of it inside an arm64 Linux container via Docker/Colima, since a produced ELF binary can only really be verified by executing it on Linux. The darwin path was checked the same way, natively, against 149 of 149 Homebrew formulas with real executables in the current index.

The one file that must never be touched

Fixing that crash didn't fully explain it, and the real explanation turned out to be more specific than "too many rewrites" in general. A full gdb backtrace on the still-crashing case pointed at _dl_start — inside glibc's own dynamic loader, before it had even gotten as far as reading the program it was asked to run.

glibc's own loader is a uniquely fragile file. Unlike every other binary on the system, it has no interpreter of its own — it is the interpreter — and it relocates itself by hand at process startup, in code that assumes its own program headers exactly as the toolchain originally laid them out. Homebrew ships it with no RPATH at all, confirmed directly. The "add every dependency's directory unconditionally" fix, applied without exception, gave the loader an RPATH section it had never had — and inserting that new section was enough to break its self-relocation on the very next run. It has no legitimate use for an RPATH of its own anyway (it never resolves a dependency through one; every other binary's RPATH is what it resolves), so the fix was to exclude it from this pass entirely rather than try to compute a "safer" version of the same change.

A dependency being present isn't the same as a file needing it

One more bug came from the same root cause in a subtler form. hop injects glibc as an implicit dependency into every Homebrew-Linux artifact's closure, since some formulas' bottles are built against a newer glibc than a given host provides and genuinely need it. But not every formula that ends up with glibc in its dependency closure actually needs Homebrew's own glibc over the host's — some bottles are built with a completely ordinary, literal system interpreter path, proving they were never meant to use Homebrew's own loader at all.

ncdu is exactly this case, confirmed by downloading its untouched bottle straight from ghcr.io and inspecting it directly: its interpreter is the plain system path, not a placeholder. Blanket-adding Homebrew's glibc directory to its RPATH anyway put two independently built, ABI-incompatible copies of libc.so.6 in reach of the same process — which failed with an internal symbol-versioning error at startup, rather than the harmless unused search path an unnecessary RPATH entry usually is. The fix: only add glibc's directory unconditionally when a file's own interpreter is actually being repointed at Homebrew's loader — the same condition the old, pre-consolidation code used, restored rather than reinvented.

Where this genuinely can't reach

Not everything a Homebrew bottle does is expressible as a placeholder token or a load-command entry, and a small number of cases are honestly out of reach for this mechanism, not merely unhandled yet:

  • Ruby's own VM bootstrap bakes a literal, non-placeholder absolute path directly into libruby.dylib — confirmed with strings on the extracted library. There's no token here to substitute; the path is simply wrong once relocated, so ruby and its one dependent, lolcat, are left out of the index rather than shipped broken.
  • neovim's lpeg binding calls Lua's require(), which dlopen()s a literal /home/linuxbrew/.linuxbrew/opt/lpeg/lib/liblpeg.so baked into the compiled binary as a runtime string constant — not an ELF NEEDED entry or RPATH search, so patchelf has nothing to rewrite. This one happens to work on a real Homebrew install purely because Homebrew's official Linux prefix genuinely is that exact path.
  • A handful of runtime librarieslibstdc++, libgcc_s, libffi, libxml2, libpcap, libbz2, libexpat, libtiff, libldap, libgomp among them — aren't declared as a dependency by the affected formulas' own Homebrew metadata either. A minimal machine without them fails the exact same way under real brew install; this isn't a gap in hop's relocation, it's a gap in what the bottle promises to provide.

Python needed a mechanism beyond relocation entirely rather than falling into either category above: its own sys.base_prefix is baked in at build time the same way Ruby's is, but unlike Ruby, hop ships a working fix for it — a small wrapper script that sets PYTHONHOME (and restores the calling formula's own site-packages via PYTHONPATH) before handing off to the real interpreter binary, transparent to every existing reference to it.