Platform support and prebuilt binaries/postinstall scripts

Development
#1

Currently, as per the install script, the supported platforms for ReScript are: macOS, Linux, Windows, FreeBSD and OpenBSD.

With the recently completed CI work, we will be able to provide prebuilt and tested binaries for

  • macOS Intel
  • macOS ARM
  • Linux x86-64
  • Windows x86-64

On postinstall of the rescript npm package, the install script mentioned above checks if a prebuilt binary is available. If not, it will try to build from scratch:

  • Ninja (from vendored sources)
  • OCaml (from vendored sources)
  • The ReScript compiler binaries

IMHO building from source in postinstall for “exotic” platforms (for which no prebuilt binaries are available at the moment) has the following disadvantages:

  1. Installation of the rescript package takes a long time.
  2. We can not be sure if it actually works. I do not know if anyone is regularly testing installations for e.g. FreeBSD or OpenBSD. In my experience, if something is not tested by CI, there is a high likelihood that it has already stopped working some time ago.
  3. Even if still working in principle and even though only (I think) a C compiler, make and python are required as prerequisites for the postinstall script, there is a good chance that the build/installation may fail on some machines because of a local setup issue.
  4. Due to the imposed limitations (minimal dependencies), the install scripts IMHO bring a lot of complexity/restrictions to the project setup. E.g., we cannot fully move to a dune-only setup for building the compiler binaries if the install scripts are not using dune, but a homegrown ninja setup instead.

So my idea was:

  1. Let’s find out which platforms the community actually wants/needs us to support in addition to the ones CI is already building binaries for. E.g., I think there was some interest in Linux ARM. But is anyone actually using FreeBSD/OpenBSD?
  2. Get CI builds for those platforms running so that we get prebuilt, tested binaries for them.
  3. Simplify the install scripts to always use those prebuilt binaries.
  4. Be able to simplify the project setup without being restricted by the postinstall process.

What do you think?

Which platforms are you actually using or planning to use?

  • macOS x86-64
  • macOS ARM
  • Linux x86-64
  • Linux ARM
  • Windows x86-64
  • Windows ARM
  • FreeBSD
  • OpenBSD
  • NixOS
  • Other (please specify)

0 voters

6 Likes
#2

We should consider prior art here. How does darthsass handle this?

#3

I use ReScript on NixOS, which requires either building the compiler from source (very slow) or wrapping npm/yarn to run a custom post-link script to fix the shipped binary.

#4

I guess the question is: if your platform is not supported, hence you need to build from source, when would you like to build? Currently, this happens automatically on npm install, and would take several minutes.

I guess the current scheme is justified if the value provided by the current scheme, compared to people organising their own builds (takes 3 or 4 commands manually or in a script), offsets the costs of maintaning a more complex setup for the compiler.

#5

I’d imagine the kind of relevant feedback for this would have the form: "if the auto build from source on npm install were to disappear, this would be a problem for me because…`.

#6

What exactly needs to be patched/fixed in the shipped binary for NixOS?

#7

I don’t understand enough about it to explain, but for instance:

# before
❯ file ./node_modules/rescript/linux/rescript.exe
./node_modules/rescript/linux/rescript.exe: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=4e0d8d85f85ba4e3ac000f9a9671c088d2a000cd, stripped

# fix
❯ nix-shell -p autoPatchelfHook stdenv.cc.cc.lib --run "autoPatchelf ."
automatically fixing dependencies for ELF files
searching for dependencies of ./node_modules/rescript/linux/bsc.exe
searching for dependencies of ./node_modules/rescript/linux/rescript.exe
searching for dependencies of ./node_modules/rescript/linux/refmt.exe
searching for dependencies of ./node_modules/rescript/linux/bsb_helper.exe
searching for dependencies of ./node_modules/rescript/linux/ninja.exe
  libstdc++.so.6 -> found: /nix/store/dd8swlwhpdhn6bv219562vyxhi8278hs-gcc-10.3.0-lib/lib/libstdc++.so.6
setting RPATH to: /nix/store/dd8swlwhpdhn6bv219562vyxhi8278hs-gcc-10.3.0-lib/lib

# after
❯ file ./node_modules/rescript/linux/rescript.exe                      
./node_modules/rescript/linux/rescript.exe: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/65hafbsx91127farbmyyv4r5ifgjdg43-glibc-2.33-117/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=4e0d8d85f85ba4e3ac000f9a9671c088d2a000cd, stripped
#8

I think we’d have to create statically linked executables to avoid the NixOS issue.

I found this article on that subject:

1 Like
#9

Ok, now that I have discovered how polls work I have added one above. :slight_smile: