RFC: Monorepo support

This post is to specify the possible monorepo structures that ReScript should support. As a first step, let’s discuss where the rescript binary should live or how we can assume where it is found by tooling such as rescript-vscode and other editors.

Binary path

The latest rescript-vscode release unfortunately broke resolving the rescript binary path for npm or yarn workspace projects (also called monorepos). While we at least also have a workaround with the new VSCode setting rescript.settings.binaryPath, I think standard workflows should still work out of the box.

To make this functionality more reliable in the future, let’s specify where the binary path(s) should be. Currently, it is installed into node_modules/.bin. Your monorepo setup might be more complex, so please comment, especially if you use some of the more sophisticated monorepo management tools like lerna, turborepo and so on.

Sidenote: The aforementioned bug has been fixed already, at least for the simple case of yarn workspaces.

7 Likes

Some things that came to my mind regarding the binary path lookup:

  • What if I have rescript installed globally and not in my project? Then I guess it will not be found and I need to specify the path manually?
  • The current implementation walks up the filesystem tree until it either finds a node_modules/.bin or reaches the filesystem root, right? I think it should not go up any further than the directory that the user opened in VS Code.
1 Like

Additionally, if support for multiple rescript versions in the same monorepo is a goal, what if packages have different versions specified?

I’m using yarn v3 workspaces right now. To my knowledge, looking for node_modules/.bin will work for everything but plug n’ play modules?

Ping @ostera (IIRC you’ve been talking about monorepo related things before)

Previous versions at least worked together with https://github.com/reason-seoul/yarn-plugin-rescript. I did not test it with the newest version though.

We are using turborepo with pnpm. rescript is installed separately for every application, and a shared code moved either to public npm modules, or a package, that’s connected as "files": [../../packages/rescript-shared]. It’s done this way to enable watch mode without starting a separate rescript compiler for the package.

1 Like

do you use rescript-shared as pinned dependencies in the apps?

do you run all dev at once ?

do you have the issue in VSCode when you close a terminal pane it doesn’t kill the process running by turborepo?

do you have to sacrifice the Dev experience of the terminal logs?

No, because the watch mode doesn’t work with pinned dependencies.

We run rescript compiler for every application separately.

Turborepo is mostly used for production builds, or building application dependencies before starting nextjs in dev mode. We run rescript compiler separately.

That’s the reason why we run rescript compiler separately, so we have nextjs, rescript and relay logs in different terminals.

1 Like

interesting.

I was running a similar setup but with pinned-dependencies, until one day I had to do a hot fix and I couldn’t update the pinned dependencies. So out of frustration I removed turborepo and stayed with pnpm workspaces. Still have issues from time to time with the updating of dependencies but not that bad after running pnpm i and rescript clean.

I might give turborepo another chance although I’m not sure how much of a benefit it is to add that dependency to my stack considering that I develop without a team.

I’m a big user of monorepos with Yarn. I’m not actually aware of any things currently not working in such a set-up.

A public example is my Twitch overlay (currently on an unmerged branch, hopefully Soon :tm: updated on main).

Beyond that I have a private repository for our product front-end and we recently converted the project I’m working on with @zth to a monorepo as well (code for that is not yet public).

1 Like

What if I have rescript installed globally and not in my project? Then I guess it will not be found and I need to specify the path manually?

I think the best way for the VSCode extension to locate the rescript binary is to ask the package manager that’s in use?

For yarn:

$ yarn --version
3.2.1
$ yarn bin --json 
{"name":"npm-run-all","source":"npm-run-all","path":"/Users/alexander/Projects/front-end/node_modules/npm-run-all/bin/npm-run-all/index.js"}
{"name":"run-p","source":"npm-run-all","path":"/Users/alexander/Projects/front-end/node_modules/npm-run-all/bin/run-p/index.js"}
{"name":"run-s","source":"npm-run-all","path":"/Users/alexander/Projects/front-end/node_modules/npm-run-all/bin/run-s/index.js"}
{"name":"bsc","source":"rescript","path":"/Users/alexander/Projects/front-end/node_modules/rescript/bsc"}
{"name":"bsrefmt","source":"rescript","path":"/Users/alexander/Projects/front-end/node_modules/rescript/bsrefmt"}
{"name":"bstracing","source":"rescript","path":"/Users/alexander/Projects/front-end/node_modules/rescript/lib/bstracing"}
{"name":"rescript","source":"rescript","path":"/Users/alexander/Projects/front-end/node_modules/rescript/rescript"}

For npm:

$ npm --version
8.12.1
$ npm bin
/Users/alexander/Projects/front-end/node_modules/.bin
$ ls -al `npm bin`/rescript
lrwxr-xr-x  1 alexander  staff  20 Jul 25 10:37 /Users/alexander/Projects/front-end/node_modules/.bin/rescript -> ../rescript/rescript

Similarly pnpm

$ pnpm bin       
/Users/alexander/Projects/test/node_modules/.bin

Similarly to get the global folder containing binaries Yarn supports yarn global bin and NPM and PNPM both support passing --global to the <exe> bin command.

While calling the package manager may be considered slow, this only needs to happen once after the lock file (package-lock.json or yarn.lock) changes and can be cached otherwise.

The package manager to use could be read from the packageManager in package.json (if the repository is using Node’s corepack). Alternatively it can be guessed based on the presence of a yarn.lock or other lock file.

I also wanted to test what this would look like in a repository using Yarn PnP. It turns out that if we actually use the package manager, this will just work :partying_face:

$ yarn bin --json
{"name":"bsc","source":"rescript","path":"/Users/alexander/Projects/test/.yarn/unplugged/rescript-npm-9.1.4-75c65d01e7/node_modules/rescript/bsc"}
{"name":"bsrefmt","source":"rescript","path":"/Users/alexander/Projects/test/.yarn/unplugged/rescript-npm-9.1.4-75c65d01e7/node_modules/rescript/bsrefmt"}
{"name":"bstracing","source":"rescript","path":"/Users/alexander/Projects/test/.yarn/unplugged/rescript-npm-9.1.4-75c65d01e7/node_modules/rescript/lib/bstracing"}
{"name":"rescript","source":"rescript","path":"/Users/alexander/Projects/test/.yarn/unplugged/rescript-npm-9.1.4-75c65d01e7/node_modules/rescript/rescript"}
$ /Users/alexander/Projects/test/.yarn/unplugged/rescript-npm-9.1.4-75c65d01e7/node_modules/rescript/rescript -h
Available flags
-v, -version  display version number
-h, -help     display help 
Subcommands:
    build    
    clean
    format
    convert
    dump
    help
Run rescript subcommand -h for more details,
For example:
    rescript build -h
    rescript format -h
The default `rescript` is equivalent to `rescript build` subcommand

As you can see Yarn keeps an unzipped version of the command available for invocation.

2 Likes