Proposal: bsconfig.json paths property

1

Why?

I’m a big fan of monorepos. ReScript has some support for them via symlinking package directories in node_modules.

The goal of this proposal is to allow you as the user to specify path resolutions relative to bsconfig.json rather than relying on symlinks.

How?

I propose that we extend bsconfig.json to include a paths property.
This property would be a series of entries which re-map packages to lookup locations relative to bsconfig.

Example

Given a directory layout:

.
├── apps
│   └── app
│       ├── bsconfig.json
│       └── src
│           └── main.res
├── bsconfig.json
├── libs
│   └── number-adder
│       ├── bsconfig.json
│       └── src
│           └── NumberAdder.res
└── package.json

With a layout like above we can define a root bsconfig.json as:

{
  "name": "root",
  "version": "0.1.0",
  "sources": [],
  "bs-dependencies": ["@app/number-adder", "@app/app"],
  "pinned-dependencies": ["@app/number-adder", "@app/app"],
  "package-specs": { "module": "es6", "in-source": true, "suffix": ".bs.js" },
  "paths": {
    "@app/app": "./apps/app",
    "@app/number-adder": "./libs/number-adder"
  }
}

Other considerations

The path mapping should only need to be done once in a repository. So given the above folder structure ./apps/app/bsconfig.json should only need to specify a bs-dependency on @app/number-adder. When parsing ./apps/app/bsconfig.json we should look up the directory tree for path mappings and find them in ./bsconfig.json. Only if no mapping is found would we start looking into node_modules.

Proof of concept

I’ve made a PR, of dubious quality, with a proof of concept here.

8 Likes
2

I think this is a great idea! Pinned dependencies can be a bit confusing so the added flexibility will help a lot. It may also help improve the rescript vscode extension which currently picks up the symlinked code when using “go to definition”.

A few suggestions:

  • paths is a bit confusing to have alongside sources. Perhaps call it pinned-sources as a parallel to pinned-dependencies?
  • Could this information be used to generate pinned-dependencies? This would remove the duplicate manual config.
  • I wonder whether it could be done as an array of paths, with wildcards, similar to yarn workspaces? The package name could be read from the bsconfig.json in each directory, it would be great to not need to specify it here as well. As a future improvement, this concept could perhaps read the path list directly out of the monorepo config (package.json for yarn workspaces or workspace.json for nx.dev).
1 Like
3

Thanks for your feedback!

paths is a bit confusing to have alongside sources. Perhaps call it pinned-sources as a parallel to pinned-dependencies?

good shout, I’m not sure about the pinned as this also works for dependencies specified in bs-dependencies. Maybe resolutions is better than paths?

I lifted the name from the similar concept in tsconfig files.

Could this information be used to generate pinned-dependencies? This would remove the duplicate manual config

Perhaps it could. I’m not sure having a path resolution specified should imply pinned though. This would need to be discussed in more detail I think. Or we can delay a decision on this until the proposal is out there.

I wonder whether it could be done as an array of paths, with wildcards. […] As a future improvement, this concept could perhaps read the path list directly out of the monorepo config

I’m sure it would be possible. I have three concerns with this suggestion

  1. I’m not sure the maintainers want to couple to a specific monorepo technology
  2. I don’t think we should advocate a specific brand of monorepo through support. E.g. pushing people towards workspaces/nx/lerna/pants/bazel or whatnot
  3. It’s fairly easy to create a script that updates these path mappings for you, meaning we can solve this in user land for now and make delay making a decision until real world usage of this feature emerges.

Example of a script that would update paths:

const glob = require('glob')
const fs = require('fs')
const path = require('path')
const root = require('./bsconfig.json')
const paths = root.paths
const configs = glob.sync('**/bsconfig.json')

for (const config of configs) {
  const name = require(config).name
  paths[name] = path.relative(__dirname, path.dirname(config))
}
fs.writeFileSync(path.resolve(__dirname, './bsconfig.json'), JSON.stringify(root, null, 2))
4

resolutions would work, but I stand by my suggestion. pinned-dependencies is the list of bs-dependencies that are to be resolved as local source folders. Quoting from the docs:

these are your local packages that should always rebuild when you build your toplevel

That seems like exactly what you’ve written here.

I guess it can be added later. I don’t want to delay this from potentially getting into v10. As I said above I think it does imply that; it’s why I made the suggestion :slight_smile:

I didn’t want to advocate specific monorepos, just support as many as possible. But fair point.

Focus on the other part of my suggestion, then, to read the package names directly from bsconfig.json. The compiler already reads the file, it might as well use it to avoid duplicating the package name.

5

Re: pinned-depdencies

I’m still very by this concept, why aren’t bs-dependencies always rebuilt? The build system is plenty fast enough to be able to do that. If that were the case this would be a non-issue.

Alas, here we are, and the concept is here. As such I’m not against your 2nd suggestion of auto generating pinned-dependencies from the path resolution map.

read the package names directly from bsconfig.json

Actually, can you give an example of what the bsconfig.json in the OP example would look like when your suggestions are applied?

6

After some discussions I think a better way to achieve this would be to use import maps.

Import-maps is a feature that is making it’s way into browsers, and is already supported by Deno, and the SystemJS module loader. It makes sense to lean on prior art here.

So what does this mean in terms of this proposal?

I think we can make import maps work by:

  1. When running rescript build the compiler looks for an importmap.json file in your cwd (alternatively we can add an -import-map=[path] option
  2. If the import map exists the imports are loaded into a cache
  3. During resolution of bs-dependencies we first check for a hit in the import map cache
  4. otherwise we use the node_modules resolution

Goals

The goals of this proposal are:

  1. To better support monorepos
  2. To allow each package in a repository to be easily published to NPM and consumed

Options considered (so far)

  1. Paths property in bsconfig.json
  2. Allowing relative bs-dependencies
  3. Import-maps

Pros and cons of the options

Paths property

{ "paths": {"@app/number-adder": "./libs/number-adder"},
  "bs-dependencies": ["@app/number-adder"]}

Pros:

  • It’s explicit
    Cons:
  • In order to make it performant enough we’d need each package to specify it’s mappings (no recursive lookup up the folder tree)
  • Need to define out handling of paths property in packages consumed from node_modules

Relative bsconfig

{"bs-dependencies": ["./libs/number-adder"]}

Pros:

  • easy to understand
    Cons:
  • In order to maintain publishability of sub-directories we’d need to supply a different bsconfig.json when publishing

Import-maps

// bsconfig.json
{"bs-dependencies": ["@app/number-adder"]}
// importmap.json
{"imports": {"@app/number-adder": "./libs/number-adder"}}

Pros:

  • Standards based
  • Centralizes location of path mappings
  • No change to bsconfig.json (maintains publishability)
    Cons:
  • Extra file in the repository

Links:

2 Likes
7

Ad extra file in the repo: The WICG examples look like valid JSON to me. Would it not be possible to embed that in the bsconfig as well? Maybe it could be part of bsconfig AND work as a standalone file.

Although, personally I would not mind an extra file if I get a simpler monorepo setup for it.

1 Like