Yup, I had read that. Bazel is very different than something like a bundler. (I believe Pants and Buck are very similar, if you’re familiar with them) but I understand your point. This was just a hard requirement for someone I was trying to help introduce Rescript into their company’s monorepo.
Yeah kinda familiar with them. In fact I had to explicitly reject proposals to integrate with some of these based on a few criteria. The curse of selling robustness and speed is that we have to ensure that there isn’t a bottleneck somewhere else anyway, like in a wrapper build (or userland library or whatever else).
So was there no way to just run bsb one-shot?
I tried to make that happen because yeah… I knew doing anything too custom would just cause pain. But we got stuck with Bazel having some strict requirements for the output location of build files. Essentially (if I’m remembering this correctly, it was a few months ago), Bazel provides you a location and you have to build into that location. Any files built in your project’s directory are ignored (as a project is only considered input, not output).
1 Like
Does Bazel deal with js? Suppose you always check in generated code, do you need an abstraction layer? Note we recommend to check in generated code
1 Like
Yes, it actually has a fairly good system for dealing with just JS. I know that’s the recommended Rescript pattern, so that was my first suggestion. Unfortunately, hit two issues:
- The build/release engineers just wouldn’t allow it. They brought up worries about security, blessed dependency management, compatibility worries, and a few other things. I explained there wouldn’t be any compatibility issues… because JS is JS and the native code is just in node_modules at runtime… but overall it was a tough sell.
- The team really didn’t like the idea of reading both rescript and JS for every Pull Request.
(I have to reiterate that this wasn’t my company. I just spent a long weekend helping someone on a small team in a big company try to get started integrating ReScript.)
I believe two things would have gotten them past “Go”:
- something like a config option to specify the location of “lib” on the machine (outside of the source code)
- the ability to run
rescript buildfrom anywhere on the machine, and point to the project directory (I believe this is just a Bazel choice. They only execute commands from the root of the monorepo.)
I have no idea the cost of that, so I’m not suggesting it, it’s just what would have helped in the Bazel situation. Also, our initial implementation before dropping using bsc, was based on GitHub - ostera/rules_reason: 📐Reason/OCaml rules and tools for Bazel, which uses bsc and I know some people still use.
The build/release engineers just wouldn’t allow it. They brought up worries about security
I am completely lost. Is not it way more secure to check in generated code to avoid running native tools on the production server? Imagine babel’s dependencies are hostile, it is really hard to detect given such dependencies are huge. Note we are doing a slightly better job where the compiler is self-contained, but it is generally a better idea to avoid attack surface.
- The team really didn’t like the idea of reading both rescript and JS for every Pull Request.
This is not a bad thing, since if you do refactoring, and the generated js are almost the same, much more confidence. But it is configurable, most review tools can detect generated code with some work.
The ability to run
rescript buildfrom anywhere on the machine, and point to the project directory
We can do some work to let rescript to figure out the root project, feel free to raise an issue.
We can also add support to flatten the js output directory to make it easily monitored. We can do some work to make rescript work better with bazel, but it is definitely not recommended to integrate bsc with bazel.
To make it clear, integrate bsc with another build system is not supported, you are pretty much on your own and we may change bsc flags which break your work.
I am completely lost. Is not it way more secure to check in generated code to avoid running native tools on the production server?
I’m reaching the limits of my memory on the situation, but I’m 100% certain the production servers don’t build. The CI/Build servers (with security-approved dependencies, tooling) build and then put it in a docker container which does nothing but node server.js… it’s a common pattern. This was company was in the finance area, so I image they have stricter controls than most.
To make it clear, integrate bsc with another build system is not supported, you are pretty much on your own and we may change bsc flags which break your work.
I/They never assumed it was supported. I just tried to help get someone who was excited about rescript, but had very constrained build conditions, onboarded. We tried using bsc to get around the constraints, so I thought it would be worth bringing up here.
I use bsc to generate .cmt files on the fly from stdin. I use them to read signatures and augment code with annotations like here (changes in text editor are buffered, they are not present on disk):
3 Likes
I’m sure this is already known, since I found out from forum posts, but in any case. I’ve been using bsc -i lib/bs/src/Cli.cmi to help me generate interface files to tighten module APIs.
Re-typing or fishing to copy paste the signatures can get kind of boring.
3 Likes
This is something we want to support. We came across similar situtations in the vscode plugin( cc @cristianoc )
I am glad that you went that far.
What’s your current architecture?
How do you figure out those command line flags currently?
Is your plugin written in OCaml? How do you consume those cmt files?
Yes, this will be supported. I have a vague memory that the editor support will also ship this feature (cc @chenglou )
1 Like
The editor calls bsc on the .cmi file.
Once this is available as main command, it can be used instead.
2 Likes
Plugin code is in java.
I read the ninja build file and try to extract correct flags for temporary compilation (using different paths for ex).
It’s empirical knowledge, but it works for now.
I also have to keep different implementations that depends on the version of bs, ninja format have been changed recently.
I am completely lost. Is not it way more secure to check in generated code to avoid running native tools on the production server?
it’s generally avoided in a monorepo world to commit output artifacts (for rescript, it’s javascript) to the repository. I think this simplifies the DevX for a project: it’s easy to formalize the automatic build, and easy to onboard people from other language.
But I see @Hongbo’s point of adding rescript incrementally to a code-base as the justification to just committing JS.
To make it clear, integrate bsc with another build system is not supported, you are pretty much on your own and we may change bsc flags which break your work.
Yeah to add to this point, I’m well aware of this, most Bazel integration of a language would anyways pin down the exactly compiler using SHA hash of the repository having the compiler. So that helps with reproduciable builds, creating a frozen compiler to avoid incompatibility of flags, and achieving some added security, supposedly. If compatibility issue arises, most likely, we have to update the build rules for the new language.
@Hongbo
All points aside, I think the difficulty I’m dealing with interfacing rescript instead of bsc was that rescript is a compiler + a build toolchain. It deals with the project at a bigger scope ( handling dependencies from package.json, handles intra-module dependencies).
I wanted bsc because it exposes an interface that deals with individual module. Just a compiler (bsc) fulfills that needs. If we can expose a stable granular API from rescript, then I don’t have to go down that rabbit hole. (Kinda similar to the C/C++ situation, I think as @Hongbo mentioned somewhere in this forumn)
@johnmegahan GitHub - iocat/rules_rescript: Bazel Rules for Rescript This is the bazel build rules that I’ve been working on by the way.
1 Like
are you available some time for a chat? would be happy to figure out what is needed. Feel free to DM or send an email
I wanted bsc because it exposes an interface that deals with individual module.
There is a significant difference between bsc and babel.
bsc needs to compile its dependencies before itself, there is no such requirement for babel. So you would see a lot of babel plugins for webpack, but this model does not work with rescript.
If we can expose a stable granular API from
rescript
let me know if you have some concrete ideas
Hi thanks!
There is a significant difference between bsc and babel.
I think I’m lost here. We’re discussing bazel integration, not babel. (TBH I don’t know babel all that much to judge the similarity).
If we can expose a stable granular API from
rescriptlet me know if you have some concrete ideas
I can expand on this, but I think you have more build internals knowledge to give the judgement whether it’s possible. And I can be completely wrong:
rescript compile ast ModuleA.res --out ModuleA.ast
rescript compile iast ModuleA.resi --out ModuleA.iast
rescript compile cmi ModuleA.iast --out ModuleA.cmi
rescript compile cmjjs ModuleA.ast ModuleA.cmi --out ModuleA.cmj --out ModuleA.js
rescript compile cmicmjjs ModuleA.ast --out ModuleA.cmi --out ModuleA.cmj --out ModuleA.js
rescript compile cmicmjjs ModuleB.ast --out ModuleB.cmi --out ModuleB.cmj --out ModuleB.js --deps dirA/ModuleA.cmi --deps=dirA/ModuleA.cmj --deps=dirA/ModuleA.js
It’s essentially a draft of a low-level API that most normal users wouldn’t need, but tool developers might appreciate.
1 Like
Ah, yes, I should have called out Bazel more clearly.
Oh wow, your work looks much more up-to-date. Thanks for sharing! Seems you have a much better grasp of the workings of Bazel than I do. I did have a play around and was able to get fairly basic things building with bsb… but it’s very limited.
snapshot testing of graphql-ppx for regression in the compiled output.