Greetings everyone! I’ve been writing rescript for quite a while now but this is my first post here. Nice to meet you
We have a very large rescript app and building it from scratch takes a few minutes to complete. Locally it’s somewhat rare that we need to rebuild everything, but we need to do it every time our CI runs.
For other long-running tasks (such as yarn install) we use CircleCI caching to speed up the process. We’d love to do something like this for our compiled rescript code, but we have a problem, which is that the generated code for the dependencies gets spread across the various folders in node_modules.
Inside of our main project, we can set "in-source" to false in our bsconfig.json, and all of the compiled files will be generated in the lib/src directory. But that only covers the files in our project, not the dependencies, which means there’s no point to caching that directory because without the dependencies everything will have to be rebuilt anyway.
So long story short: is there a way to extend in-source: false to cover not only the current library but also all of the dependent libraries, such that everything the compiler outputs gets put into the same directory so we can cache it?
Hmm perhaps the problem is that we’re caching our node_modules before we run our rescript build, maybe if we ran it afterwards we’d benefit from the correct caching behavior. Worth a shot!
Well unfortunately rescript still builds everything from scratch even when we build before caching node_modules. I also added the lib folder to the list of cached folders, but it still rebuilds everything. Hmm. I wonder what rescript is using in order to determine cache busting.
Do you use the default builder or rewatch? I guess the default builder if you use in source false. Would be worth trying with rewatch if you get any improvement. I hope some contributors that have a better understanding of the builder of rewatch could chime in like @cometkim or @rolandpeelen.
I’m using the default builder, we’re on a somewhat old stack, bs-platform v9. We’ve been blocked from upgrading due to various dependencies in our app that have conflicts with the latest version. Perhaps a later version would solve this problem for us?
Well yeah you could try on a sample project if the latest versions or rewatch improve the situation, but I’m pretty sure you could achieve a better performance with a good caching strategy.
That was my thought as well, I just need to find what the “good caching strategy” is Maybe some others here have figured out how to cache builds for their apps. Of course for a lot of apps the compiler is so fast that it’s not really necessary.
are you sure that all the compiler intermediate files are indeed cached and not ignored by some ignore file?
I don’t really have the need for that anymore, but I know we managed to have really fast rescript builds on CI in my previous company by creating a base docker image with the node modules.
If you use rewatch and a monorepo setup, your compile time should already be lowered by quite a lot, then you can start caching all rewatch temp files to only rebuild the packages that changed, you can likely go down to a few seconds only for the build part itself.
I’m pretty sure that they’re cached. I’m instructing circle to cache node_modules and lib folders after we complete the build. Perhaps the problem is that, if the cache logic uses timestamps, the timestamps are not correct after the cache is restored; for example, if the logic expects that the .bs.js files should have later timestamps than the .res files.
I briefly tried using rewatch but unfortunately ran into a bunch of errors with reason dependencies, which it appears that rewatch isn’t able to handle. Bummer. Then again, rewatch also requires rescript rather than bs-platform, and as I mentioned we’re still stuck on an older version.
to be honest, things should just work out of the box, including the caching, but if it does not, there’s no way anyone will invest any time trying to fix bugs in a version of the compiler that was released years ago. My first recommendation would be to upgrade. You can just convert the reason syntax to rescript with bsc, then the upgrade should be really straightforward as long as you keep uncurried to false. You’ll then have some time to convert to uncurried until v12 is released.
At some point you should get rid of those dependencies, if you want to stick to ReScript.
I just found out that @dsiu forked Relude to ReScript. Maybe he could help out here.
It could at least make it possible to upgrade to ReScript 10 or 11 (with uncurried mode disabled), but I would strongly recommend you to stick to the official standard library.
I’m not sure if you thought I had but I never asked or expected anyone to invest the time to fix bs-platform v9.
Upgrading is pretty much my only option it seems. I’ve been spending the past week or so trying to update. It’s not easy at all, unfortunately. After I went through and compiled various dependencies to rescript, I thought it would be as simple as setting uncurried: false in my bsconfig, but that breaks the dependencies which are uncurried-compatible. I don’t know why uncurried is a global setting and not per-bsconfig.json (now rescript.json!).
Anyway upgrading is a huge hurdle, there’s a ton of work to do to get everything working again. We have about 200k lines of rescript code in our codebase, not to mention in dependencies (funny that @fham mentioned relude, I had to completely rewrite one of our libraries to get rid of that one because I figured there’d be no way I could get it to work uncurried…).
We kind of slept through the whole discussion about curried/uncurried that happened a few years ago on this forum, it’s funny, practically no one mentioned in that discussion how this change would break thousands of modules… but what’s done is done. Slowly but surely we’ll get our code where it needs to be to upgrade.
Maybe some others here have figured out how to cache builds for their apps. Of course for a lot of apps the compiler is so fast that it’s not really necessary.