[ANN] ReScript Retreat Roadmap Summary 2024

Hello everyone!

As you may already know, we had our ReScript Retreat 2024 event from 22-25th of May in Vienna to work together in person and sync up on our processes.

There we discussed our goals and ambitions for the next major release ReScript 12 (and beyond).

Since this is relevant for the broader community, we wanted to give you an overview of what to expect for ReScript 12.

At this point we also want to thank our ReScript Retreat sponsor cca.io for providing us funding for the amazing event location! Also special thanks to @cknitt and @fham doing the big lift helping in organizing this event. A blog post about the event will follow soon.

1) Make rescript-core part of the compiler

Owner: @zth, @bloodyowl

rescript-core is considered stable enough to be part of the rescript compiler itself. It will replace all the competing standard library like modules, such as Js, Js2, Belt and the modules inherited from the OCaml stdlib (including, to some extent, Pervasives) to finally align on one streamlined API. Nice side-effect there is that the full API surface will be in ReScript syntax, so no ominous .ml files to jump to when looking up standard library functions.

Compatibility packages for the old stdlibs will be provided. Belt will continue to be maintained in a separate package.

We won’t go into details, but the main challenge here is switching compiler internal modules and tests to the new uncurried conventions.

2) Integrate rewatch as the new ReScript build system

Owner: @jfrolich @rolandpeelen and @cknitt

rewatch is an alternative build system for the ReScript compiler built in Rust. Contributors and power users had the chance to give rewatch a serious spin for their production codebases and it was deemed to be a superior solution over the original Ninja based build system that currently ships with ReScript (both in build times for incremental builds, as in feature wise e.g. with better monorepo support).

Our plan is to make rewatch the new ReScript build system that is responsible for calculating module dependency graphs and to efficiently compile modules. It will replace ninja and bsb and will help us build new capabilities such as faster/better monorepo support which was not as easy to do with the previous solution due to design constraints.

Once again, we are going the route of “vertical” integration by maintaining a tool that is custom tailored to our needs. This change is planned to not cause any serious breaking changes and users will continue to build their projects with rescript build and rescript build -w as always.

Depending on the usage patterns in the community, we may be thinking about removing the "in-source": false configuration within rescript.json (putting build artifacts into the lib/js folder), but this is still up for discussion and depends on how much of a breaking change this would be.

3) First class RegExp syntax

Owner: @glennsl and @Maxim

ReScript will finally get proper Regular expression syntax that will be similar / the same as in JS:


// Before

let r = %re("/b/g")

// After

let r = /b/g

A WIP branch was pushed during our retreat.

4) Better for/of and iterator support

Owner: @cometkim / @cristianoc and @JonoPrest.

ReScript will receive new capabilities to define and iterate through an iterator value, similar to JS.

Also, a builtin iterator type with corresponding bindings will be exposed for direct use.

In addition, there will be first experiments implementing a continue and break statement for loops to make the usage of loops more ergonomic. The first MVP will compile a break statement into a try/catch block, throwing an error at the break point.

5) Remove curried mode

Owner: @zth / @cristianoc

In ReScript 11, functions are uncurried by default. There’s still an option to opt into curried mode, which was originally introduced to not break any existing codebases and allow users to gradually migrate app and library code to ReScript 11’s uncurried semantics.

In ReScript 12, curried mode will be removed, which will be considered the biggest breaking change for this release. Removing curried functions will greatly reduce the complexity of the language, both on the user side and on the tooling side.

6) Remove .ml compilation support

Owner: @zth / @cristianoc

Given the fact that curried mode will be eliminated, .ml code will not be able to compile because OCaml only knows the concept of curried functions. Moving forward, ReScript 12 will drop support compiling any .ml files to JavaScript. As a consequence, all builtin ReScript modules will only be provided in .res syntax as well, further improving the user experience from an editor tooling perspective (go-to-source, type introspection, etc.).

Even though this is considered a breaking change, this will only impact a small fraction of our user base.

7) Improve reliability of the JSX parser for custom hyphenated prop names

Owner: @mununki

ReScript 12 will come with some internal parser improvements to fix issues such as Jsx parsing error where using hyphen prop name · Issue #6723 · rescript-lang/rescript-compiler · GitHub.

8) JSX: Use abstract types for React components (maybe)

Owner: @mununki

This is a lower level change in how React components are represented in the type space. The details why this change is needed can be found in this PR discussion.

This is considered a breaking change, although this will probably not impact many users.

9) JSX: Introduce preserve mode (maybe)

Owner: @mununki

This feature is mainly about preserving the original JSX in the compiled output, without applying any transformations. This feature should enable easier integration in non-React UI frameworks that rely on JSX.

10) Improve TS types output via genType

Owner: @cometkim / @cristianoc

The goal is to emit types that are more idiomatic to TypeScript users, including the generation of d.ts files.

11) Drop bs- prefixes in rescript.json

Owner: @cometkim

After our initial migrating from bsconfig.json to rescript.json, there were still some bs leftovers within the config file itself. In ReScript 12, this will be fixed by dropping all legacy bs- prefixes. For compatibility reasons, the deprecated names will still be supported.

12) First class embedded language support prototype

Owner: @zth / @jfrolich

As a first step towards better meta programming support, we’ll start experimenting with a new builtin mechanic that tackles the “embedded language” problem, such as embedding graphql / css / edgedb queries etc. in your ReScript code.

The solution will be based on @zth’s rescript-embed-lang syntax extension.

13) Misc: Aligning Documentation, VSCode marketplace, Governance

Owner: @ryyppyy / @fham

  1. The rescript-lang.org repository will be moved from the rescript-association to the rescript-lang org. This will also require us setting up a new public Vercel organization to hopefully allow easier collaboration and easy deployment of contributor branches.
  2. The rescript-association.org website will be migrated to rescript-lang.org for better discoverability and more official character. We’ll also think about better solutions to allow easy donations (via stripe, GitHub sponsors, etc.)
  3. The VSCode extension will be moved from the original chenglou92 organization to the actual official rescript-lang handle. This will include using the newly introduced VSCode extension deprecation mechanism to allow a seamless transition.

I hope this gives you a good idea on where we are heading, and we are looking forward for the big upcoming changes such as the new ReScript build system and the new builtin Core library that will ship with rescript@12.

As always we’ll make sure to keep the breaking change to a minimum and will give enough time to let you try release candidates and provide feedback to allow us to have a smooth transition phase.

Happy hacking!

27 Likes

Hi, I am working on the IntelliJ IDE plugin and after reading the roadmap, I am worried about the migration from ninja to the new rewatch build system.

all Jetbrains editors are buffering the code updates in memory, and they are not flushed to disk immediately.

To be able to have fast feedback on errors, the current workflow is that I copy the current edited file to the disk in a temporary folder.
Then I can run a ninja build command on that specific folder with all the directives or -I from the rules that I extract from the ninja build file.
It works well and I can have faster errors for a much better user experience.
It is also nice because ninja is so small and fast that I can launch a process each time and I don’t have to maintain a background process.

I have no idea what the rewatch build system is, but I am worried that such a workflow can’t be possible in the future. Not being able to process one file on demand would lower the user experience a lot.
Would that be possible ?

also, I am a big fan of insource=false.
We already have a lot of reason/rescript files and I like to hide the js files in a lib folder so that the number of files to look at are not duplicated.

2 Likes

I would like to echo that I also would be in favour of having a way to transform a ReScript file to in-memory JavaScript. I mentioned this in Tighter Vite integration · rescript-lang/rescript-compiler · Discussion #6753 · GitHub, where I hint at an optimized Vite integration.

@hgiraud are you able to pull this off today using ninja?

using a stream instead of writing to disk would be perfect, but I.m not doing that, I have to write updates to disk.

Would be worth opening a dedicated thread for IntelliJ support. Feels like you should also be able to use the tools that the VSCode extension is using to provide code completion etc.

Also, the new build system will accommodate to whatever we need, so maybe there’s a way to expose the data you need through different mechanisms / interfaces.

Feel free to hit me up in a DM and we can figure out how to solve that in the IntelliJ integration. Does it already use the language server?

Nice, I will DM you.
the plugin is not using LSP at all, parsing and resolutions are done purely with java and jetbrains SDK.
But syntax/type errors are delegated to the compiler.

I run my TypeScript projects with outDir set, on a large codebase in-source compilation can get unweildy. It would be nice if this option was retained.

2 Likes

I would like if this wouldn’t happen, it will break all usages of styled-ppx and it’s currently the only way I can do to share the same CSS definitions and small runtime from Melange and ReScript. There’s also other cases such as atd, where atdgen generates OCaml code used from both ends.

Probably other cases as well, probably not direct applications using OCaml syntax with ReScript, but some dependant libs are.

I respect the unapeal of OCaml, and also why you would want to discourage/break with curry, but this seems like another breakage that makes the authors of those libs over-complicate out lives for keeping a feature that used to work ok.

3 Likes

TLDR; We could leave ml compilation in for a little while longer but it would break anyway because ml code can’t express uncurried mode.

Some background:
To be clear, it’s not about finding OCaml unappealing, it’s about a unified experience where everything you interact with is in the ReScript syntax, and by that can take advantage of all of the work that has been done and will be done with new features etc. Get a unified experience with editor tooling, and so on. This is very important to us and we’ve made large efforts to unify everything under the ReScript syntax.

In v12+ there won’t be a way to write code in ml that compiles in uncurried mode, since most of uncurried mode is at handled by wrapping things in the AST with constructs that can’t be expressed in regular ml syntax. The way it’s handled in ReScript is in the parser itself, which is partly why we want everything to use ReScript syntax - uncurried mode comes for free.

So there’s really no choice. We can leave ml compilation in v12+ but most things around it would break anyway because it doesn’t parse the code to uncurried mode.

Styled-ppx is awesome and I know you’ve put a lot of effort into supporting both ReScript and Melange for it, and I really appreciate that (which I hope you know). We obviously don’t want or like making life harder for maintainers of popular libraries in the ecosystem. But we do have a clear goal with where we’re taking ReScript (which I hope we’ve been clear about) and it’s about the unified and vertically integrated experience. Uncurried mode and the ReScript syntax is very important pieces of that vision, where we’ve decided that the trade off is worth it. And we don’t take that lightly, we know how much of a change uncurried mode is for the ecosystem.

As for the specifics, my immediate thoughts for solutions would be:

  • For styled-ppx, even though it’s additional maintenance burden, maybe maintaining separate ml and res code is acceptable? I don’t know if you’re planning to support uncurried mode in styled-ppx itself though?
  • Atdgen is really cool and I know there are a few who still use it with ReScript. But, the real solution here now that ReScript is its own language is to implement a dedicated ReScript mode for atdgen, just like there’s a mode for Java, TypeScript, etc. That will be also be better in the long term because it’ll be able to take advantage of ReScript specific features going forward. As with most other use cases this is something we can hopefully do a community effort to support if there are enough users.

I hope this post made the decision a little clearer.

6 Likes

About ATD, I’ve been a big user for quite many years in both OCaml and Rescript, but the situation feels a bit suboptimal now and some of the rescript users would like to have a dedicated mode for rescript and I think it would totally make sense now that we have optional fields in ReScript for example. The (de)serialization would also be much more efficient using the new JSON bindings allowed by rescript untagged variants and using arrays instead of lists.

I totally understand how complicated the situation can be for you now but as zth said, it’s really not a decision we’ve made lightly.

4 Likes

In v12+ there won’t be a way to write code in ml that compiles in uncurried mode, since most of uncurried mode is at handled by wrapping things in the AST with constructs that can’t be expressed in regular ml syntax. The way it’s handled in ReScript is in the parser itself, which is partly why we want everything to use ReScript syntax - uncurried mode comes for free.

I see. There’s a way to achieve uncurried via OCaml syntax in ReScript that worked well, via [@b]. What about using it to make those ast with it and keep OCaml then? is that out of the equation?

The code that I need to “duplicate” is indeed a pain in the ass, since are both 2k files from Css and CssJs definitions: styled-ppx/packages/css/js/Css_Js_Core.ml at main · davesnx/styled-ppx · GitHub styled-ppx/packages/css/js/Css_Legacy_Core.ml at main · davesnx/styled-ppx · GitHub

I could eventually write them in ReScript and transform it to OCaml, but its painful.

1 Like

Uncurried mode actually isn’t the same as the old uncurry semantics, it’s another thing entirely. As an example, internally in uncurried mode each function is wrapped in a variant constructor Function$ that describes its arity and the function itself. Same goes for a bunch of other things like types and so on. A lot of uncurried mode is handled at the syntax level and in the parser itself - the proper AST is constructed by the parser directly when parsing.

So there’s simply no way to express what’s needed in ml syntax. But if everything is ReScript syntax, most of uncurried mode comes for free since the parser does all the heavy lifting. No special care is needed from the user itself. This is a big part in how we’re trying to make uncurried mode as easily adoptable as possible. Outside of adapting to not use autocurrying (partial application is still a thing, just that it’s explicit), users shouldn’t really need to do anything special to adapt to uncurried mode.

Got it, will see what are the paths forward. Thanks @zth

2 Likes

Maybe there’s an older rescript version that can reprint ml as res. You can reprint Reason syntax in older versions but I’m unsure about ml, maybe that works too. If so I think just reprinting your main ml source to Rescript as needed should work.

1 Like

I wonder what are the pain points for implementing preserve mode ?