[Help needed] ReScriptifying Belt / Js API (sync back docs)

Development
1

I am currently spear-heading a somewhat annoying task that I could eventually require help with.

We want to upstream the latest API doc texts from rescript-lang .org to the actual Belt / Js source files within the rescript-compiler repo. As soon as this is done, we will convert the actual ml / mli files to .res / resi.

This will give us following advantages:

  • Familiar syntax for core modules (when e.g. cmd-clicking on Belt.Array.map)
  • On-hover docs contain examples in ReScript syntax (instead of ML)
  • It’s a first step to switch over to markdown for doc headers. Easier to get started to write doc strings, formatting works with most editors / LSP setups OOTB. Less work for us to generate the required output for rescript-lang .org.

Here’s the pending PR: https://github.com/rescript-lang/rescript-compiler/pull/5361

As soon as we synced and transformed all the files, the next step would be the automatic extraction of all the docs via a simple extraction tool (as seen on my experimental-docgen branch on the syntax repo).

Additionally, we will need to teach the ReScript syntax on how to parse / print docstring syntax (right now we rely on ocaml.doc decorators). But this will be another separate topic to work on.

If you are interested in helping me syncing the docs, here’s how:

  • Check out the aforementioned PR
  • Take a file from the checklist
  • Find the relevant md doc file on the rescript-lang org repo (the files stored in latest/)
  • Copy all the docstring texts (including the codefences with examples) to the mli doc headers, including the general module description in the (the <Intro> semantics should be omitted for now)
  • Send a PR to the pending PR and ping me in case I am unresponsive. I will try to merge and tick the checklist as quickly as possible

Rinse and repeat for all the Belt modules, then we will do the same thing for Js. Let me know if you are keen to help!

(Note: There’s probably ppl asking why we are not reusing ocamldoc / jsdoc for the docstring syntax. We find the less-frills approach a little more appealing, in a similar vain as Go does… just let the user write docs without dictating too much format… we will try to do the rest. There were some thoughts to generate md files the same way as we generate .js files: Automatically without much ceremony on generating docs separately. Releasing docs would be as easy as checking them into your git repository. We will explore and experiment as we go.)

12 Likes
2

Update:

  • Belt apis are synced up to the .ml / .mli files.
  • Created a script that creates a shell script to convert all relevant files to .res / .resi
  • Next up: Sync the docs for all the Js.* modules from the current state of rescript-lang .org
  • Relevant PR is the same as mentioned in the original post: https://github.com/rescript-lang/rescript-compiler/pull/5361

Current Challenges:

Some modules use preprocessing with cppo

There’s currently a few files relying on cppo — a preprocessor that allows generating multiple files from a .cppo.ml / .cppo.mli file based on certain environment variables via #ifdef instructions.

This makes the job of rescriptifying specific Belt files more complex, but there are ways to work around it, such as:

  1. Check if cppo’s tokenization is flexible enough to allow .cppo.res files. If so, we can modify the cppo calls in our final jscomp/cppoVendor.ninja script so that the preprocessed output doesn’t include line directives → a syntactical construct that’s not supported by the ReScript parser.
  2. Remove the cppo facilities for Belt altogether and maintain each preprocessed variant of the file independently. This may make maintenance of the originally generated files more complex, but also allows us to have more specific comments / better tooling support (cppo files are harder to work in, since its special syntax is not supported by the parser).

There are many more files in the rescript-compiler codebase relying on .pp.ml and cppo.ml files. Not sure how many of them need more rescriptification (most likely not that many… the goal is to only rescriptify files that are user-facing).

ReScript currently doesn’t support semantic /** */ comments

As for right now ReScript ReScript doesn’t parse /** */ comments in a meaningful way, so we rely on the very unnatural @ocaml.doc("") attribute to express our docstrings.

@cristianoc started first explorations on how to fix this (see this PR). The first attempt will be to just parse and print @ocaml.doc("") attributes as /** */ comments. Later on we will think about the different scenarios where docstrings should attach to, and probably also move from the @ocaml.doc naming to something like @rescript.doc to prevent semantic collisions between ocaml / ReScript doc strings later on.\

The main goal here is to make our belt_*.res[i] files look acceptable.

7 Likes