ReScript documentation completeness

Carrying over a conversation from the Discord channel about documentation…

First of all, just want to thank the ReScript team for all their hard work putting together an amazing project. I came to ReScript from TypeScript and I just love the language. It reminds me a bit of when I first learned Ruby in terms of the “funness” of using it, it just sparks joy. So thank you :slight_smile:, I’m grateful.

I feel like the ReScript docs have a split personality. On the one hand, ReScript positions itself as a complete language rather than just an OCaml syntax. On the other hand, the docs appear to have been written with the intention of leaving out some of the more “complex” parts of that language, i.e. advanced/discouraged parts inherited from Reason or OCaml (such as phantom types, abstract types, until recently the pipe last operator). These two ideas are dissonant, in my opinion.

I’ve been working with ReScript for the past month and even as a newcomer, I’ve found myself regularly referring to the ReScript, Reason, OCaml, ReScript blog posts and old BuckleScript docs. Some things that I use in my code that aren’t covered in ReScript’s docs:

  • Phantom types
  • Abstract types
  • Polymorphic variants
  • Extensible variant types
  • Various BuckleScript decorators (e.g., @bs.deriving abstract, @bs.obj, @bs.return)
  • Lightweight FFI attributes without bs. prefix

Anecdotally, I see newcomers on the Discord channel regularly ask about syntax and the response is “oh that’s not in the docs because _______.” I also see responses from knowledgable community members expressing surprise that someone would need to go beyond the ReScript documentation, also somewhere from a core dev in a GitHub issue for the rescript-compiler repo. There seems to be a disconnect between established community members and newcomers regarding what the docs need to cover.

As a newcomer, what I’d like to see is ReScript’s docs cover every feature that’s supported by the compiler, including all the parts from OCaml (e.g., extensible variant types) that aren’t documented in the Reason docs. I get the impression that the team is attempting to limit what newcomers are exposed to in the docs possibly because the team intends to remove language features and doesn’t want users to write code that depends on them (I remember reading a response from a core dev somewhere to a docs question stating that a certain language feature wasn’t documented because it wasn’t “supported”). Or, the intention is to create a “beginner-friendly” language for JS devs to not overwhelm them.

Regarding the first (possible) intention, I could understand that concern, but if that’s the concern then there are some serious mixed messages being sent out. When the Reason/ReScript split was announced, the team made it clear that ReScript would be backward compatbile with Reason syntax, and by extension I would expect standard OCaml syntax, for a long time. So I’m left wondering what it means that certain features supported by the compiler might not be supported by the dev team (sorry, I wish that I could find the quote).

Regarding the second intention, I personally think that hiding complexity in the reference docs is counterproductive and sets the wrong tone. I’m not interested in learning a toy language, I want to have access to ReScript’s full power! Writing docs in a way that expresses that full power while also remaining user-friendly is a challenge, but that’s why technical writing is a critical skill. With the understanding that this is mostly a volunteer project, I see that the team has precious few resources to focus on documentation, but I would rather that the official docs err on the side of completeness. Honestly, I’m guessing that the typical profile of someone learning ReScript is a mid-level or senior dev who may have multiple languages under their belt and some rudimentary familiarity with functional languages. You should assume that we know what we’re doing rather than assume that we’ll feel overwhelmed when we see how much the language can do.

In an ideal world, I’d like to see three separate sets of docs: a “guide” that looks like the current ReScript docs, a “manual” that’s a comprehensive/exhaustive explanation of the language that includes every feature supported by the compiler, and an “API” docs section similar to what’s currently in place but more fleshed out. I understand that this takes time that the team may not have, which is why I’d like to contribute.

More importantly, though, writing good docs requires intention and clear knowledge of the team’s vision for the language. Because right now, as a newcomer looking at the docs, it’s unclear what the team wants ReScript to be. Is it its own language or is it an OCaml syntax? (Yes, I know it’s both, but you understand what I mean.)

6 Likes

Hi @earksiinni, thanks for your feedback, it is appreciated a lot.

Note the documentation is clearly lacking, if you find something missing, it is probably that it is not documented instead of that we intentionally hide it and contributions are always welcome.

We would like to evolve ReScript as its own language, we reuse OCaml’s type system and make it best suited to target our own runtime. We are not dogmatic to what OCaml gives us, for example, we added uncurried calling convention to make js interop better. The deviation from OCaml’s type system need be justified with huge benefit, so such changes in the type system is expected to be very slow.

Note one thing on the roadmap is that we plan to remove ocaml’s classes and objects (we will keep the structural typing though), so far the feedback from the community is that such changes are welcome

6 Likes

Hey. Thanks for the good discussion starter. I’d like to add my personal experiences related to this topic. During my “first year of ReasonML/ReScript” I had to learn following things to complete the fairly complex feature I was building:

  • Extending polymorphic variants (with all the upper/lower boundary stuff)
  • Functors (I have a huge one, in a critical position in the app, could not live without it, afaik)
  • Abstract types
  • All kinds of bindings to very weird and polymorphic Js component apis.
  • Finally really understand currying and pipe first/last (now that I do, I feel like a better programmer :stuck_out_tongue: )
  • Many other things…

From the list @earksiinni gave I was only able to live without phantom types (now that I’m aware of them, they seem cool, btw) and I don’t really know what do lightweight FFI attributes mean, so not sure about those. I also didn’t have to write my own ppx nor did I find a good use case for first class modules yet. But they all seem extremely relevant. And there might be some language features I really would have liked to know about but still don’t.

What I’m trying to say is that I feel that OCaml type system being so strict, it is actually quite necessary to have all of these tools available to be able to accomplish what one wants to accomplish. It’s probably possible to make some very simple apps without some of them, but eventually you want to deeply understand all of them.

I realize the ReScript team is small and there’s huge amount of things to do. I’m wondering if it would be a good idea to just start small and add a “placeholder” page into the documentation for all topics that need to be covered. Personally I think a page with a title, small 1-5 line syntax example in ReScript and links to blog posts (even if they’re about ReasonML/OCaml) would be better than no information at all. Based on that, it would be quite a lot easier for the community to add content into the pages, and finally the links to blog posts etc could be removed (or kept as reference). I’m guessing somebody has a vision about how the documentation should be structured, but most of us don’t, so unless that is somehow clearly communicated, contributing can be hard.

4 Likes

This is good to know and a relief. Some of the responses that I’ve seen in the Discord channel and on various GitHub issues gave me the strong impression that the team was intentionally trying to “dumb down” the docs to appeal to a wider audience. In particular, how some people have registered surprise that a newcomer would need to go beyond the current docs or that the current docs are incomplete.

An important challenge for the docs is going to be how to communicate “hey, we currently support this feature because we’re just straight OCaml + our additions, but we will be removing bits here and there.” I haven’t worked with a language before that simultaneously adds and removes from another language, and I would guess that most people haven’t, either. The structure of the docs can and should reflect this unique approach. I believe that the correct attitude is to document every single supported OCaml feature (including stuff like classes and objects) while clearly indicating which ones are going away and why.

I also believe that a comprehensive approach to documentation will spark healthy debate about which OCaml features are part of the language and which aren’t. (Writing crystallizes thought!) For that very reason, woe betide the writer who starts such a project without buy-in from all the core team members for a comprehensive approach. I can foresee a situation where people say “yeah go ahead and start writing,” then a writer–after investing many hours in the project–faces a backlash once team members start seeing that they are exposing unloved parts of the language, at which point pressure will mount to omit those parts from the docs. And then we’re back to a situation where newcomers are confused because they need features that the core team has informally decided aren’t worthy without formally communicating that fact.

To be more explicit: I want to take on this challenge, but I want to make certain that any effort I put into documentation would be taken seriously. I see too many open source projects where the docs are an afterthought and a project’s core team ends up adopting an almost macho-like attitude of “move fast and break things.” I’m not saying that’s ReScript’s attitude, I’m just saying that it’s exceedingly common and I don’t want to waste my time or anyone else’s on a dead end. Docs should be a first-class citizen, but too often they’re treated as a mere reflection of the engineering work rather than an active engagement with that work, if that makes sense.

Oooh, pick me, pick me, pick me!!! :wave::wave:

I definitely do have a vision! And I would love to contribute! @Hongbo, what’s the best way to do this? Should I open up some PR’s and eat this elephant one bite at a time? Like I said, though, I wouldn’t want to start without buy-in from the core team for my vision, which I can continue to elaborate here or elsewhere if there’s interest. (Btw, who is “the core team” exactly?)

1 Like

It’s funny that you mention this, Javier Chávarri’s blog post was a huge eye-opener for me both in terms of expanding my computer science/compiler knowledge and my understanding of ReScript as a whole. The post focused on a single feature, but that deep dive is what caused the whole language to really click with me.

I hadn’t thought of it that way before, and I totally agree with you. Maybe this is obvious to folks who are coming from ML-type languages (I’m not one of them).

I think that this is a good idea and I have two concerns:

  1. There needs to be a design solution that visibly indicates in the index which pages are placeholders. There’s little I hate more in documentation than clicking on a linking with the expectation that I’m going to learn more about something and instead I just get a TODO.
  2. There needs to be a real commitment to iterate on those pages. Projects take the placeholder approach way too often and then you get stale TODO docs for years and years. Maybe having a paragraph on each placeholder page with some “meta-documentation” that explains the challenge of documenting that feature could engender empathy on behalf of the reader while also serving as a “hot reminder” for the team to make sure that the placeholder doesn’t get too “cold” and eventually forgotten.

Hi, thanks for your time writing up your concerns!

I think the best way to clarify a situation regarding the language and its plans is by starting a discussion on the forum, instead of relying on random github issues / discord conversation (the team is not active on Discord, and we want everyone to use the forum), or drawing your own conclusions in other ways. So it’s good you opened this discussion.

I don’t personally want to see the documentation “dumbed down” as you say, but I’d also definitely don’t want to see yet another OCaml manual. It’s not really famous for its simplicity (just to point out that it is using BNF grammar to explain the syntax). We put most focus in the features that were absolutely necessary to get up and running with the language (esp with ReScript / ReactJS), before we scale up with the more advanced content.

Just so you know, we recently added back the docs for @deriving @obj here in one cohesive page.

Just to give you some context on time estimations: This page (+ making sure all examples are translated correctly for the <8.2.0 docs) took me around a day (actually a holiday) to finish, even though we had all the content available already. Double checking, getting feedback, rewriting, reordering, having merge-master-anxiety, … it takes a lot of time, even if it doesn’t look like much.

Generally speaking, it’s not like we are writing a section here and there when we feel like it… we need to really consider our writing style, think about the “public api” (we try to not break any links when refactoring the docs; header titles are used for permalinks) and we need to block dedicated days to work on different parts of the docs.

We also have a lot of “double maintenance” effort right now, because we currently try to sync our newest additions to the older docs variant in Reason syntax, because we care about our existing users who didn’t migrate yet. As long as we don’t do a “feature freeze” regarding our <8.2.0 docs, we will have a harder time moving faster in the future. We are getting to a point where we reach feature parity with the old BuckleScript docs, so when this happens, we will stop making changes to the Reason syntax docs and focus solely on new docs instead.

OSS Coordination is Expensive

You said it right: Writing docs is a tremendous amount of work, and I don’t see a point in accepting hit & run contributions for important parts of the docs (except for finding typos / small corrections). Most of our OSS contributors don’t have the time to spent huge amount of hours in one single piece of text, and I regularly got requests from potential contributors that they want to do some tasks, where I spent a lot of time writing up the concrete steps, and then the person in question doesn’t follow up with the actual contribution. It makes me feel I am wasting my time trying to coordinate these kind of efforts, but I also don’t expect anyone to work for free, so it’s understandable if ppl don’t follow up in the end.

To sum it up: There’s a lot of things happening behind the curtains, since we are not a “fun little OSS project” anymore.

The Complexities of our Situation

Not even if I had infinite time and resources, would I ever attempt this. Did you have a thorough look through the OCaml manual or the ReScript source code? I admire your enthusiasm on the topic, since I started out with the exact same opinion when I created the reasonml.org project (a community project attempting to document the whole Reason ecosystem, so it’s completely independent from OCaml resources, that luckily later on turned into rescript-lang.org), we even had some ideas on automatically translating the OCaml manual to Reason flavoured markdown back then.

This is all irrelevant now, because we don’t focus on native anymore.

Official doc writing is also politics (that’s just how it is). I think the easiest way start writing content (and actively contributing to the community) is by writing blog posts or other independent content. Heck, even I need to publish my own blog posts, because certain topics are just not made for the official docs, or because some other team member disagree on the content.

I really hope I am misunderstanding this, but this sounds borderline insulting to me. Do you think nobody here ever had thought about a proper vision? I expect you mean it in a productive way, and I am sure we all got a certain idea on how this thing needs to be pushed forward.

Since you seem to be super motivated to work on the docs, let’s maybe talk about some actionables.

How can you contribute?

I am currently busy with some important tasks that can’t wait, such as

  • new rescript-react docs (80% done)
  • upcoming import syntax + making a 8.4 doc feature upgrade (bs prefix drop, mono-repo support,…)
  • syntax widget that lists every single decorator / language construct

There is one particular feature that is pretty high priority, so contributions would be very valuable. Let’s describe it in more detail:

Doc Writing for the Syntax Widget

The syntax widget is btw a really good entry for future writers, for those who are really serious about working together with us to bring the platform forward.

It consists of writing a list of features / syntax constructs you’d like to see on the widget (which allows fuzzy searching), and writing up separate files with examples / descriptions of said constructs. It’s pretty self contained, so I can see a lot of potential having outside contributors working on separate sections.

How does it work? Check out the working branch for this PR, there is a folder misc_data/syntax that has markdown files for each feature that’s going to be listed in the widget (link to the folder).

It would be great if we could get an exhaustive list of all keywords, relevant decorators, and also fuzzy-search keywords to predict the potential user input in the text field. We track the list of data for the fuzzy finder here for now, but I actually would like to extract that data from the frontmatter of the syntax snippet files.

It’s really hard to write everything down and explain everything, so I would also be happy to onboard new contributors via Zoom remote call, or similar, just to be able to talk to each other, so it’s easier to coordinate the work.

Other Tasks

If the syntax widget doesn’t strike your appetit, here is a list of things I actually wanted to work on next (from highest priority to lowest).

  • abstract types
  • poly variants
  • extensible variants (even though i am still not sure how they are useful, except for exn)
  • {.} / {..} types and :> coercion
  • co- / contravariance (<+a> / <-a>)

These topics might be a good start for smaller contributions, and it should be pretty straight-forward to coordinate the work in the rescript-lang .org issue tracker, since it’s already predictable where the content should end up.

If you have any other topics you want to see on the docs, make sure to create an issue first, so we know how to approach the topic, and how to coordinate it.


Let me know if you are interested in any of these upcoming tasks. Even if it doesn’t look like it, we are constantly working on the docs, even if most ppl just don’t realize it. Hopefully it will not be for long, and the docs will be as feature complete as everyone wants them to be.

Happy New Year!

9 Likes

@ryyppy let me just say first off, that was an AWESOME reply. Thank you for taking the time to be so thorough and to communicate how much you care about the docs!

Let me also apologize if I gave the impression that I think that any of the team doesn’t care about the docs. My perspective is one of a newcomer. I’ve only been using the language for about two months and I’ve just noted that the docs lack a lot of what I’ve needed, and that a lot of other newcomers also seem to not find everything they need in the docs. I know that docs require a lot of time, but also realistically, I know that lots of OSS projects simply don’t prioritize them or even take a blaise or even arrogant attitude, pointing one toward blog posts or code examples on GitHub, as if those are the same thing as a good set of docs. Not saying that this is ReScript! It’s just that I’m wary of being let down many, many times regarding documentation and OSS. Just being cautious before getting too invested.

Let me posit an important distinction. You’re right that the forum is probably the best way to get direct access to the team. However, the impression that the project gives goes beyond those venues under the team’s control, whether fair or unfair. I’m telling you that for better or worse, this is the impression that a newcomer may receive as they approach your project.

Goodness, no! No disrespect to the OCaml documentation team, when you start using BNF in syntax examples, you’re just targeting a different audience, lol.

I’m not 100% crazy about TypeScript’s documentation, but I really like its structure: get started, handbook, handbook reference, tutorials, etc. Of those, I think that the handbook and handbook reference are the most vital. I would see the ReScript equivalents as the current “guide” and a new section called “manual.”

100% hearing you loud and clear. I think I’m an unusual developer in that I really, really, really, really, really am passionate about good documentation. I feel like it’s where I could make the biggest impact for ReScript, y’all are doin’ a bang up job on the engineering already :slight_smile:. So my comments are more to show you my passion and willingness to help out as well as an awareness that docs go much deeper than “throw up some code, mission accomplished,” rather than to knock your efforts.

And yet, this is exactly what I propose. Not all at once, but perhaps a multi-year effort for which I could get the ball rolling and of which I would be just one small part. We start with the most high-value parts and grow outward.

Some people would have called BuckleScript a crazy idea, yet here we are.

There’s politics and then there’s politics. If by “that’s just how it is,” you mean, “look, if you wanna cook in the kitchen, then you’re gonna have to get dirty,” then I’m not interested, I will stay cooking at home :slight_smile:. But if you mean, “yeah, doc writing can bring that up in people, so I’m glad that you reached out so that we can clear up any confusion before you start because we have an inclusive culture and I would hate for you to be turned off by wading into a small team’s dynamics totally unaware,” then I’m totally in agreement and willing to put in the work. So far, I have no reason to doubt that it’s the latter and not the former. Everyone’s been kind and helpful.

Maybe the best approach for me then would to be to write my own parallel guide or book hosted elsewhere, kind of like Basarat Ali Syed’s TS book, which has become a normative reference in that community. This makes me sad as I would much rather have it as part of the official docs, but if that effort would be too much of a distraction, then I can do my own thing and merge pieces back in upstream as appropriate.

That said, my gut feeling is that there’s a lot more that could and should go into the official docs. There’s a line, but my personal opinion is that we’re nowhere close.

Certainly didn’t mean to offend or to imply that no one has a vision. First and foremost, I was replying to @TomiS, in full agreement that most programmers don’t have a vision for how documentation should look. Most, but not all. I don’t think that’s a controversial position. I think I’m unusual in that, for a developer, I’m really a bit obsessed with documentation.

More fundamentally, as a newcomer that vision is unclear. All I’ve seen is other people like me frustrated at the docs not being comprehensive enough and being told that the docs are “better than they used to be,” or expressions of surprise that someone would need to look elsewhere other than the official docs, or something similarly apologetic (followed up always with practical help–this community is awesome). I haven’t seen a documentation roadmap, so I don’t think it’s unfair to say “Hey, I’ve got a vision! Can I pitch it and help out?” Lots of OSS projects that don’t care that much about docs and as an outsider it’s hard to tell the difference between “we don’t care” vs. “we do care but are are stretched thin.” Sadly, my experience is that most tend toward the former.

And to push back a bit, you say that others have a certain ideas on how to push things forward, but does the project in fact have an overarching plan for the documentation that everyone is signed up for? That’s what I’m offering to help coordinate, not as a solo cowboy but in cooperation with the community. And yes, with my input of how I’d love to see the docs ideally, but not my input alone. That’s my vision. But if the team has a vision, again, I’m game to pitch in. Natura abhorret vacuum.

You got it! I’ll take a look at the PR you linked to and try to put something together over the next week. I’m in the middle of a house move so my time is limited right now but I’ll probably have more time in a couple of weeks.

I’m down. Hope we didn’t get off on the wrong foot :slight_smile:.

To you, as well :slight_smile:.

1 Like

A similar situation is happening in the OCaml documentation as well. The OCaml Platform team are stretched thin and find it very difficult to update the docs on the official website, ocaml.org. So a few people got together and created the OCamlverse wiki. You can do something similar. Materials from the ReScript wiki, let’s call it, can later go into the official ReScript docs once they get ironed out and fit in the ReScript team’s vision.

EDIT: I just want to add, I appreciate @ryyppy’s answer as well. It’s difficult and time-consuming to do open source. Maintainers have lives too. ReScript is a very cool project and we all want it to have great docs to reflect that, however it’s not an immediate process and it will take time. So I think we shouldn’t be disappointed at having to publish different documentation sites (like Basarat) and instead view it as a positive sign of community engagement. That said, I also want to emphasize: please no BNF :slight_smile: if a syntax reference is needed, SQLite-style railway diagrams are way, way, better.

4 Likes

Something else missing from the ReScript docs that I only found out because I was reading Real World OCaml and decided to try it out in ReScript–you can use underscores in numeric literals, e.g. 3_000_000 :smiley:

3 Likes

I wished OCaml had better official resources than just the manual. But not all hopes are lost, so maybe we will see a new upcoming version of ocaml.org, who knows?

I think OCamlverse’s contributor graph over 3 years shows it really clearly: If you don’t have one single person doing all the work behind it, who is going to do the coordination? We are already doing this unthankful work (making sure the docs are alive), so I’d really appreciate improved contribution processes on the official docs.

I have nothing against external resources like blog posts / books btw, I actually would really appreciate some good ReScript articles. I’d also like to see republished blog posts from previous Reason blogs, but this is very boring and unthankful work.

I was taking some time organizing the rescript-lang.org issue tracker, thinking about missing docs in the manual and tagged them with a help wanted / manual tag. Please report more issues if I missed anything, and feel free to tackle topics if you feel like it.

I also updated our CONTRIBUTING.md and README file, explicitly stating that we are looking for outside contributions for docs as well.

I hope this will clear up that we are not trying to exclude anyone from contributing to the docs. Just to say it again, the docs are really important to us, and we try our best to document all language features (ReScript syntax + compiler). It’s not our intent to intentionally leave out features on purpose.

Cheers.

7 Likes

Update: I went ahead and created an umbrella issue that covers exactly what I tried to express a few posts ago.

It lists all the decorators / language constructs that popped into my mind. Our goal is to have small explanations / quick examples / reference to thorough doc pages to help ppl look up specific syntax tokens and get a quick answer when they are stuck finding the name of certain construct.

6 Likes