Given that GPT/ChatGPT is all the rage nowadays, some of us have been throwing ideas around for how we could leverage that type of thing to improve things in and around the ReScript ecosystem.
Are you interested in/knowledgable around these topics, and interested in exploring applications for it in the ReScript ecosystem? Please reach out! We’d love to explore this area, but have limited bandwidth ourselves right now.
Some ideas we’d like to explore:
Bindings. Can we train a language model to automate most/all of writing standard bindings by teaching it how to write idiomatic ReScript bindings by reading for example TypeScript definitions? Writing good bindings is of course contextual and more an art than science, but large chunks of it should be possible to automate this way.
Inspired by Supabase Clippy: ChatGPT for Supabase Docs, can we build our own “chat bot docs”? Where you can just ask the bot how to do things in ReScript, and get sensible replies back.
Do you have other ideas in this space that could be useful in a ReScript context? We’d love to hear them. This is an interesting area we hope we can explore and find good applications for.
I am interested. Using Copilot now has helped with simple tasks.
I vote for bindings generation, that is the most useful case I can think of.
It would be great to add it to the Rescript VSCode Extension to fix bugs of highlighted code, or automate tasks like extracting a piece of code into its own function/component. Or merging isolated components into a bigger component.
I’ve tried ChatGPT to create bindings but it seems to have no clue what is Rescript.
The fact that Rescript language is changing so much it makes it harder for AI to keep track of it. Eg. Bucklescript, Rescript with Belt and now Rescript core. But maybe with fine tuning we could pull it off?
A Clippy for Rescript would be great and could co-live with the playground.
Also interested! I use Copilot every day for writing rescript, its results need tweaks sometimes of course, but I generally find it to be extremely successful and frankly life-changing. It tends to work better on large files where it has lots of sample code to look at. Repetitive tasks such as refactors, or rewriting code from JS into RS, work especially well (a little tip – paste your JS in as a big code comment, then add a comment below saying something like “// the above code rewritten in rescript”, then trigger copilot below that).
There is an open source solution for “chat bot docs”: DocsGPT. Maybe we could try it. (Edit: I have tried to run it locally and unfortunately I can’t build the python application on my machine )
Another idea I had is to train our own Copilot-like model (i.e. OpenAI Codex) on ReScript specifically. Copilot has the problem that it is not trained on very much rescript, and rescript can often look a bit like JS, so Copilot can occasionally suggest JS as a result.
Update: managed to get it running and I ingested the entirety of the rescript docs .mdx files. There’s some problem with the OpenAI embeddings in relation to the format of these docs, but someone on the DocsGPT team is working on it with me! Should have more info in the next couple of days.
As for CoPilot and bindings generation, I think regardless of what we do we’ll need to train a model ourselves, to some extent at least. That might also be an opportunity, if we can curate what it’s trained on and ensure it’s good, up to date ReScript code.
Bindings are a standout problem, and exploring GPT family of tools could potentially fix this. +1 to this,
However there’s one more place we can seriously improve on: ReScript compiler’s error messages.
Its error messages are far better than mainstream programming languages (for me that’s Java), but the ideal version of what I’d like the compiler to tell me is how to correct it, rather than just stating what’s wrong.
This might draw comparisons to Copilot, but it is more specific. While Copilot generates the solution code for a programming problem stated in natural language, this would boost the compiler’s error reporting skills by having it generate the correct code for the specific context the error was reported and then suggesting it to the user.
Hi
Im one of the maintainers of docsGPT @benadamstyles asked for some help on our discord so here are the results,
some answer examples (hope they are right)
Thanks so much @dartpain. I’ll try loading this up now. @zth if I get this working, what do you think the next steps should be? Should we host this somewhere? We’ll need to think about how we manage the costs associated with hitting the OpenAI API.
I have deployed the DocsGPT python application, trained specifically on the latest ReScript documentation, to the internet (using Railway), and built a custom front end for it. You can try that out here:
Please don’t share this around on social media yet, my Railway server limits are quite small! Totally fine for anyone who sees this to take a look and use it, but any serious amount of traffic would hit the limit and turn it off.
Well I had to tweak the text prompt (internal context sent to GPT to tell it how to answer) a few times and it seems to be pretty reliable, and impressively creative as I’ve come to expect from these LLMs.
There are some quirks which we might be able to resolve with further prompt improvements, or if not we would have to make really clear to users that they can happen. One example is that it made up Belt.Array.groupBy() – but if it did exist (and I’ve wished more than once that it did), this is exactly how it would work
But like I say, I think we can probably prevent this by tweaking the prompt. So overall I think that this is already ready for a “Beta” release if that is something that we want to do.
I’ve also been thinking about collating a collection of “good” JS/RS bindings and creating embeddings (basically, “training”) for that too, as well as the RS docs. Then you could ask this thing to create bindings for you as well as documentation. So that’s my next step.
Oh, now there’s an idea… we could totally do that, right? Extract all rescript-tagged code blocks from the markdown, and run them in a browser build of the compiler (is there such a thing?). Report any errors to the user and say something like “this code does not compile, maybe try asking in a different way?”
Run the python app on the rescript docs server (Vercel, right?) and add our own chat UI to the rescript docs.
Then even better would be integrating it into the VS Code extension directly, so you could hit the python app from VS Code.
Regarding 1: The website runs on vercel, yes. ChatUI could live in a separate page on the docs if needed? Ideally we’d have a single OpenAI api key, but I am not sure how much traffic it will generate.
I suppose it depends on what budget there is for this. We could throttle requests to OpenAI from our side and ask the user to provide their own key if it’s a problem.
this is pretty awesome, but I’ve noticed it sometimes get confused and mixes Reason and Rescript syntax because I guess it got trained on both versions of the docs.
)
we could totally do that, right? Extract all rescript-tagged code blocks from the markdown, and run them in a browser build of the compiler (is there such a thing?). Report any errors to the user and say something like “this code does not compile, maybe try asking in a different way?”