AI bindings for ReScript + thanks for the meetup

Thanks for the great meetup yesterday - it gave me some ideas. I’ve been thinking about AI generating bindings before. In essence, on the assumption that you are using the (non-existing) binding in a mostly correct manner, we can extract bindings from the way you are using them. I’m sure you get the idea.

I’m working on a tool to auto-generate the bindings as needed using AI. Ruminating over things, I came up with the following scenarios:

1. The user is using bindings (that don’t exist yet) incorrectly.

An example of this would be sort of doing something like:

let foo = Dayjs.turnBackTime()

There are variations to this that are more or less troublesome than the example above. These will be hard to handle until another layer can be added that maps usage to maybe publicly available API specs.

2. The bindings are used properly, but do not exist at all.

This is the simplest case, it seems. We can just try to compile, notice that we’re referencing a module that doesn’t exist, check whether there is a src/bindings/ModuleName.res file. If there isn’t, have an AI read the erroring file and generate bindings for that particular file in the bindings dir. And hope for the best.

3. The bindings are used properly, and there exists some bindings, but some are missing.

Slightly more complex, possibly, but GPT4 (gpt-4-1106-preview) actually seems to do a better job here. We provide the erroring source file to the AI, along with the bindings file. We ask the AI to adjust it by adding the missing bindings after having read the erroring source file.

4. The bindings are used properly, and we generated bindings, but we screwed up

This will likely be the next case that I have to handle. I’m not done yet. It’s really hard to catch some scenarios because, as you know, the AI is not deterministic (maybe I should not be using the chat completions AI - I will have to experiment but appreciate all input).

I think we can detect it by comparing compiler output. E.g., if we generated bindings, did the error message change before and after generating? If so (unless we end up in scenario 3), we probably screwed up with the generation and generated garbage.


Anyway, I just have a POC so far but it’s doing its job alright. I don’t want to get your hopes up that this is even possible (as of today) for real-world use cases, I’m unsure.

The plan is to have it run “wrapped” around the compiler. It would take over the role of bsc watch and monitor for file changes, and then run continuously as needed as a layer between you and the compiler, reading error messages. Seeing the error messages, it will be able to:

  1. Automatically fix code (e.g., write bindings)
  2. Provide additional helpful information on the error message.
  3. Provide you with suggestion for how you can fix the error and, if a known solution is there, fix it for you.

Sort of like a more automatic GitHub co-pilot, but actually useful.

Anyway, let me present to you, revalkyr (working title, lol - GPT4 came up with the name for me and actually wrote most of the first prototype that I binned for me):

  1. We have the source file Main.res like this:
src
└── Main.res

Its contents:

let main = async () => {
  let fmt = (await Ky.get("http://foo.bar/dateformat"))->Ky.text
  let d = Dayjs.make()->Dayjs.format
  Js.Console.log(d)
}

let () = ignore(main())

As you can see, we can’t compile it because we don’t have the bindings for Dayjs and Ky. We then run revalkyr and this happens:

Compilation failed because bindings for the module 'Ky' are missing (detected in src/Main.res on row 4).
Attempting to generate bindings for Ky (with src/Main.res in mind).
We'll put the generated bindings in src/bindings/Ky.res
Bindings were generated for Ky in src/bindings/Ky.res so we will attempt to compile again.
Compilation failed because bindings for the module 'Dayjs' are missing (detected in src/Main.res on row 5).
Attempting to generate bindings for Dayjs (with src/Main.res in mind).
We'll put the generated bindings in src/bindings/Dayjs.res
Bindings were generated for Dayjs in src/bindings/Dayjs.res so we will attempt to compile again.

After that, the src tree looks like this:

src
├── bindings
│   ├── Dayjs.res
│   └── Ky.res
└── Main.res

And we have exactly the bindings we need for the project to run:

Dayjs.res:

// Dayjs.res

type dayjs

@module("dayjs") external make: unit => dayjs = "default"
@module("dayjs") external format: dayjs => string = "format"

Ky.res:

// Ky.res

type response

// Assuming Ky.get returns a Promise of response type
@module("ky") external get: string => Js.Promise.t<response> = "get"

@send external text: response => Js.Promise.t<string> = "text"

There may be inconsitencies in naming, comments, etc. - but who cares if all your bindings can be autogenerated. It builds and runs now. Over time, we could likely converge towards some standard bindings that we could include for known modules (sort of like npm @types). It would probably be good enough in most cases.

Beyond that, I think we are at a point in time where we can make huge gains in terms of community and exposure if we can integrate more AI tooling, something that I think is more difficult and risky for the larger, established players to do. I’ll keep playing with this for a bit, once I get it a few steps ahead – IF I do – I will just open source it on GitHub.

Feel free to chime in with your thoughts. Also, sorry for wall of text.

6 Likes

This sounds quite interesting, please keep us posted on your progress!

Yeah, we’ll see. There are some difficult nuances that arise, or rather quirks with some packages. For example, ky.get is the default export that is aliased (e.g. ky() ir the same as ky.get()) so that should be @scoped to default. It’s hard to know some things in advance, but I imagine one could also have a library of bindings that are ready-made as starting points for the AI.

Anyway, I switched to the assistants API and it’s actually doing a decent job. I also made sure it downloads the NPM package readme and whatnot and sends it to the AI, which, as improbably as it might seem, actually made a decent positive change. This AI likes loads of text thrown at it.

I’m also parsing the AST now to figure out exactly what references we can find to the external, so we can be more precise in our request to the AI. I don’t utilize that well enough yet though.

I’ll be setting up a handful of test cases and automating all of them so I can get some metrics on what’s working and what’s not. It’s difficult due to the stochastic nature of the AI assistants, but obviously we’ll get converging number over time.

So far, after putting maybe 30 hours into it (and double that in ruminations, lol), it’s actually doing a better job than I expected it do. So there’s that. There’s also some things that have to be ironed out with regard to detecting changes in compilation errors (which might indicate that we broke something with the bindings we introduces).

3 Likes

Just because it’s sort of funny, here’s a log from running the latest version of it:

Spun up AutoBindings service.
Spun up BindingsGenerator service.
Spun up Compiler service.
Spun up GitHub service.
Spun up NPMJS service.
Spun up SourceFileMgr service.
Spun up UrlFetcher service.

Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Module Ky is missing. This is something we can fix!
NPMJS :: Looking up GitHub repository URL for ky on npmjs.com...
NPMJS :: Found it! https://github.com/sindresorhus/ky
BindingsGenerator :: Asking AI assistant to generate bindings...
SourceFileMgr :: Wrote ../test1/src/autobinds/Ky.res (890 chars)
Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Oops, we generated bad bindings!
SourceFileMgr :: Deleted src/autobinds/Ky.res
Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Module Ky is missing. This is something we can fix!
BindingsGenerator :: Asking AI assistant to generate bindings...
SourceFileMgr :: Wrote ../test1/src/autobinds/Ky.res (299 chars)
Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Module Dayjs is missing. This is something we can fix!
NPMJS :: Looking up GitHub repository URL for dayjs on npmjs.com...
NPMJS :: Found it! https://github.com/iamkun/dayjs
BindingsGenerator :: Asking AI assistant to generate bindings...
SourceFileMgr :: Wrote ../test1/src/autobinds/Dayjs.res (569 chars)
Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Oops, we generated bad bindings!
SourceFileMgr :: Deleted src/autobinds/Dayjs.res
Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Module Dayjs is missing. This is something we can fix!
BindingsGenerator :: Asking AI assistant to generate bindings...
SourceFileMgr :: Wrote ../test1/src/autobinds/Dayjs.res (228 chars)
Compiler :: Compiling...
Compiler :: Compiler finished.
AutoBindings :: Bindings are ok - or at least we can compile.
1 Like

So I just uploaded the repo if anyone wants to have a look. I only have three trivial test cases right now. There might be a plot twist and I just have to trash the entire thing, who knows.

Positives: It’s already working way better than expected. I’m actually pretty amazed.
Negatives: I have no clue how it’s going to handle real-world cases with more complex apps.

3 Likes

Very nice!
How do the results look like? They are not checked into the repo.

The thing is, the AI isn’t entirely or maybe even somewhat deterministic. It can come up with all sorts of bindings, and again, bindings aren’t 1-to-1 (such as TypeScript @types). They look different each time. And I don’t really mind that, because:

  1. As long as they work as intended, who really cares?
  2. There will likely be convergence over time due to a) better AI and b) a better tool that can take its experience into account, e.g., if we build a repo of oft-occurring bindings that are adequate, we can use them as a basis for generation.

That said, here are a few examples:


Mongoose

Example 1

// revalkyr 2023-11-10 12:28:36
// The Mongoose module binding
type t

// Assuming `connect` return a Promise with an unknown successful return value (`'a`)
// due to `x` being ignored in the provided `.then` callback
@module("mongoose") external connect: string => Js.Promise2.t<t> = "connect"

Example 2

// revalkyr 2023-11-10 12:32:04
// Mongoose.res (ReScript Binding for Mongoose)

// Define a placeholder 't' type for future expansion if needed
type t

@module("mongoose")
external connect: string => Js.Promise2.t<t> = "connect"

Example 3

// revalkyr 2023-11-10 12:32:57
// Mongoose.res (ReScript binding file for the Mongoose module)

@module("mongoose") external connect: string => Js.Promise2.t<Js.Json.t> = "connect"

Example 4

// revalkyr 2023-11-10 12:34:22
// Mongoose.res

// Define the type for the connection result which is returned from connect.
type connectionResult

// Mongoose module with a connect function
@module("mongoose") external connect: string => Js.Promise2.t<connectionResult> = "connect"

Dayjs

Example 1

// revalkyr 2023-11-10 12:28:05
// Dayjs.res - ReScript bindings for Dayjs

type t

@module("dayjs") @new
external make: unit => t = "default"

@send
external add: (t, int, string) => t = "add"

@send
external toDate: t => Js.Date.t = "toDate"

Example 2

// revalkyr 2023-11-10 12:31:43
// Dayjs.res

// Declare the Dayjs module with necessary function bindings
type t

@module("dayjs") @new external make: unit => t = "default"
@send external add: (t, int, string) => t = "add"
@send external toDate: t => Js.Date.t = "toDate"

Example 3

// revalkyr 2023-11-10 12:32:34
// Dayjs.re
type t

// Assume `make` returns a new Dayjs object
@module("dayjs") external make: unit => t = "default"

// Assume `add` takes a number and a string representing the unit (e.g., "months") and returns a Dayjs object
@send
external add: (t, int, string) => t = "add"

// Assume `toDate` converts a Dayjs object into a JavaScript date object
@send
external toDate: t => Js.Date.t = "toDate"

Example 4

// revalkyr 2023-11-10 12:33:37
// Bindings for Dayjs module

type t

// Equivalent of dayjs() in JS to create a new Dayjs object
@module("dayjs") @new
external make: unit => t = "default"

// Binding for the toDate function to get a JavaScript Date object from Dayjs
@send
external toDate: t => Js.Date.t = "toDate"

// Binding for the add function, to add time to a Dayjs object
// The second parameter type is kept abstract (a plain string) for simplicity
@send
external add: (t, int, string) => t = "add"

As you can see, they look different each time. The first line (// revalkyr …) is inserted by the tool just to keep track of which files are auto-generated, but the rest is pretty much straight from the AI assistant with some simple string cleaning.

2 Likes

I’m hitting a sort of early hard stop here, meaning the AI is just unable to consistently generate bindings that are good enough. There’s no point in letting it try a thousand times, might as well hire monkeys to write Shakespeare’s work…

Anyway, there are a few ideas I have that would mitigate this:

  1. More AST parsing. I’m doing it a little more now, or at least using it a little more. I’m actually generating a suggestion for how the bindings could potentially look. For example, if it seems that module Ky is missing in one of my source files, it parses the AST and sees that I am using, e.g., Ky.get and Ky.text. Then it suggestions to the AI that a bindings file might look something like:
// Ky.res
type t

@module("ky")
external get: ... = "get"

@module("ky")
external text: ... = "text"

This actually helps quite a bit. I’m being lazy here, more AST parsing means we can generate better suggestions (e.g., with types that fit my source’s particular use case).

  1. I know we can’t have a @types repo any time soon, or I think it wouldn’t make sense (maybe someone wants to argue against me here - I might be wrong, but I don’t think I am) because of how ReScript and JS don’t map 1:1 but rather n:1, meaning one JS setup can be done a number of ways in ReScript depending on your objective, style, preferences and whatever else context.

That, however, doesn’t mean we can’t have any use for pre-made bindings. We can potentially store “rigid” bindings for NPM packages that fit for a particular use case, and then add an AI layer on top of that to tailor to the user’s use case. I’ve only tested this trivially, so I don’t know how it scales.

  1. Before adding more test cases, I’m going to do some experimenting and start gathering long-term statistics for the cases. Like, the average time it takes to solve a case, and the success rate. We’re going to need consistency, and it needs to be delivered at least within tens of seconds.

Last run (pushed to the repo):

---- trivial-dayjs ok after 14.3s ----
---- trivial-ky ok after 19.8s ----
---- trivial-mongoose ok after 18.6s ----

ok: trivial-dayjs, trivial-ky, trivial-mongoose

So at least it’s managing to solve the current test cases.

1 Like

Came to the forum to share ideas about this, and here you are wayyy ahead.

Haven’t read through all your progress yet to unlock any new ideas. But seeing as how you’re at a new roadblock, maybe a custom GPT for rescript is enough wherein the latest rescript binding docs are up to date, fed to GPT, rescript devs can generate piecemeal best-attempt bindings. To have whole library generation seems like a much harder task.

The problem I have is that i have to wake for openai to update their model to include the latest content. They’ve just now updated to April 2022, so I think it’ll be able to generate better async/await rescript bindings.

TLDR: While it’d be nice for an agent to get feedback loop from the compiler and continually fix the bindings until code compiles, it seems easiest to post javascript or typescript defs and get something close to working,

Yeah, I’m trying, but even for trivial cases, fixing one case sort of breaks another.

I think we’re not really there yet. I’m running a custom assistant fed with all documentation ReScript + a whole bunch of binding files, using gpt4-1106-preview - so I’m actually running the absolute top-of-the-line AI here and it just isn’t quite enough yet.

I’m even feeding it back the error messages, source code and what not on subsequent iterations, asking it to correct its own mistakes - but it usually ends up introducing more mistakes while fixing the prior ones.

This is a major pain point in ReScript for me, all the externals/bindings are definitely what keeps me from using ReScript more. If I had someone write bindings in the background for me, I would be using ReScript for mostly everything.

2 Likes

Ah that’s unfortunate. It’s true in my experience too. I end up reworking most of the bindings. These LLMs just saves me from starting at a completely empty canvas.

And yes, it’s also a barrier for me too. I can do it for my own projects, but having to introduce a team to it is too difficult. Even if I take the approach of least type safety.

I’m curious about the specific pain points you have been running into with the bindings. Is it the burden of maintaining them in a large project over time, or that some functions in JS land are to polymorphic (and so kinda annoying to deal with in rescript)? (or something else?)

In a way, writing bindings is sort of like reifying the documentation/contracts of whatever library you’re using or writing bindings for, which you would need to understand anyway.

(Anyway, I’m not disagreeing with you all about bindings being a pain point…I’m just interested in your experiences with them.)

Well, let’s say you had to write all TypeScript typedefs yourself, and that you couldn’t use the language for your own code base without doing so…

It takes too much time, tne JS side is often ridiculously polymorphic even for well-known, large and established libs.

If I write improper bindings, then my code doesn’t compile. It takes a lot of time for each binding. And if I change the binding later on because I didn’t implement all of it from the start, then I have to go around and change every usage in my code. And without it, I can’t even compile. Of course I could just type everything as 'a, but then there’s no point in using ReScript whatsoever.

If there were @types for ReScript (which I’m not sure will ever exist), I would be using ReScript for absolutely everything. One of the greatest experiences I had when I refactored at 20k lines ReScript project for 2 days untl it compiled again, and it ran without a single bug. Good luck doing that in JS.

Hence to attempt to look toward AI. I’ll try for abit more, but I’m more pessimistic now. It’s like the AI brought it up very quickly to a certain level, but getting above that level seems almost impossible.

I think you hit the nail in this reply

We already have >85k commits worth of types colocated in the @types github repo. If an AI could pull the already written TS types from npm and convert them to “best guess” Rescript bindings, that would 10x my Rescript productivity.

Bindings are not a dealbreaker, but they surely do take time, which means less time I have to write code.

Even if the AI wrote incomplete bindings and filled the bindings with opaque types or some “unfinished” comment flag, that would be incredibly helpful. I could finish a binding pretty easily if the structure was already there.

If we are able to use the existing work in @types, Rescript would get a massive level up.

For the issue of determinism, you might try to call the API directly and set the temperature to 0. That’s not entirely deterministic, but pretty much deterministic in practice.

So for the issue of determinism - it seems the Assistants API won’t let you set neither seed (which is a newly supported feature) nor temperature. I have to revert to using the Chat Completions API to do that, which means I have to upload the entire context for every call. That’ll probably prove to be expensive (for now), but it might still be worth a try.

Maybe I could download @types repo files and parse their trees and try to do conversions, “segment” by “segment” - segment here meaning the smallest unit of self-contained info in the typedef file (TS). After 200-300 lines of code, the AI starts confusing things way too much, it can only reliably reason about like 30-40 lines of code at a time.

Hi,

Are there any ongoing initiatives for generating bindings with AI?

I can’t find what is showcased in https://www.youtube.com/watch?v=3MzqLLLI8ts&t=3599s
Nor does https://github.com/philiparvidsson/revalkyr exist anymore.