oCaml framework or ReScript Nodejs for backend?

[geo_logic]

Just wanting to know the best and not so best outcomes of using both really. I am leaning to more oCaml but if this grows into something more, hiring people may be difficult on the oCaml side.

Plus / minuses for integrations with platforms like AWS, CockroachDB, etc. (mainly the ease of use with databases and handling large amount of multifaceted data structures).

How scalable is it in regards to extending base features into a more complex architecture?

Frameworks that could make this a bit faster?
Thanks!

[chenglou]

Hello! On the server side, we just compile to Node.js, so the risk factors are known. I recommend you to try that route.

In case you’re not aware: we ask folks to write and use runtime-free bindings. So if ReScript ever not works out for you, you can just remove the source code and format the output a bit and move on =)

[johnmegahan]

Yeah, like chenglou said, generally, the best and worst outcomes are just reflecting attributes of JavaScript/NPM/Node, but there are a few ReScript-specific things. Do you have experience with Node services in production? What is it that steers you towards OCaml? I think those would have a big impact on any answers.

The company I work for has a ton of ReScript backend code in production and I’ve spent a lot of time in this area thinking about these same issues. Without knowing more details, it’s hard to say but I’ll try to drop a few notes from using ReScript for some large backend projects over the past few years. We’re fairly happy with things. Our problems generally don’t have anything to do with ReScript.

If JS is important to you, then 100% go with ReScript. If you have JS code you need to interact with, or you feel like Node is a really good solution for your problem, then ReScript is what you want. If JS is not important to you, I’d suggest writing little servers in each to get a taste of what you’ll be working with. Try how things like async control flow, db interaction, testing, ide integration, documentation, learning material, and debugging feel. Different people/companies have different expectations and a week of exploration here could save a world of trouble down the road.

…though we all know the real trouble never shows up until you’ve sunk a year into a project.

If you’re considering OCaml, F# could also be a good fit, feels similar, and is generally easier to hire for and get started with (in my experience).

nPlus / minuses for integrations with platforms like AWS, CockroachDB, etc.
(mainly the ease of use with databases and handling large amount of multifaceted data structures).

You mostly just write FFI bindings to NPM libraries, same as frontend, and so how nicely things integrate is very dependent on the libraries you use and how well they fit your situation. If you aren’t happy with the JS libraries you can find, you’ll probably spend time writing your own framework/infrastructure-layer modules… I know we have.

How scalable is it in regards to extending base features into a more complex architecture?

Do you have examples of what you mean here? Base features of the language? The language itself isn’t really written to be extended.

As far as complex architecture, a lot of things depend on the kinds of async control flow you’ll need. On the backend you don’t have a framework like React to do most of the heavy lifting. You’ll probably use a library like Fastify for your server, you’ll have a DB library, and the rest is up to you. You’ve got promises and callbacks, usual JS/Node stuff… if you need more than that, ReScript doesn’t add anything new in this area. I’ve written our own effect/async tooling and libraries but I don’t really suggest that. It was a lot of work and just happened to be something I’m into. OCaml and F# have more things out-of-the-box around concurrent and async programming… but with that greater control comes greater complexity to manage.

Frameworks that could make this a bit faster?

Again, it’s all Node stuff. I don’t know of any frameworks but it depends on the type of service you’re writing. Fastify is nice, GRPC is nice… GraphQL solves some people’s problems. It’s more that there are a lot of libraries and you build your own framework with them.

I feel like OCaml isn’t too different in this area Ocsigen is the most “frameworky” thing I can think of but I just might be out-of-date. F# feels a bit more “batteries-included” and “just use our frameworks”.

A few other notes:

• Thanks to Node you never have to worry about threading. This is very nice but you have the usual Node performance limits, and the usual Node solutions to those limits. Spin up a bunch of instances and stick them behind something to spread the work out.
• I’d avoid using the bindings that other people have open-sourced. We’ve had too many issues from using 3rd-party bindings (and libraries in general). You’ll want to have control over those, especially when it comes to updates. We also only write bindings for the functions we use instead of trying to write FFI for a whole library at a time.
• Quality of JS libraries varies greatly. Some libraries where a clever, weird interface is just a little interesting in JS can be terrible in ReScript. For example, I recently lost a few days tracking down issues with all of our Knex queries when we updated to the latest ReScript version. But at the same time, the AWS libraries have been great!
• You’re generally able to build the same data structures in both ReScript and OCaml. But if you’re using JS libraries to interact with databases, you’re just going to get JS Objects that are friendly to writing JS back, and if you just use those JS Objects or you try to convert them into a different data structure is up to you.

[geo_logic]

Yeah to be fair, I will need some language to do some heavy lifting. I am fairly sure I will need to split things into separate areas with different languages as I know for sure Node will not be able to handle things as I need them to be handled. It’s great in most use cases for web and what not but having to continuously map out async processes is a bit strenuous on development in general. Yeah I know I’ll have to do that in the frontend but that is to be expected. I know ReScript is fantastic on basically all points, but the reason I was leaning to more of an OCaml type of structure was that it somewhat resembles ReScript and kind of would make it easy to learn more of.

Yes I know it isn’t quite similar but it’s like learning swedish after you already know English(side note bad analogy). I’m looking for something to handle multiple processes at once without needing to streamline async calls. I will have to do some research as it will be a pretty largely complex project with multifaceted structures, processes etc. I am starting to get a realization that probably the best would be to differentiate the two in a since where frontend is ReScript and backend could be F# with some c++ and python depending on what is being used.

I’m new to the whole OCaml area and I normally have used Perl in most cases and node js but I am really liking the syntax, functionality and structure of ReScript as it makes things easy as well as long. I can agree on the 3rd party usage, I have come across a lot of them where sometimes they just bug everything out and do not work. So I try to keep 3rd party packages limited. Even though it is a lot nicer to just say hey let me install, write 2 lines and I’m done. Instead of bind this bind that wait it’s bugging out can’t use it, or can I, no I can’t .

Thanks @johnmegahan and @chenglou for the insight!

[chenglou]

johnmegahan Gave a nice answer which we can agree with, so I’ll highlight a few things:

@johnmegahan

though we all know the real trouble never shows up until you’ve sunk a year into a project

Pretty much.

@johnmegahan

if you need more than that, ReScript doesn’t add anything new in this area. I’ve written our own effect/async tooling and libraries but I don’t really suggest that. It was a lot of work and just happened to be something I’m into.

Yeah we disrecommend rolling a different async mechanism unless:

• You know what you’re doing.
• Your team is onboard.
• You don’t push this hard publicly in the community. It’s a lose-lose for both maintainers and publisher.

@johnmegahan

Thanks to Node you never have to worry about threading. This is very nice but you have the usual Node performance limits, and the usual Node solutions to those limits

Yeah for our current story, it’s better to work with these limits than not. These are usually acceptable. If not, then indeed we’d recommend looking elsewhere (for now), e.g. F#.

@johnmegahan

• I’d avoid using the bindings that other people have open-sourced. We’ve had too many issues from using 3rd-party bindings (and libraries in general). You’ll want to have control over those, especially when it comes to updates. We also only write bindings for the functions we use instead of trying to write FFI for a whole library at a time.

Wisest words I’ve seen this week. I’ve tried to recommend this many times but it seems this is something folks have to learn for themselves, especially when they come from the JS community.

@johnmegahan

• Quality of JS libraries varies greatly. Some libraries where a clever, weird interface is just a little interesting in JS can be terrible in ReScript. For example, I recently lost a few days tracking down issues with all of our Knex queries when we updated to the latest ReScript version. But at the same time, the AWS libraries have been great!

Definitely prefer the boring bindings and ideally the boring JS apis, all things equal!

[yawaramin]

@geo_logic

Yeah to be fair, I will need some language to do some heavy lifting.

You mean CPU-bound work? (As opposed to I/O-bound.)

@geo_logic

I’m looking for something to handle multiple processes at once without needing to streamline async calls.

Can you explain what you mean by ‘streamline async calls’?

[geo_logic]

oh well I mean is, my head, that consistently implementing asynchronous activity to do heavy numerous calls to semi mimic(not even close), multi threading.

Yeah some of it will be CPU bound so I’m trying to make a one stop shop choice on languages. I’m sensing though, however, I might need numerous ones.

[geo_logic]

Oh @johnmegahan your and @chenglou 's answer was the answer I just couldn’t pick both and didn’t want to seem as if I was spamming when I was trying to figure out why I couldn’t pick both

[yawaramin]

OCaml and ReScript both have good support for concurrent programming (async), so you wouldn’t be hand-writing polling calls or anything, it’s all abstracted away behind a promise interface just like in JavaScript. And for multi-threading the recommended solution for both (Node and OCaml backends) right now is actual multi-processing. Let me know if you need any more pointers.

[geo_logic]

Hmm yeah I need to have constant data processing and generating specific files on the fly. I think maybe a rescript node c++ approach with some python may be fine.

[dangdennis]

You can take the aws lambda approach to data processing.

Use lambdas and queues to essentially have multi-node (lambdas) processing your work. This is so common aws even introduced steel functions.

If you want to start quick, I highly recommend using serverless-stack to setup your infra. You can write your code logic in RS

[geo_logic]

I’m not doing anything super simple to need aws take a hold of everything I do. Lambda has been known to cause a lot of issues, especially when bringing on people or learning it in general and handling multi congruent processes that are quite demanding. I do understand where you are coming from but it’s like saying I’d make an OS on amazon when I can just use c++ and actually have a reliable system. I think that, again, you are giving great advice I just think that AWS is kind of like TypeScript, it’s just a hype.

[dangdennis]

Strange that you call Typescript hype. Why’s that?

Your use case does seem very nuanced. I do agree that if I were to start a data pipeline, I’d use the simplest tool possible - single VM or even a powerful laptop in a high performance language. It sounds like other languages/runtimes may fit your use case more: Go and goroutines, Rust and actors. Then again, even single threaded OCaml with promises could be enough.

[geo_logic]

It’s a Hype because everyone jumped on without any regards to what it actually does and doesn’t do just because others saw it as “great”. Every time I have a conversation with people who use TypeScript, they cannot explain the actual benefits of it or the not so good benefits of it. So therefore it’s a hype. Yeah I’m not too into go or rust. I see where you are coming from, and your advice is great, but I am pre-thinking so many good and bad things that can happen using those languages and it’s better if I just do a little mix of things.

P.s. also if anyone uses TypeScript, do not take this personal, I’m pretty sure not everyone who uses TypeScript are like the people I have listed above. Granted it’s oversaturating the market because it gives people the feeling of using a typed language and that’s really about it.

[ryyppy]

@geo_logic

So therefore it’s a hype

That was a long time ago. From a theoretical perspective, TypeScript is way past the hype and a well established mainstream technology. I know we all love ReScript and other alternatives, but we shouldn’t get in some weird denial here

Or to put it differently, based on your previous sentence:

@geo_logic

Every time I have a conversation with people who use TypeScript, they cannot explain the actual benefits of it or the not so good benefits of it

I think that’s almost a literal definition of “mainstream usage”. TypeScript probably did a good job making the technology easy to use, so nobody even knows what it’s like without it.

[geo_logic]

I can understand that. With that being said, all because it is mainstream, doesn’t mean it isn’t a hype. It’s still pushed without most people seeking alternatives. It’s like when American Football, or any sports for that matter, is pushed in your face repeatedly even though you never have nor ever will watch it, it’s still a hype because it’s constantly shoved into your face as being great or something you should do because of this or that.

I’m not denying that they haven’t marketed TypeScript greatly or become a “mainstream language”.
All I am saying is I don’t need a super linter or something to force my code to look great. That’s what standards are for. All I’m doing now is adding another unnecessary library to my code. It’s not that ReScript is great, or basically any other alternative is, it’s the fact that they built TypeScript on lies. If this was the 1800s, Microsoft at this point would be snake oil salesmen. It’s just another big company trying to expand it’s reach because they know what developers like, and that is shiny new things.

But again, it’s my personal opinion, I’m not expecting everyone to like it nor agree with me. If you love TypeScript, great keep loving it, I just don’t really want it anywhere near my codebase.