I know there is a Pervasives.Stream that implements a pull-based stream type and related functions. As far as I can tell, Pervasives.Stream has been mostly deprecated in the OCaml community due to several issues, and they often suggest other libraries for stream implementations. ReScript & ReasonML have access to some streaming libraries, but it would be nice to have a sort of “default” streaming library baked into Belt, since it’s a pretty common pattern for web development.
As for the design, I don’t think it needs to be even 20% as complicated as rxjs or mobx, maybe something similar to Wonka?
I know it’s probably a bit more complicated than it looks at first glance, but it might be worth it. Just throwing it out there.
Wouldn’t this be better suited as a userland library? The implementation of the Stream library is not very good and is now calcified in Pervasives. Wouldn’t it be safer to have this in your own software or on npm? Writing foundational interfaces for a stdlib is very tricky…
That’s a good point. Might be better off in userland for now. But it’s not like tricky “foundational” libraries haven’t been implemented in Belt before. I’m thinking of Map/Set/List in particular. Those are opinionated in many ways.
I guess it’s a difference if we are talking about foundational data structure implementations (a topic Bob seems to be quite knowledgable about) or observable streams
In JS land, there are quite a number of observable libraries… wonka, rxjs, callbag, mobx, Node streams etc. etc.
Doesn’t it highly depend on the specific use-cases what implementation to use? I am not too familiar with using observable libraries myself, but I followed many discussions in the JS world, and there doesn’t seem to be consensus either.
Doesn’t it highly depend on the specific use-cases what implementation to use?
Sure, but I really just want a replacement for the Pervasives.Stream module, which is just a simple iterable pull-stream model. I remember hearing a talk that identified 4 different kinds of streaming models:
synchronous, pull-based
asynchronous, pull-based
synchronous, push-based
asynchronous, push-based
Within those categories we can find a lot of differences, e.g. API design, compatibility with the Iterator/Iterable and AsyncIterator/AsyncIterable protocols in JS, async schedule configuration, etc.
But what I want is basically just a normal synchronous pull-stream, and maybe an async implementation. Since we know we can rely on the JS runtime (we don’t have to consider cross-platform semantics), the async version should be pretty straightforward. Push-streams are a lot more complicated to implement IMO, but I suppose it’s worth thinking about.
Either way, users are welcome to reach for other streaming libraries. But it would be nice to get something more-or-less baked into the language.
The project I’m working on has dealt with streams and iterables a little bit. We will probably end up open sourcing our standard library built on top of Belt, but I’m not sure how useful it will be to others.
We have what I guess is a push stream; it exports an opaque type that can be subscribed to, with listener functions i.e. type listener('a) = 'a => unit;. The stream can be mapped, filtered, temporarily blocked, a bunch of handy operations. It’s only 60 lines of code and at the core is a very simple emit function:
let emit = (stream, value) =>
if (stream.locks < 1) {
stream.listeners->eachU((. listener) => listener(value));
};
Beyond that, we are working with a library that returns iterable / generator values. We don’t have a way to create these, yet, but consuming them is easy enough without needing to resort to raw ES6 injection.
We started with a lazy sequence LazySeq with an opaque type implemented as:
type t('a) = (. unit) => option('a);
And then map the iterator protocol to that type in JsGenerator (here t('a) is again opaque):
type jsGeneratorResult('a) = {
value: Js.Nullable.t('a),
[@as "done"]
done_: bool,
};
type t('a) = {
next: (. unit) => jsGeneratorResult('a)
};
let toLazySeq = gen =>
Tiny__Core__LazySeq.make((.) => {
let result = gen.next(.);
if (result.done_) {
None;
} else {
Js.Nullable.toOption(result.value);
};
});
Our binding to functions that return an iterator use JsGenerator.t as the return type.
At least it’s advertised as a JS / TS library and it’s loosely based on the callbag spec… I wonder how many observable implementations are out there? Probably will write my own one eventually and call it “rescript-better-streams”
