How to decode a javascript object

[jmagaram]

I’m using Firebase and when you load data you get a plain old, but opaque, javascript object, like { flavor:"vanilla", isFavorite:true }. I think this is different than JSON, right? So I’m trying to decode this opaque type into ReScript into a type like type iceCream = { flavor:string, isFavorite:bool }. I assume I need to look for the “flavor” key and if it exists, check that it has a string value. Then do the same for “isFavorite”. And if I’ve extracted both, stuff them into my strongly-typed ReScript type, or generate a None. I can’t figure out how to get started with this. The Js.Json functions deal with JSON and I don’t think this is one of those. The features in ReScript that deal with Object types work with strongly-typed objects - known ones - not opaque ones where the structure is unknown. I’m sure this is an extremely common task - working with data imported from somewhere else - and just don’t know how to get started. Is there a built-in standard library for this kind of thing?

type iceCream = {flavor: string, isFavorite: bool}
let parseIceCream = x => {
  // what goes here? what is the type of x?
}

Js.Dict seems to want all the values to be the same type.
Js.Obj gives me the names of the keys but I can’t extract values.

I could write a lot of this on the javascript/typescript side and import it. There are schema tools like jquense/yup: Dead simple Object schema validation (github.com)

[chris-armstrong]

You have a couple of options to do this in a reasonable manner:

1. Use the Dangerous type cast to convert from type t to iceCream (where t is the original type from your bindings to Firebase of the returned object) by writing:

external parseIceCream: t => iceCream = “%identity”;

(note this is not typesafe at all)

2. If you’re parsing from a JSON string, use a library like jzon or funicular, which are useful abstractions over the builtin Js.Json that will convert in a type-validating manner.

[spyder]

While the Js.Json functions do deal with JSON, for most purposes it’s quite sufficient for analysing opaque objects if you declare the bindings as Js.Json.t instead of completely opaque.

From there many solutions exist to analyse it. The two @chris-armstrong suggested are probably your best bet.

I’m quite fond of decco, but it hasn’t been updated for ReScript (The API will work on ReScript code, if you can translate the examples, but the project is ReasonML and eventually ReScript will drop support for ReasonML). It doesn’t scale well in my experience but for small projects it’s brilliant.

[ryyppy]

@chris-armstrong

external parseIceCream: t => iceCream = “%identity”;

(note this is not typesafe at all)

The name is rather misleading since you are literally not parsing here. I’d call it external unsafeToIceCream instead. IMO this option is the most frictionless and pragmatic one… if you have a complex api with proper api-spec, might be worth generating types / identity functions to ease the process.

@chris-armstrong

If you’re parsing from a JSON string, use a library like jzon or funicular , which are useful abstractions over the builtin Js.Json that will convert in a type-validating manner.

Be cautious with third party decoder libs. Any decoder that uses result types as a return value is already opting for a slower, and more memory demanding API. A thing you definitely don’t want in a core essential such as JSON decoding.

Relevant discussion:

[RFC] New Promise Binding

On the subject of adding result to the bindings I agree with @ryyppy decision. I used reason-promise quite a bit and in our case it actually did more damage when the codebase grew. Most of the time we’d have to get back to simple promises and create our own “monad” stacks on top of it. Some thoughts on “over”-using result I found in the wild (just opinions of course, also f# , useful nonetheless) https://eiriktsarpalis.wordpress.com/2017/02/19/youre-better-off-using-exceptions/ https://docs.m…

There used to be a library called bs-json which raised DecodeErrors instead to mitigate any intermediate object allocations (that can be disastrous in any more serious app JSON payload) and allow better stack traces.

[chris-armstrong]

@ryyppy

@chris-armstrong

external parseIceCream: t => iceCream = “%identity”;

(note this is not typesafe at all)

The name is rather misleading since you are literally not parsing here.

My apologies, I didn’t mean to make out this wasn’t parsing, it’s just a unsafe type-cast as the documentation implies: I was just using the name the author established

@chris-armstrong

There used to be a library called bs-json which raised DecodeErrors instead to mitigate any intermediate object allocations (that can be disastrous in any more serious app JSON payload) and allow better stack traces.

This is also a good option I should have mentioned - whether you want a exception-based library or a result-based library probably depends on your use case (I’ve been working with API inputs, so a result-based library makes more sense to me, as it explicitly surfaces parsing errors at the error boundary in a way that is easy to resurface them, but if you were reading from a DB like Firebase where your app authored the data, exceptions are going to be more suitable). In any case, the Js.Json library is a nightmare to use directly for anything non-trivial, so I would only use it for very simple use cases.

[jmagaram]

This is very helpful. Thanks. I’m using the Firebase SDK which converts the wire data to plain old JavaScript objects. It’s not JSON strings I’m sending and receiving from the SDK. So I’m wondering if I should be using something like https://github.com/sideway/joi instead to validate that the data I’m receiving has the right shape and constraints. If it does then I could do the “unsafe” cast to work with it in ReScript. I assume that to use the libraries you’ve mentioned here I’d have to reconvert the objects to a Json string.

[jmagaram]

I’m confused by this. I’m getting a deserialized object from Firebase SDK. I guess this is typed as a totally open `a or {…} in ReScript. I can’t be guaranteed of the shape of the data. How could I use Js.Json library to see if it has a “flavor” property that is a string, for example, without first serializing the whole thing back into a Json.t?

[chris-armstrong]

To summarise : you’re getting a plain JavaScript object back from Firebase, and you want to validate its the correct shape before you cast it to a corresponding record type inRescript.

In that case, you could use a JavaScript based validation library like joi or ajv (which is precisely the method I use and recommend in Typescript), or you could work out how to kick off the validation component of the aforementioned Rescript libraries without the JSON string deserialisation step.

I know this is possible with funicular (I’m the original author), but there isn’t an API entry point to make this easy. If you check out the API docs for jzon, this should be theoretically possible with the decode methods. For bs-json, it seems you just need to call your decode function with the Js.Json.t, skipping the parse step. In any case, you’ll need to typecast your object to Js.Json.t first.

[jmagaram]

Yes, exactly. Making progress finally…

external convertToJsonUnsafe: 'a => Js.Json.t = "%identity"

let convertToJson = i =>
  switch i->Js.Json.test(Js.Json.Object) {
  | true => Some(i->convertToJsonUnsafe)
  | false => None
  }

describe("validation experiment", () => {
  test("json", () => {
    let doc = {"flavor": "vanilla", "isFavorite": true}
    let flavor =
      doc
      ->convertToJson
      ->Option.flatMap(Js.Json.decodeObject)
      ->Option.flatMap(Js_dict.get(_, "flavor"))
      ->Option.flatMap(Js.Json.decodeString)
      ->Option.getWithDefault("does not have a flavor!")
    flavor->expect->toEqual("vanilla")
  })
})

[spyder]

Yes, exactly. With bindings to a library the value can be Js.Json.t directly, so you don’t need the conversion. And all of the ReScript libraries mentioned in this thread (also bs-json) will help avoid writing manual Js.Json and Option.flatMap to decode the value.

[spyder]

@ryyppy

There used to be a library called bs-json which raised DecodeErrors instead to mitigate any intermediate object allocations (that can be disastrous in any more serious app JSON payload) and allow better stack traces.

bs-json still exists, but I’m surprised that you see the result-based API chosen by other libraries as slower. I would’ve thought all the exception throwing/catching (particularly when extra code is generated to manage inspecting the exception object) would make such code harder for JS VMs to optimise.

[yawaramin]

@chris-armstrong

In any case, the Js.Json library is a nightmare to use directly for anything non-trivial, so I would only use it for very simple use cases.

I just want to clarify, in case anyone got the wrong impression, that Js.Json and bs-json are two different things.

• Js.Json: Js.Json | ReScript API - shipped with ReScript, low-level JSON manipulation
• bs-json: https://github.com/glennsl/bs-json - third-party library, high-level JSON encoder/decoder combinators

bs-json works great, starts pretty simple, and can handle quite complex scenarios if the need arises.

Another point I would like to make, with claims of performance, it’s better to always benchmark for your own use cases rather than take advice from message boards (I say this, despite saying it on a message board).

[notchris]

Does anyone know the status of all these projects:

bs-decode, bs-json, funicular, jzon?

[yawaramin]

I don’t know the others but i know bs-json is production-ready and well-maintained.

[nkrkv]

I’m using bs-json and jzon in production (small scale). But I’m biased, because jzon is a lib made by me.

[carere]

Didn’t know about Jzon , thx @nkrkv for the info, i’ll try this one

[DZakh]

@ryyppy

Be cautious with third party decoder libs. Any decoder that uses result types as a return value is already opting for a slower, and more memory demanding API. A thing you definitely don’t want in a core essential such as JSON decoding.

I’ve added an exception-based API for the rescript-struct additionally to the result-based one. I remembered the post, so I decided to check out how it will affect performance in the benchmark I’ve mentioned before in the post.

And the result is that exception-based API is just 2-3% faster. So it’s more a matter of convenience than performance.

But if we are talking about internal code and not public API then the difference might be noticeable. In my case I managed to increase performance by 20-25% replacing all results to exceptions inside of my library.