Updating Object, the Object/Record experience is poor

Tl;DR

I am suggesting that Object should have the ability to be updated safely, and that open structural comparison should be easier for types.

I am also suggesting that Records and Objects should have a cleaner update mechanism.

More broadly, I am suggesting that Object/Records should have a more FP native way of updating to allow for more use cases and easier adoption of ReScript.

Use Case

I read through a lot of these forums, and typically it just comes down to someone saying, “use case” before any meaningful conversation can be had. So, let’s get that out of the way.

I am reviewing ReScript for production usage, and our current use case is to pass data to functions that only care about specific properties of that data and ignore all other properties (keys). This is a common practice in JavaScript, but more common in Clojure. The fancy term is Data-Oriented Programming. However, I think this is just part of Functional Programming.

The domain we are modeling is a user support system, which the ticket models alone can have multiple levels (I think 3 or 4 max) of nested data. (Not my choice FYI).

The idea behind the use case is that you may have a data that is structurally defined, but a specific function that does not care about the complete structural definition or if that definition changes. The function only cares about a specific portion of the data.

Concrete Example

To be more concrete. Imagine we have this person type.

type person = {
  "name":string,
  "email":string,
  "age": int,
  "address": {
     "street": string,
     "city":string
   }
}

Now imagine the data can be bigger than above, it could have former address, credit cards… all the great things are domain model requires to be stored and passed as data.

Now, a common thing for people to want to do is update thier email address.

Okay, in ReScript Objects are forever immutable, so we would probably want to model our data as a Record. So, we could build a function like:

// Made shorter for better reading, pretend there is a bunch of other properties
type person = {
  email: string,
  age: int,
}

let updateEmail = (p:person, newEmail:string) => {...p, email: newEmail}

As a side note, the ability to update Records is rather simplistic and leaves a lot wanting. If all your Records are 1 level deep, then this works fine, but as soon as you get to Records in Records, the above syntax is not a smooth update.

Updating Records done, but it doesn’t really meet my requirements. This updateEmail() function is highly coupled to the data type person. However, the business requirements may (and did) change. So instead of just person, we now have a business type and a reseller type. These types are similar, but also different.

We could solve this problem making our Record more generic, make it cover all use cases, and some type of flag to identify between different entities.

This would create a new entity type Record.

type entity = 
| Person 
| Business 
| Reseller

type entity = {
  email:string,
  age: int,
  address:address,
  entity: entity
}

The consequence is that a person will have unused business properties, and vice versa.

Maybe the consequences are not acceptable. Maybe it does not make sense for the business logic. Maybe the entity logic gets more complex (e,g. there needs to be many categories of business). There could be a number of reasons why one would not want to do this, even the simple reason that they want their data to be modelled separately.

What is it does do is tightly couple your updateEmail() function with the data type. If we ever wanted to update the email of anything, we cannot. We have to use an entity type (previously a person type).

To make updateEmail() more reusable, and not connected to entity directly, it would require updateEmail() to only take data that has an email property but doesn’t care about any other property. Which is not possible with Records, and as the documentation suggest, Object should be used.

So, we could write this with an object, but… oh no :scream:, we cannot update objects in a safe immutable way. ReScript has limited that.

In addition, creating a function to structurally compare is akward. You have to write a function like this:

let updateEmail = (obj:{.. "email":string}, updateEmail:string) {...}

The syntax {.. "email":string} is not documented, difficult for bigger objects, and can cause some odd error messages see Issue #5594.

I am happy this syntax exists otherwise; we would be stuck. What I am suggest is that the ability to duck type with object types like in TypeScript would provide a better experience for those coming from JS/TS world.

(I currently have no suggestions beyond looking towards TypeScript)

My Current Solution

I require this structural typing; my current solution utilizes Deep Patch | Rimbu. It is rather ugly, and hard to sell ReScript to my company when something as simple as updating an Object safely is such a chore.

Here is my solution:

type person = {
  "email": string,
  "name":string,
  "age": int
}

let me:person = {
  "email": "sean@example.com",
  "name": "sean",
  "age": 32
}

@module("@rimbu/deep") external patch: (~state:'a, ~patch:'b) => 'a = "patch"

let updateEmail = (state:{.. "email":string}, update:{.. "email":string}) => {
  patch(~state=state, ~patch=update)
}

Js.log(updateEmail(me, {"email":"updated@example.com"}))

The reason the property state and update have the same type {.. "email":string} is to make sure that whatever Object you pass to update will conform to the Object passed to state.

It’s not perfect, and if someone has a better way, please let me know.

It would be nice if I could define a type that is open, and I could use that type across both state and update without having to rewrite the type signature twice.

Bonus points, if I could make updateEmail() even more generic, something like updateProperty() where the consumer could pass their own open type to the generic function updateProperty().

Like this in TypeScript:

updatePropety<{email:"string"}>(obj1, obj2)

This could probably be done with PPX (however the documentation around that is very outdated and lacking depth), but I could not after hours of testing, find away in ReScript alone.

What I Suggest

What I suggest is that ReScript provide a native way to safely update both Record and Object. I think both Immer and Deep Patch | Rimbu provide safe, efficient, and native feeling object copy/update.

Maybe ReScript could provide a special syntax operator, such as patch, produce, or update. I imagine something like this:

let updatedState = update(state, (draft)=>{
  draft["email"] = "new email"
})

Where in this case, draft is a special mutable version of the Record or Object you passed into state. This way we can update properties in a type safe fashion.

Of course, this should be done efficiently, meaning that only the updated properties are on the new returned object, every property that is a reference is still a reference.

This would probably break with Reason history, and even break with other JS compiled FP languages like Elm or PureScript. However, I think it would give ReScript an edge as updating objects in this matter feels imperative but is actually safe.

Summary

Whatever happens, updating objects today is a frustrating (boilerplate) experience, because the language was design not for the use case presented here. Updating Records is not much better.

Maybe the answer is “this is not a use case ReScript wants to cover”. That is okay. However, it kind of sucks because on my check list ReScript falls into that good place between TypeScript and PureScript. The only area it is missing is a way to safely update Objects.

(Also, apologies the word “poor” in my title was click-baity)

2 Likes

How would you do it in TypeScript? Wouldn’t you also use rimbu/patch or immer?

In the TS/JS world. Indeed, Immer or using Immutable.js are popular.

Immer is more like what I suggested should be native to ReScript.

Immutable it is more like Clojure with get and set functions. That could also be a possibility.

The FP purist would suggest lens, like in monocle-ts

However, all the above provides TypeScript type safety (even if the type safety in TS is not sound) because of TypeScripts advance type system. See rimbu/patch.ts for an example. I tried but could not find or produce anything that could replicate Immer or Rimbu natively with type safety intact.


I personally, would use Immer.

Rimbu is rather new to me, and I am testing it out right now for production level stuff. It brings the best of both Immer and Immutable.

If I continue with ReScript, I will call out to Rimbu/patch to update Objects or more likely I will take the code from it and just use that as a base.

1 Like

It would be nice to have the functionality you suggest. If I understand correctly, it wouldn’t even need a language-level change, a decorator could be added that could generate the needed update code. E.g.,

@deriving(accessors) // This would generate accessor functions
type hasEmail = {"email": string}

// E.g.,
let person = {"name": "Bob", "email": "bob@email.com"}
let updated = person->set_email("bob@newemail.com")

// This could produce JS code like:
// var updated = Object.assign({}, person, {email: "bob@newemail.com"})

An important piece of the puzzle would be using Object.assign (or spread syntax) in the generated code, which would ensure all existing properties would be preserved.

Types likes hasEmail can be combined together using ReScript existing object type merge syntax, so you can build quite complex models out of simpler ones.

This would be my recommendation.

2 Likes

This is doable with a decorator, under some assumptions, but only because just about everything can be done with a decorator. (The assumptions being you know everything about the type, which can often be false when objects and type inference are involved).

There’s the separate question of what is the price of introducing more magic in terms of maintainability etc. This is a question more for the user of such a mechanism than the mechanism itself. E.g. change type and follow type errors: how much is that process affected by such a change, I don’t know.

The more compelling message, to me, from this and other use cases, is that the current objects fall short in a number of ways. Open objects in particular are extremely awkward to deal with. Update methods are another example.
The root for all this is trying to bolt some form of object semantics onto a type system that was not designed for that exact purpose.
It would be good for now to collect a number of use cases and wishes for the future, in case we get to the point of implementing objects in a first-class way. A.k.a. re-implement objects from scratch.

3 Likes

In term of proposal, I would salvage very little of this TLDR. I would focus on the problem and not jump too quickly to conclusions.

@yawaramin
Yeah, this would kind of work.

See, I actually thought of the decorator process as well, which is the reason I looked into PPX. However, I realized it would take me a bit too long to figure out how to write a PPX than the time it was worth during my evaluation of ReScript. Also, (if you cannot tell from my first post) I ultimately think this functionality of updating Objects should be part of the vanilla experience.

So, long story short, the decorator syntax might be better than a keyword as it feels the “ReScript way” to use a decorator.

Counter point, even with the decorator syntax, the situation is still awkward. Although less awkward than what I am doing now.

Note about Object.assign

Object.assign() is a great idea. I kind of forgot about this function, but actually Immer used this function until 2018 to make the copies you mutate. That got rid of it to support IE 11.

I tried to update my code to use JS.Obj.assign().

let updateEmail = (state:{.. "email":string}, update:{.. "email":string}) => {
  Js.Obj.assign(state, update)
}

But the return of Js.Obj.assign is {..}, meaning I lose all context of what the type of return object.

Note from future Js.Obj.assign will not work in ReScript, because it only allows two inputs. You would need to do Js.Obj.assign(Js.Obj.assign({}, state), update)), However, ReScript does not allow {}, so you still need to do what I did below.

For my code, I can fix this with some ugliness, but it works😊.

let updateEmail = (state:{.. "email":string}, update:{.. "email":string}) => {
  
  ((_:'a, _:'b):'a => {
    %raw(`
      Object.assign({}, state, update)
  `)
  })(state, update)

}

However, as far as I understand, decorators can only write more ReScript, not JS. I could be wrong about that. If I am not, that means that Js.Obj.assign() cannot be used directly, and we need to keep type context with interop tricks.

Solution With Decorator

Assuming there is no language-level change, which again it would be cool if it was just part of the language, I would imagine you would setup the decorator instead so it produces ReScript code like this:

@deriving(accessors)
type name = {"name": string}

@deriving(accessors)
type contact = {"contact": { "email":string, "phone":int }}

// Auto-generated from decorator - would not see in your code
let set_Name = (state: {.. "name": string}, update: {.. "name": string}) => {
	((_:'a, _:'b):'a => {
    %raw(`
      Object.assign({}, state, update)
  `)
  })(state, update)
}

// Auto-generated from decorator - would not see in your code
let set_Contact = (state: {.. "contact": {.. "email":string, "phone":int }}, update: {.. "contact": {.. "email":string, "phone":int }}) => {
	((_:'a, _:'b):'a => {
    %raw(`
      Object.assign({}, state, update)
  `)
  })(state, update)
}

This way you can use the call like so


@deriving(accessors)
type contact = {"contact": { "email":string, "phone":int }}

person -> set_Contact(
{
   "contact": {"email":"updated@example.com", "phone":person["contact"]["phone"]]}
})

Where person is of type

type person = {
	"name":string,
  	"contact": {
  		"email":string,
      	"phone":int,
      	"address":string
  }
}

Summary

So, decorate is a possible solution, but boy is it awkward especially when start talking about complex Objects or having to create one-off types of every update. Compare the above solution to just this:

let updatedState = update(person, (draft)=>{
  draft["contacts"]["email"] = "updated@example.com"
})

To @cristianoc point, Objects are just awkward. For example, you cannot compose their types. It is very clear that someone prefers Records. The missing spread syntax to compose Object types means you end up writing more verbose Object types than you may necessarily want. They don’t compose like Record types or object types in TS.

@seancrowe I also wanted to ask: in the formulation of the problem there seems to be an implicit assumption that functional update is required, vs also considering imperative update. Is that a personal preference, or a consequence of the way the data is going to be used?

@cristianoc
Good question. Yes, to both.

As I have gotten older, and gone through the pain of many failures, I have come to the conclusion that in almost all cases a functional approach is the correct approach. That in FP, both functions and data are first class citizens - data should be immutable, functions should be as pure as possible

Since I see problems from that perspective, it is both a person preference but also a requirement for my business logic.


As side note, ReScript is being evaluated, because it is very simple FP in comparison to say PureScript. If ReScript was not (mostly) strictly FP, I wouldn’t even bother considering it over TypeScript. The reason is three parts -

  • If you do not enforce FP, people will fall back to their imperative/OOP habits. My entire team would be sneaking in imperative style, and FP without properly enforced boundaries is not reliable.
  • TypeScript has a massive ecosystem that works natively with an imperative (and OO) style.
  • Trying to do FP in an in-between language (like JavaScript) can be rough. It typically means that you need a bunch of libraries to get going - libraries that all have different idioms and typically do not work with each other.

So, if your goal is that Objects allow mutation without a copy/update, then from my perspective that should not be allowed. The more imperative ReScript becomes, the less unique qualities it would have over TypeScript.

As mentioned above, there is a simple mechanism for updating that can be used to make updates feel imperative and native while keeping the same safety benefit of no mutations.

Thank you for providing context on the goals of the team.
One thing I was wondering, is whether immutability buys you everything you need in the context of the examples given in this thread. This is where I would expect you’d want to hide the representation details a little bit more, to enforce whatever invariants make sense. Rather than just having a deep copy abstraction together with exposing the internals of the data.

2 Likes

Note that in your Object.assign examples you are actually mutating the state object, a mistake people often unintentionally make. The function updates the left-most object in-place with the properties of all the following objects, so to create a new object you have to be explicit: Object.assign({}, state, update).

@yawaramin Good catch! As you wrote, easy mistake to make. This is why reading the documentation is important :joy:. I updated my examples, and also made a note that Js.Obj.assign() would actually not work because ReScript does not allow empty Objects.

@cristianoc Yes it does. The examples seem simple, but in a real app, knowing that the data you are working under will never change allows immense confidence and allows you to write pure functions. Without immutability of data, then you can never write a pure function (a bit of a lie, you can deep copy all data coming into a function). In ReScript data is immutable. You must declare otherwise, or breakout of the system to have another outcome. This means you know about it upfront and can handle it in a safe way.

If you don’t subscribe to that way of thinking, then it all seems like fluff. The best way to describe it is to imagine if the underline logic in functions could be changed on the fly (without changing their reference). If that was true, then your confidence that calling the substr() function is low because you cannot guarantee the underline logic wasn’t changed by some side effect in the system. You would need to program defensively or introduce a whole bunch of bugs because underline function logic is mutable.

I know that function logic being mutable sounds crazy, but from my perspective, data being mutable is equally as crazy.


Important to note, Object.assign is not a deep copy, it just copies reference except for those things which are only values (like numbers). So, while still more expensive than mutation, it is not as expensive as you think it would be.

Mutation is still faster, by like a lot. In practice though, mutation vs Object.assign is not going to be your bottleneck or even noticed as a performance cost. However, it is important to keep in mind when designing an application.

1 Like

Actually, ReScript does allow empty objects, and Js.Obj.assign will work fine. Check the API docs: Js.Obj | ReScript API

E.g.,

let update = (obj, props) => {
  open Js.Obj
  ()->empty->assign(obj)->assign(props)
}

@yawmaramin Thanks :grinning:, if you cannot tell, I am new to ReScript.

So, I think I found a way to get what I want with 100% no change to language features. The bellowing naming convention is still TBD

@seancrowe “yes it does”. Not sure my question was clear.
The properties of immutability are well known.
My question is: are you sure there aren’t more properties you’re interested in enforcing. It seems an incredible coincidence that exactly and only immutability is all you need.
Maybe that’s what you meant, but just wanted to make sure.

1 Like

As mentioned above, I am able to replicate the behavior I want without any new language constructs. This works by the decorator auto generating a mutable object type and the function that expects that mutable type. However, this is all just a trick😉.

@yawaramin after testing, Object.assign() does not work as it copies everything by reference, so that means sub-objects are by reference. That means when you edit the sub-object, it is editing the reference. Therefore, we need to up roll our own object update script.

You can find the full solution on this playground and written below.

The object update JavaScript is just a quick naive attempt. Also haven’t tested every use case nor am I certain this is the smart way to do it as this copies objects that are not even modified. Is this any better than just structuredClone()? I do not know.

Again, this is just prototype. I still need to think how this would be implemented for performance.

I would also need to figure out how to utilize PPX if I want this functionality today. That is a whole other beast.

Here is the code if the playground link goes bad.

@decorator
type person = {
  "name": string,
  "age": int,
  "contact": {
    "address": {"street": string, "city": string},
    "phone": string,
    "email": string,
  },
}

// This get automatically generated
type __update_person = {
  @set
  "name": string,
  @set
  "age": int,
  @set
  "contact": {
    @set
    "address": {@set "street": string, @set "city": string},
    @set
    "phone": string,
    @set
    "email": string,
  },
}

// This get automatically generated
let update_person = (
  state: person,
  update: (__update_person) => __update_person,
): person => {

  let updatedState = %raw(`(state) => {
    const copy = (item) => {

    if (typeof item != "object") return item

    const props = Object.getOwnPropertyNames(item);

    return props.reduce((newObject, prop) => {
      const value = item[prop]

      newObject[prop] = copy(value)

      return newObject
    }, {})

  }

  return copy(state)

  }`)


  let castUpdatedState:('a) => person = { %raw(`
        (t) =>  t
    `)
    }

  state -> updatedState -> update -> castUpdatedState
}

let me: person = {
  "name": "sean",
  "age": 32,
  "contact": {
    "address": {
      "street": "321 Main Street",
      "city": "Chicago",
    },
    "phone": "5556661234",
    "email": "sean@example.com",
  },
}

let updatedMe = update_person(me, (draft) => {
	draft["contact"]["address"]["city"] = "New York"
  draft
})

Js.log(me)
Js.log(updatedMe)

I’m going through your original use case again, and finding it difficult to justify using an object type when a record type will check almost all your boxes.

This sounds like a data model design problem to me. Let’s think about what person, business, and reseller might have in common. I would model it like this:

type person = {
  name: string,
  email: string,
  age: int,
}

type address = {
  street: string,
  city: string,
}

type contact = {
  person: person,
  address: address,
}

type business = {
  reseller: bool,
  contact: contact,
}

Now, updating an email needs to impact only a single data type, person. I think you said that your data model already exists, and it was designed to work with more loosely-typed systems like JavaScript and Clojure. So it’s natural that you’re feeling the pain of dealing with it in a strongly statically-typed system. Perhaps the way forward is to have conversion functions that take the ‘bag of data’ as input and output more structured data like shown above.

@yawaramin
Yeah, I agree that a different data model could make Records a viable option. I think your model above makes sense.

I still think that what I have come up with and my suggestions can still be good for ReScript or the ReScript community. The concept of creating more abstract functions that work over the required data structure alone, has merit. It’s the famous quote that is probably overused: “It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.”

(Side note, I never read the book, mostly because I do not plan on using Pascal)

I will let things ponder around and see what I can break :grinning:. Thanks for all your help.

@cristianoc

My question is: are you sure there aren’t more properties you’re interested in enforcing.

For Objects? I dunno, maybe? Right now, besides the lack of Object type composition, I cannot think of anything. My mind might change, I am only on day 3 of ReScript

1 Like

Obviously you’d have different pros and cons when using different languages or types of data structure, but I think it usually pays off to follow the idioms of a language instead of trying to apply to it techniques from your old stack.

Out of curiosity, what made you go away from languages that seem closer to your workflow like ClojureScript?

At work I’ve converted a pretty big code base from typescript that used similar techniques with structural typing and big objects to rescript using records and you clearly lose a bit of flexibility, but you also have great benefits. Firstly, nominal typing allows much clearer error messages. Secondly, having interfaces allows to exhaustively check every use when you change the shape of your data and this can be extremely useful when refactoring. It definitely requires to comprehensively change your way of thinking and likely refractor a significant part of your existing business logic, but you wouldn’t feel like you’re swimming against the flow.

2 Likes

@tsnobip
I am a proponent of static types (maybe future me will change his mind). Currently we use TypeScript, but the type system is not trustworthy, and the OO nature of the language basically means that unless the entire team works really hard to follow FP rules, imperative and OO programming slips into everyday code.

I have come to the conclusion that trying to do FP in non-FP language is more out of necessity to manage state and effects. It is ultimately better to move to a FP language if the ability is there.

I agree with your assessment, and everyone here is very friendly in suggesting I think about my data in a different way. I think that is a valid approach, and something I already do in TypeScript, and like you I believe it will carry over into whatever FP language we choose to go with. However, I also do a lot of duck typing (structural typing) with TS.

(Also, i cannot stress that making your functions more generic, makes them more resuable and composable)

So indeed, I could reorganize data and not need Objects. However, I recognized during my investigation of ReScript, that with a very small addition, ReScript could very easily support a very common workflow in a safe FP way. It seems silly to not have this functionality as it literally causes no breaking of FP principles (for most people), and it is type safe. Like a no brainer.

(The fact it fits my data model is a win-win)

The big picture is I think ReScript has that benefit of being in between TypeScript and PureScript. TypeScript does not enforce functional principles. PureScript is great :grinning:, but it feels foreign to JS developers. ReScript feels like JavaScript and with the ability for safe Object mutation, it would feel even more like JavaScript without losing any principles.


Secondly, having interfaces allows to exhaustively check every use when you change the shape of your data, and this can be extremely useful when refactoring.

Side note, I would like to see in ReScript an ability for building out runtime checking methods from types. I know types are compiled away, but something like reschema, that generates these methods, but built into the language as an official library.

1 Like