Call for testing: bs-platform@8.3.3-dev.1 (better -make-world performance)

2

I have a monorepo I can test it on. What setup are you expecting for bsb? I currently have bsb installed as a dev dependency in each package.

The repo is https://github.com/jacobp100/technicalc-core

3

There are no breaking changes yet.
With these changes mentioned above, you should see a noticeable speed up in -make-world nop build and hopefully no regressions

2 Likes
4

Just tried it in one of our or projects, here are my findings:

  1. yarn install took an eternity for some reason.
✨  Done in 279.54s.
  1. On the first run of yarn bsb -clean-world -make-world, I got the following errors:
sb: error: build.ninja:15: expected '=', got identifier
build  src/Persistence.reast : build_ast_from_re $src_root_dir/src/Persi...
       ^ near here
Failed
bsb: error: build.ninja:15: expected '=', got identifier
build  src/Styles.reast : build_ast_from_re $src_root_dir/src/Styles.re
       ^ near here
Failed
bsb: error: build.ninja:15: expected '=', got identifier
build  src/App.reast : build_ast_from_re $src_root_dir/src/App.re
       ^ near here
Failed
...
  1. On subsequent runs of yarn bsb -clean-world -make-world, I got this output:
Cleaning... 18 files.
Cleaning... 11 files.
bsb: error: loading 'build.ninja': No such file or directory
Failed
Cleaning... 0 files.
bsb: [15/15] src/Json.cmj
bsb: [18/18] Json_encode.cmti
bsb: [8/8] src/atdgen_codec_runtime.cmj
bsb: [10/10] atdgen_codec_runtime.cmti
File "bsconfig.json", line 5:
Error: Unsupported jsx version 2
For more details, please checkout the schema http://bucklescript.github.io/bucklescript/docson/#build-schema.json
error Command failed with exit code 2.

From the output, it is unclear which dependency has the unsupported JSX version (atdgen_codec_runtime is actually the last successfully compiled one).
Also, the URL from the error message (http://bucklescript.github.io/bucklescript/docson/#build-schema.json) gives “404 Not Found”.

  1. When accidentally running yarn bsb -clean-world -make-world in the root folder of the monorepo (which does not currently have a bsconfig.json, only the subfolders have one), I got the error message
File "<my project root>/bsconfig.json", line 1
Error: Invalid json format
error Command failed with exit code 2.

Instead of “Invalid json format”, it should say “File not found”.

5

Hi @cknitt, thanks for the feedback, this is very helpful.

yarn install took an eternity for some reason

I guess you are testing it on a Mac where I did not make a prebuilt binary

  1. On the first run of yarn bsb -clean-world -make-world, I got the following errors:

This is that you are running an old version of bs-platform which generates some old syntax of build instructions, a second build will get rid of them – to avoid such confusion, I made some changes to support both

From the output, it is unclear which dependency has the unsupported JSX version

Now we print a log when bsb is building a dependency

Also, the URL from the error message (http://bucklescript.github.io/bucklescript/docson/#build-schema.json) gives “404 Not Found”

Fixed.

6

Thanks to the feedback from @cknitt, I made a second release for testing 8.3.3-dev.2 (for Mac/Linux platform)

Some design decisions:

  • We can only support one entry point right now, since some configs have only one entry point, notably package-specs, we get from the toplevel package specs that whether its dependencies should be using es6/commonjs etc.

  • We need some user input to tell which packages are pinned packages, I am thinking of introducing an extra file, pinned-packages.json so that bsb will run generator rules for pinned packages.

There will be three kinds of packages

  • toplevel packages (decide which output format to be used, dev dependency/generator rules always run, warnings on )
  • pinned packages (dev dependency/generator rules always run, warnings on)
  • dependency packages (dev dependency/generator rules not run, warnings off)

Let me know what you think

7

@Hongbo Would it be possible to add a new property pinned-dependencies to bsconfig.json file? That would avoid the need to add new config files (there are already too many in JavaScript and OCaml imho) and it would also be consistent with other fields like bs-dependencies.

Another alternative is to allow for objects inside bs-dependencies, e.g.

  "bs-dependencies": [
    "foo",
    { "name": "bar", "pinned": true }
  ],
4 Likes
8

I thought about this, in that case, you need comment and uncomment it some time?

9

Yes, I guess. But in my experience, this does no happen very often. You pin, then you develop for some time, then potentially you unpin when changes are merged upstream, if ever. But switching back and forth is not common (in my experience, curious what others think).

10

Why dont we open an issue where people can report issues?
Anyway, seems like this breaks on a dependency we had(which didnt break before) -> reason-date-fns
(seems like its a problem in the dependency, but still this is a regression)

11

Regarding the extra config, please not yet another config file, can’t stress this enough, we should think how to remove things, not add. maintaining my dependencies in two places due to reason is already annoying(yarn/npm should be the de-facto place where i define my dependencies). now i’ll have to maintain yet another obscure(and it’ll be obscure…) config file for 1 tool.

We should really really work extra hard on aligning ourselves to how yarn/npm is doing this kind of things. a dependency, if we were to use package.json to determine dependencies(as we definitely should IMHO), this becomes much easier, no?

1 Like
12

Ok, pushed forward by manually fixing the reason-date-fns.
Now it just doesnt work:
I’ve package A that depends on package B(A–>B)
I run make-world on A. it runs fast(not that we care about speed at this point at all).
I change something in B that should break the build(e.g introduce a new argument in a func)
Run make-world again in A -> Everything passes(but i see some different logs)
When i do an actual dev flow, e.g make-world on A than watch on A, it breaks in even worse way(e.g some weird error)

Sidenote:
Can we please stop emphasizing speed? its such a grave mistake. my team and i want documentation, dev-experience improvement(better errors, better tooling etc). Its already fast enough but what good is fast compiler if we spend hours trying to understand weird errors and non deterministic behaviour?

13

Do you have anything we can reproduce step by step?
Btw: using a hash tone is not constructive, we don’t prioritize things based on the volume of criticism, it probably cause an opposite effect . @cknitt ‘s feedback is a role model

3 Likes
14

Regarding reproduction, i dont have anything at hand. but i would think that any simple repo with such constructs will behave this way. happy to do that but it’ll take me a bit - lmk if you want me to do that.

15

So in your proposal, I assume that the pinned package settings only work for toplevel packages?
Is that common that you want to pin a transitive dependency?
Assume that you have packages, a , b ,c and d where a depends on b and c, b and c depend on d. With your pinned setting only b and c can be pinned, d can only be pinned when you add it to the dependency of a, would that cover most needs?

16

Hi, it is always suggested to report bugs with a reproducible repo, that would be better use of time for both.

17

Hi @Hongbo! You are right, I had tested on macOS, that’s why the yarn install took so long.

Thanks a lot for the new test version (8.3.3-dev.2)! I tried it with the same project as before, here are my findings:

  1. If I run bsb in a directory without a bsconfig.json, I now get the correct error message <my_project>/bsconfig.json: No such file or directory. :+1:
  2. On the first run of yarn bsb -clean-world -make-world, I got an error because of an incorrect bsconfig.json in one of the dependencies. It seems that previous versions did not check for this? After fixing the dependency’s bsconfig.json, I was able to continue.
File "<my_project>/node_modules/@reason-react-native/push-notification-ios/bsconfig.json", line 2:
Error: package name is expected to be @reason-react-native/push-notification-ios but got @reason-react-native/__template__
For more details, please checkout the schema https://rescript-lang.org/docs/manual/latest/build-configuration-schema
error Command failed with exit code 2.
  1. On the next run of yarn bsb -clean-world -make-world, I still got
bsb: error: build.ninja:15: expected '=', got identifier
build  src/Persistence.reast : build_ast_from_re $src_root_dir/src/Persi...
       ^ near here
Failed
bsb: error: build.ninja:15: expected '=', got identifier
build  src/Styles.reast : build_ast_from_re $src_root_dir/src/Styles.re
       ^ near here
Failed
bsb: error: build.ninja:15: expected '=', got identifier
build  src/App.reast : build_ast_from_re $src_root_dir/src/App.re
       ^ near here
Failed
  1. bsb now outputs which dependencies it is building which is very nice. :slight_smile:
  2. Build failed because warning 103 occurred and a dependency had configured -warn-error @a. The error message said %@string is redundant here instead of @bs.string is redundant here:
  Warning number 103 (configured as error)
  <my_project>/node_modules/@reason-react-native/push-notification-ios/src/ReactNativePushNotificationIOS.re:63:22-70:22

  61 ┆ ~applicationIconBadgeNumber: int=?,
  62 ┆ ~fireDate: Js.Date.t=?,
  63 ┆ ~repeatInterval: [@bs.string] [
  64 ┆                    | `minute
   . ┆ ...
  69 ┆                    | `year
  70 ┆                  ]
  71 ┆                    =?,
  72 ┆ unit

  FFI warning: %@string is redundant here, you can safely remove it
  1. After fixing this plus several places where top-level statements were not returning unit, everything compiled fine. :tada:
1 Like
18

@Hongbo I think it’s not uncommon to pin a transitive dependency.

To show a real world example, I took Ahrefs monorepo and created a simplified version of the dependencies between packages (right now it’s all under one bsconfig, but we keep some fake pseudo-namespacing hoping to go back to pinned deps one day):

"a":  [],
"b":  [],
"c":  ["a", "b"],
"d":  ["a", "b", "c"],
"e":  ["a", "b", "c"],
...

You can see how c is a pinned dep that depends on a, b but it is also a dependency for packages d, e,…

I hope this helps, let me know if we can provide more information.

19

I think i missed something, where was “pinned” concept explained?
But if i get the rational - can’t we use package.json resolution for that?

On the repo situation, its actually very weird and inconsistent.
It’s happening on our monorepo(which is a hafty one, purely reasonml, > 10 packages) but i couldnt reproduce this in a smaller situation. It’s also not consistent.
E.g, some changes in a dependant package are being correctly detected, but some are not.

The weird thing is, i do see the error in the editor(amazing), but the compile succeeds… which is scary.
I’ll spend some more time trying to reproduce it in a smaller repo, but not sure what to do if that doesnt succeed.

As this introduces a reliability issue to the compiler, i think it’s worth to get to the bottom of this(i really hope i’m not doing anything stupid here… but i triple checked myself and everything looks legit)

20

Ok, i think i got it:


play with Test.re in “b” package(try adding/removing variables) the compiler behaves very weirdly.
21

I can’t emphasize this enough! Same can be said for emphasizing on syntax instead of semantics. (We have slightly better syntax - at cost of splitting the ecosystem and a worse developer experience for a long time - but we still have a Js.String2 module).

1 Like