@Hongbo Excellent! Getting closer to the monorepo support.
So basically, if I understand this correctly, this version should work fine in a monorepo setup except that file watching is not done yet and therefore the user always needs to run bsb -make-world in the project root whenever something is changed somewhere. Right?
I tested with one of our projects that is using a monorepo. It has four subpackages:
common-base
common-ui
guide-app
guide-web
with guide-app and guide-web using common-ui and common-base, and common-ui using common-base.
I added a bsconfig.json with the following contents to the project root:
Running bsb -make-world in the project root works fine and compiles all the projects.
Even if, like in our project, there are no source files in the root project, sources cannot be omitted from the bsconfig.json. Could this restriction be relaxed?
I now changed the interface of a React component in the common-ui project by renaming an optional parameter, adapted the calling sites in guide-app and guide-web and re-ran bsb -make-world in the project root. This gave me the following error:
We've found a bug for you!
<my project>/node_modules/guide-web/src/operations/OperationDetailsRoutes.re
It's possible that your build is stale.
Try to clean the artifacts and build again?
Here's the original error message
The files <my project>/guide-web/lib/bs/src/operations/operationDetailsTeam.cmi
and <my project>/guide-web/lib/bs/src/operations/operationGisView.cmi
make inconsistent assumptions over interface PositionStatusIcon-CommonUi
I now changed the interface of a React component in the common-ui project by renaming an optional parameter, adapted the calling sites in guide-app and guide-web and re-ran bsb -make-world in the project root.
Ah, I see the issues, we still have one problem to solve, but we are close. Thanks for the valuable feedback. Note next week is holiday, so expect slow response
what’s your expectation on rebuilding when dependencies change?
If we treat package as a black box, when your dependency gets changed, all its decendant need recompiled (some saving could be done: e.g, parsing and depfile generation), would it match your expectation?
More fine grained rebuilding needs some re-architecture
Yes, a change in a pinned dependency should trigger a rebuild of affected files in all descendants.
The goal is to have a similar experience when changing a pinned package as you have now with the main src.
In the best case, users would not even notice that they edited a dependency, because the watcher picks it up like a main module.
I agree with @fham, ideally the experience when changing something in a pinned package would be the same as when changing something in the main package. Files depending on the changed files should be recompiled, and no “stale build” errors should occur.
If the re-architecture for fine-grained tracking/rebuilding of the dependencies between packages causes a lot of effort, then maybe it would be best to do a first release without fine-grained rebuilding (treat package as a black box) and iterate on that later?
If I try to navigate from the dashboard’s module to the pinned toolkit’s module (Target.re for example) with ocaml-lsp, you will be navigated to file in packages/toolkit/lib/ocaml/Target.re instead of packages/toolkit/src/Target.re
Yes, it’s intentional. The thing is that it seems merlin search file by include paths and we treat a package dependency as a black box. However, the original absolute path is encoded in the cmt file, so ideally merlin should be able to find the original path, in that case, we don’t need install those source files
@Hongbo
I think dev dependencies of pinned dependencies are not building correctly
I have this three packages:
mainPackage
packageB
OpsAppsDemo
Dependencies between packages:
mainPackage depends on packageB and dev depends on OpsAppsDemo
packageB dev depends on OpsAppsDemo too
mainPackage and packageB have OpsAppsDemo in pinned-dependencies in bsconfig.json
Dependency pinned on @org/packageB
bsb: [1/1] demo/Demo.cmj
FAILED: demo/Demo.cmj
We've found a bug for you!
/mainPackage/node_modules/@org/packageB/demo/Demo.re:210:4-18
208 │
209 │ make
210 │ |> OpsAppsDemo.hot
211 │ |> OpsAppsDemo.render(~requireAuth=true, ~navigation=[], _);
The module or file OpsAppsDemo can't be found.
- If it's a third-party dependency:
- Did you list it in bsconfig.json?
- Did you run `bsb` instead of `bsb -make-world`
(latter builds third-parties)?
- Did you include the file's directory in bsconfig.json?
FAILED: cannot make progress due to previous errors.
Failure: /mainPackage/node_modules/bs-platform/darwin/ninja.exe
Location: /mainPackage/node_modules/@org/packageB/lib/bs
And during another CI build i got mentioned error above about inconsistent assumptions over interface
Hi, this is a misunderstanding of pinned-dependencies, such field only makes sense for main-package, it seems that you need put packageB and OpsAppsDemo pinned in mainPackage
@Hongbo so do you mean that It is only possible to have only one root bsconfig.json with pinned-packages support? There is one problem with this approach, it won’t work with https://github.com/reasonml-editor/reasonml-idea-plugin for instance, it builds closest bsconfig in case of monorepo…
Currently I’ve updated all package’s bsconfigs with local dependencies as pinned, because this follows our workflow to develop each package with own bsb process
Hi, I published 8.4.0-dev.3 (for Linux and Mac) which should have your item 3 fixed.
If not, would you upload a small repo so that I can reproduce it.
Hope everything works this time!
Hi, the current architecture requires a main entry point, in the main package you specify the pinned dependencies. That’s also how other workspace gets supported like cargo for rust.
Unfortunately we don’t support build in your dependency directory, it’s required that you run all the bsb command in the main package root directory. This is something by design, I will explain why it works that way in a separate post.
Honestly I think this breaks the purpose of monorepo, packages are not isolated as we have one entry point now. So users won’t be able to use non unique module name across packages (only with a namespace), also different versions of dependencies won’t work either?
UPD I tried 8.4.0-dev.3 and all CI builds are passing but clean rebuild of internal package fails.
So it seems that order of build matters a lot, build passes when executed via lerna, it uses their dependency graph resolution for ordering. But when you build one package it might fail due to some transitive dev dep of pinned package.
Hi, will have a look if you have a reliable way to reproduce it.
So users won’t be able to use non unique module name across packages (only with a namespace)
This is independent of the build support, in general you can not have same name of modules without namespace regardless of what build system or package manager you use.
also different versions of dependencies won’t work either?
This is currently out of our scope, we will see if we can take over the package manager in the long term.
as we have one entry point now
Yes, this is by design. Suppose you have one package which instructs es6 output, the other asks commonjs output, we need have one source of truth which the main entry point for.
Just tested.
Looking good, I see that currently the bs-config documentation is missing the pinned-deps and also the schema is missing it(so autocomplete doesn’t work).
What is the plan for making watch work? although at this point, we’re finally able to enjoy the fast compiler(no need to do a cleanbuild anymore!! yayy!)
And as always, i’m against losing focus and creating a package manager(vs first making what we currently have on par with competition). yarn is awesome, but maybe i’ve missed the discussion regarding the cost-benefit of this.
I don’t understand why users should be forced to have one entry point by design instead of being able to develop any package that can contain many pinned-packages from the same codebase? Basically having many entry points (packages)?
As I was describing my use case above with the following dependencies:
-> == dependency, ~> == dev dependency,
A -> B, A ~> C,
B ~> C
Pinned packages:
*> == pinned dep
A *> B, C
B *> C
Scenario 1: Developing B in isolation and making needed changes to C without re-building B with -make-world to get compilation errors related to changes in C.
Scenario 2: Developing A later and making the changes to B and C. I can also have more packages like A, that depend and dev depend on other pinned packages… I don’t understand why i need to have single top level bsconfig to support pinned-dependencies of already isolated packages, that are already root entry points for bsb?
Sorry if I am totally missing something… But when I am developing A it should be the main entry point for bsb, I don’t run any other processes but facing compilation error above above missing pinned dev dependency…
Might be we have a miss-understanding in entry point I understand this as following: bsconfig that is used by bsc, so at the moment of building there is always one entry point, and it should be possible to build packages with pinned dependencies on their own, because they are root packages when you build them standalone, not as a dependency, and it also should be possible to have this packages as dependencies of other packages and also as a pinned dependencies and build this other packages as entry points too.
packages
├── app # depends on my-lib, dev depends on my-shared-dev-lib, pinned deps: my-shared-dev-lib, my-lib
├── my-lib # dev depends on my-shared-dev-lib, pinned deps: my-shared-dev-lib
└── my-shared-dev-lib
calling build via lerna (yarn build which just builds every package in order that is resolved by lerna) works just fine, but if you will try to build packages/app via yarn build or root alias for this yarn build:app compiler will throw:
bsb: [1/6] dev/DevDemo.ast
bsb: [2/6] src/MyLib.ast
bsb: [3/6] dev/DevDemo.d
bsb: [4/6] src/MyLib.d
bsb: [5/6] dev/DevDemo.cmj
FAILED: dev/DevDemo.cmj
We've found a bug for you!
/Projects/bsb-smallest-monorepo-example/packages/my-lib/dev/DevDemo.re:1:1-12
1 │ DevLib.print("Hello, BuckleScript and Reason!");
The module or file DevLib can't be found.
- If it's a third-party dependency:
- Did you list it in bsconfig.json?
- Did you run `bsb` instead of `bsb -make-world`
(latter builds third-parties)?
- Did you include the file's directory in bsconfig.json?
Notice that i am build app but dependency that is missing in bsc is from my-lib
I understand this as following: