An innovative workflow for warnings/errors handling in 8.3.0-dev.1

Following the discussion of Move forward with warn-as-error by default - #16 by Hongbo, we introduce an enhanced workflow for warnings/errors handling.

Conclusion: we will treat all warnings as non-critical errors (warning numbers are configurable per project or per file), but the build system will not be blocked so that for non-critical errors, it will still type checking your whole projects and lists all your non-critical errors.

Motivation

We would like to improve the error diagnostics for editor integration. Traditionally, the editor relies on Merlin or other tools for error reporting, however, it is not reliable since such third party tools don’t have a global view of the project organization. There’re lots of frustration when the editor reports an error but in real world, the build system tells you everything is okay.

The build system is always correct except one case: warnings.

When the build system do the clean build, it will report warnings faithfully, however, during the second time, since the existing targets are already built, the warnings are completely gone, the build output can not faithfully report those missing warnings.

We need inform the build system when a warning happens to one file and rebuild it again, that’s why we are making the changes: always enforce warn-as-error.

The traditional pitfalls of turning on warning-as-errors

One major drawback of turning on warning-as-error is that it stops immediately without telling your other potential errors.

For example, suppose we have three modules, A, B, and C; the dep graph is B call A and C call B. When A has a non-critical error, the build system stops at compiling A and we don’t know the status of B or C, so we have to actually fix A first to see how it’s going with B or C.

Introduce soft-failure to improve the experience of warning-as-errors

Following the graph above, we would like to the build system to continue checking B and C even if we meet a non critical errors in A.

In this case we call A is a soft-failure, we don’t generate JS files for A, however, we still produce the needed binary artifacts for the build system to check B and C. If there are non critical errors in B, we mark B as a soft failure and continue checking C.

So essentially it will check the whole project like just turn it as warnings, there are some subtle differences:

  • When do a rebuild, those soft-failure will remind the build system to do the recheck of those soft failed modules, so the build system will tell you all the non-critical errors in the project in the rebuild. In the future, the editor can rely on it to do the error diagnostics instead of third party tools.

  • In the end, the build process will fail to ask user to fix those non-critical errors, remember the soft-failure module does not generate js file.

  • It will still be very fast. Suppose we fix the non critical error in A, the build system notice the change in A and regenerate A.js, however if its binary artifacts don’t change, B or C will not rebuild (assume it’s previous build is a success)

Potential breaking changes

Any change to the build system is subtle that it may break something. We made it available for testing your repo now in version bs-platform@8.3.0-dev.1.

To avoid breaking too many packages, when a package is built as a dependency, internally we mark off all non critical errors, in this case, only a toplevel package can fail due to non-critical errors.

For toplevel packages, we assume the developer own the project, they can choose either fix those warnings immediately or turn those numbers off.

Feedback is appreciated!

4 Likes

I just tried this out with bs-platform@8.3.0-dev.1.

It is great that now all warnings are always reported by bsb :+1:, BUT:

The fact that JS files are not generated anymore for files with warnings makes the developer experience significantly worse than before. Now if I just have an unused variable or parameter somewhere, I cannot test my code anymore until I fix that. :frowning:

(Also, warnings are now shown as errors by RLS, too. I guess that can/will be handled differently in the new editor plugin though, so that’s not much of an issue.)

4 Likes

I agree, this worsens the developer experience massively. Things that possibly don’t lead to a less correct program should still compile and yield a JS file, IMHO. I guess that is why there are warnings in the first place.

Is it not possible to patch the compiler so that warnings don’t disappear with incremental builds? I think it is super-important to have the easiest defaults here, and this would probably push off some potential new users.

2 Likes

hmm, we’re already doing a better job than any existing system so far as I know. Take dune for example, it will stop immediately right after the first error.
You can turn off warnings if you don’t care

Great you set up the previous discussion. But if you are not looking for feedback just implement it directly. People are coming with good arguments for a certain position and you don’t have a good response and just go the other way. I’ve seen this happen a number of times now. I am really questioning if you understood the arguments people have made or have a fixed opinion that isn’t going to change. I’ll happily help you with the first (you can also ask for clarification), but if it’s the second you are just waisting everyone’s time (which of course is your own choice because it’s your project). But if you ask for my feedback it’d be better to explain the trade-off that’s being made instead of completely disregarding each point (not just talking about this occasion).

This is not addressing the concerns and it doesn’t look to solve a real world issue, just putting up barriers to productivity. So to me it looks like a day to day regression and a reason not to upgrade for now. (Yes I can turn off global warnings, but that doesn’t help me because I don’t want to turn off useful warnings).

2 Likes

Hi. I made the motivation clearly in the beginning and made it crystal clear that it’s prepared for reliable editor tooling.
And I do hear the feedback so that I proposed a novel way to improve the user experience to address the concerns.
I spent lots of time (including my free time) to improve the user experience only to get your irresponsible remarks. Hearing feedback does not mean just do what you like.
Let’s be constructive

3 Likes

I try to be constructive. But the overwhelming feedback on the previous thread is that if JS is not compiled for any warning we would be way less productive in dev for sure. For CI the opinions are mixed (sometimes it makes sense to ship with a warning but in that case an annotation can fix it). I don’t see that addressed here? So my question is mostly why you are asking for our feedback if you are not addressing it (I don’t say implementing).

3 Likes

Hi, you alone don’t represent the whole community. The reliable editor tooling is something of high demand that I receive from various users feedback.
I care about every user’s opinion including you but if you keep imposing your opinion to be on behalf of the whole user base, I will not respond anymore

Ok I think I understand where this is coming from. When you develop a compiler or backend system, you are mainly in the feedback loop with the compiler. In this case it’s fine for warning to be errors because once it compiles most probably you’re on the end of the feedback loop and you can clean up the warnings.

With UI development you also want a tight feedback loop of the JavaScript, because you are actively working on for instance a component. There can be a number of variables that you are not using right at the moment but you’d like to see what is changing and see visual progress.

The productivity of the second case is inhibited with this change.

4 Likes

Exactly, when using Hot Reload / Fast Refresh with React DOM or React Native, you want to see the GUI refresh immediately when you save your changes, irrespective of any warnings that might be there.

5 Likes

Is it not possible to have both - reliable editor tooling AND - like in all previous BS versions - JS still being emitted for files with warnings?

4 Likes

Can’t we create a poll or something to more statistically gather the feedback from people?
Because I think we had some more than just jaap say, that we should still compile js files on „soft-errors“. I could be wrong though :blush:

1 Like

This kind concrete argument is welcome.
Sometimes I don’t reply, it does not mean I didn’t hear, there will always be various trade offs you are going to make

2 Likes

I am not representing the whole community, but I am describing the gist of the feedback in the previous thread. Correct me if I’m wrong. Let’s try not to engage in ad hominem attacks. (You didn’t address the content of my comment btw).

1 Like

This is something I am addressing. I am happy to hear if you have a better idea to solve it

Imagine how many people I have to talk everyday, if I have to respond to every body, my life would be miserable

1 Like

By the way I am interesting in what was not reliable when it comes to warnings in editor plugins. The warnings in ocaml-lsp are pretty reliable in my experience (but I might have just missed some unreliable cases). Are there any practical examples where they are unreliable?

In this case we call A is a soft-failure, we don’t generate JS files for A, however, we still produce the needed binary artifacts for the build system to check B and C. If there are non critical errors in B, we mark B as a soft failure and continue checking C.

Dumb question: Is it not possible to adapt the described solution so that it does produce JS for modules with a “soft-failure”? What would be the drawback of emitting JS for such modules?

2 Likes

Sorry, I hit send too quickly on my last message. Drafting a better message here.

Hi Hongbo, first of all thank you for your hard work. ReScript has been steadily improving since day one and that is–to a massive extent–down to the incredible amount of work you have done.

Anyway, my 2 cents–if there are no errors in the build, a JS file should be emitted. Not doing that is a surprising behaviour. If we want to stop the build we will use the existing configuration setting and turn on warnings-as-errors. Which I think a lot of people will also do anyway, because (as mentioned in the previous thread) it is very helpful.

4 Likes

Yes that would be an amazing solution indeed! (best of both worlds). Not sure if it’s technically possible though.

1 Like