Package index missing stuff

I can’t find packages using the package index. For example I typed the search word “fast” and it doesn’t find this…

Search for “zora” doesn’t find this…

I’m referring to the search feature at this location. I think this is a big problem for people investigating ReScript because it makes it look like there is less community support than there actually is.

Package Index | ReScript Documentation (rescript-lang.org)

As the docs state, PRs to the index are welcome.

I’m confused. The two examples I gave above have the proper naming and keywords and yet they don’t show up when you search the package index.


looks like they are there to me

Weird. Try it with an iPhone. I search for “zora” and “fast” and doesn’t work with either. Also tried on my computer using Edge browser which is Chrome underneath.

this is firefox on my windows 10 pc…

Yeah, actually, search feels pretty glitchy to me. Also, rescript-zora is searchable by “@dusty”, but not by “zora” or “rescript-zora” :sweat_smile:

So I guess the search could be improved :thinking:

I would say it is totally broken rather than glitchy. As I wrote before, I think this is a risk to community adoption - people who search the package index might come to the conclusion - as I did - that there are almost no packages.

2 Likes

It actually does what it is supposed to do, the problem is that npmjs’ apis are returning a limited number of items, and it doesn’t allow me to paginate through all the results.

When we created the index, this was fine, because there weren’t a lot of ReScript packages yet. Now we reached a point where results are getting truncated. So depending on how npmjs returns the data, it might be that in some deployments, rescript-zora will show up, and sometimes not.

You can find the fetching logic here.

If there’s a reliable npm packages endpoint to query the same data from, I’d be happy for any suggestions. Ideally we’d run our own indexer, but alas, this requires extra infrastructure that requires maintenance and also extra cash.

@kanishka: How did you solve this for ocaml.org? I guess Cambridge has unlimited resources to server infrastructure to provide the search index?

1 Like

cc @sgrove who’s spent some time with the npm API:s for various purposes. Sean might have some insight.

But isn’t that a separate issue?
The packages get fetched all at once and then the search is done locally on the client.

Maybe the fuse.js options just need some tweaks?

The index gets fetched once, but if the data-set doesn’t include library X, then there’s no way the client will fuzzy find it.

So when @jmagaram described the issue, I and @illusionalsagacity probably hit some old CDN cache where we still had access to rescript-fast-check and rescript-zora… for others, the data was already different… probably with access to newer packages instead.

How did you solve this for ocaml.org

I wish I was working on this! Jon Ludlam and others are doing the opam work.

It’s not about caching (or at least not always).
I just tried typing “fast” and got no results. Then I typed “rescript-fast” and got results.

Right, yeah this needs some fine-tuning with the fuzzy search options

1 Like

Yep, it’s better now!