Re: Index ordinal data in the taxonomy

2023-05-27 Thread Stefan Vodita
Hello, I’ve opened an issue [1] to continue this discussion and a PR [2] showing an easy way to add data about the ordinals to the taxonomy. Let me know if you think it's reasonable. Thank you, Stefan [1] https://github.com/apache/lucene/issues/12336 [2] https://github.com/apache/lucene/pull/123

Re: Index ordinal data in the taxonomy

2023-05-13 Thread Shai Erera
Hi > There's two approaches we could take initially, Both approaches look fine to me. As long as we expose the right API. I assume that if we use updatable DV, then we'll have a proper API on TaxoWrite to update the fields, but otherwise (if we'll only allow updating during Taxo rewrite) we won't

Re: Index ordinal data in the taxonomy

2023-05-13 Thread Stefan Vodita
Hello Shai, Thank you for the feedback! I'll try to answer each of the questions. > will it change the API in non-backward compatible way, or impact faceted > search performance for the common case? The new API could overload FacetsConfig.build or provide a new method in TaxonomyWriter to plug

Re: Index ordinal data in the taxonomy

2023-05-11 Thread Shai Erera
Hi Stefan, This sounds interesting and useful. It's like static scores for Lucene documents, only that we will apply them to ordinals. Since I assume it's not a very common use case though, do you know if this new functionality affects existing use cases? For example, will it change the API in non

Index ordinal data in the taxonomy

2023-05-11 Thread Stefan Vodita
Hi everyone, I work on the Lucene product search team at Amazon. We’ve been considering indexing scoring signals for ordinals into the taxonomy, which could reduce index size for some use-cases. Example Let's consider a library of research papers, where each paper is represented by a Lucene docu