[GitHub] [lucene-solr] dxl360 commented on pull request #2080: LUCENE-8947: Skip field length accumulation when norms are disabled

2020-11-23 Thread GitBox
dxl360 commented on pull request #2080: URL: https://github.com/apache/lucene-solr/pull/2080#issuecomment-732411622 Had offline discussion with @mikemccand. Maybe we can change the type of `invertState.length` from `int` to `long` and keep the current check on field length/termFreq

[GitHub] [lucene-solr] dxl360 commented on pull request #2080: LUCENE-8947: Skip field length accumulation when norms are disabled

2020-11-17 Thread GitBox
dxl360 commented on pull request #2080: URL: https://github.com/apache/lucene-solr/pull/2080#issuecomment-729475817 Original implementation accumulates `int invertState.length` (number of tokens) by term frequency and will overflow if the term frequency is too large. Can we increment

[GitHub] [lucene-solr] dxl360 commented on pull request #2080: LUCENE-8947: Skip field length accumulation when norms are disabled

2020-11-17 Thread GitBox
dxl360 commented on pull request #2080: URL: https://github.com/apache/lucene-solr/pull/2080#issuecomment-729332268 > I'm concerned about this change: other things will overflow if you have too many term frequencies in a field. Currently frequency is bounded by 2^32-1 within a doc, and