imbajin opened a new issue, #3397:
URL: https://github.com/apache/hugegraph/issues/3397

   ### Bug Type
   
   Logic / missing results in precise union queries.
   
   ### Environment
   
   Development source reviewed in PR #3395, Java 17. Reproduced with the 
production iterators and SortShuffle in a standalone JVM; no running Store 
cluster was used.
   
   ### Expected and actual behavior
   
   Once `UnionFilterIterator` has more than 10,000 map entries, it selects the 
SortShuffle/reducer path. `NoRepeatValueIterator` omits an entire 
comparator-equal group when that group occurs more than once, rather than 
returning one representative.
   
   A probe with **20,000 keys, each read twice** returns **0 rows**; expected 
**20,000 rows**, one per key. Each occurrence is a newly allocated 
BackendColumn with identical name/value bytes for that key.
   
   The same query shape should retain each overlapping key once. 
`MapUnionIterator` already does so on the alternate small-query path.
   
   ### Reproduction
   
   1. Generate keys as four-byte integer arrays and two occurrences per key, 
with fresh BackendColumn objects and arrays for every row.
   2. Populate an IntersectionWrapper from the first pass and call `proc()`; 
all keys in this dataset are repeated candidates.
   3. Construct UnionFilterIterator over an independent second pass, using the 
production BackendColumn serializer and name comparator.
   4. Consume normally with `while (iterator.hasNext()) { iterator.next(); 
count++; }`. The observed count is zero.
   
   The 10,000 threshold selects the union's sorting branch. SortShuffle has its 
own higher physical-file threshold, so this defect can occur before it writes 
an actual spill file; its file-backed path uses the same reducer.
   
   ### Cause and focused follow-up
   
   `saveElement()` preserves repeated candidates for sorting, but 
`NoRepeatValueIterator.hasNext()` skips groups where its duplicate counter is 
positive, including the final group. Fix reduction to emit a representative 
once for each matching name group, and cover repeated keys across flush 
boundaries, repeated probes, and EOF. Preserve the intended 
representative/value rule.
   
   Sources: [UnionFilterIterator and 
reducer](https://github.com/apache/hugegraph/blob/5f70abee9e620950301477d1c9702a0a9b75ad1a/hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/business/itrv2/UnionFilterIterator.java#L58-L238),
 [MapUnion alternate query 
path](https://github.com/apache/hugegraph/blob/5f70abee9e620950301477d1c9702a0a9b75ad1a/hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/business/BusinessHandlerImpl.java#L610-L650).
   
   This is independent of the in-memory map-key defect tracked in 
https://github.com/apache/hugegraph/issues/3396 and predates the changed 
iterator-state logic in PR #3395. Keep it as a separate correctness fix rather 
than expanding that PR.
   
   Issue search performed for NoRepeatValueIterator and UnionFilterIterator; no 
matching existing issue was found.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to