imbajin opened a new issue, #3397:
URL: https://github.com/apache/hugegraph/issues/3397
### Bug Type
Logic / missing results in precise union queries.
### Environment
Development source reviewed in PR #3395, Java 17. Reproduced with the
production iterators and SortShuffle in a standalone JVM; no running Store
cluster was used.
### Expected and actual behavior
Once `UnionFilterIterator` has more than 10,000 map entries, it selects the
SortShuffle/reducer path. `NoRepeatValueIterator` omits an entire
comparator-equal group when that group occurs more than once, rather than
returning one representative.
A probe with **20,000 keys, each read twice** returns **0 rows**; expected
**20,000 rows**, one per key. Each occurrence is a newly allocated
BackendColumn with identical name/value bytes for that key.
The same query shape should retain each overlapping key once.
`MapUnionIterator` already does so on the alternate small-query path.
### Reproduction
1. Generate keys as four-byte integer arrays and two occurrences per key,
with fresh BackendColumn objects and arrays for every row.
2. Populate an IntersectionWrapper from the first pass and call `proc()`;
all keys in this dataset are repeated candidates.
3. Construct UnionFilterIterator over an independent second pass, using the
production BackendColumn serializer and name comparator.
4. Consume normally with `while (iterator.hasNext()) { iterator.next();
count++; }`. The observed count is zero.
The 10,000 threshold selects the union's sorting branch. SortShuffle has its
own higher physical-file threshold, so this defect can occur before it writes
an actual spill file; its file-backed path uses the same reducer.
### Cause and focused follow-up
`saveElement()` preserves repeated candidates for sorting, but
`NoRepeatValueIterator.hasNext()` skips groups where its duplicate counter is
positive, including the final group. Fix reduction to emit a representative
once for each matching name group, and cover repeated keys across flush
boundaries, repeated probes, and EOF. Preserve the intended
representative/value rule.
Sources: [UnionFilterIterator and
reducer](https://github.com/apache/hugegraph/blob/5f70abee9e620950301477d1c9702a0a9b75ad1a/hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/business/itrv2/UnionFilterIterator.java#L58-L238),
[MapUnion alternate query
path](https://github.com/apache/hugegraph/blob/5f70abee9e620950301477d1c9702a0a9b75ad1a/hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/business/BusinessHandlerImpl.java#L610-L650).
This is independent of the in-memory map-key defect tracked in
https://github.com/apache/hugegraph/issues/3396 and predates the changed
iterator-state logic in PR #3395. Keep it as a separate correctness fix rather
than expanding that PR.
Issue search performed for NoRepeatValueIterator and UnionFilterIterator; no
matching existing issue was found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]