dsmiley opened a new pull request, #4873: URL: https://github.com/apache/solr/pull/4873
complement() and intersect() streaming expressions silently returned wrong results whenever their on= clause mapped two differently-named fields (e.g. on="_parent_document_id=document_id"), with no exception raised. Root cause: both decorators compared tuples across streamA/streamB using streamA.getStreamSort() - a FieldComparator whose left and right field names are both streamA's own field. Applied to a streamB tuple, this always read a missing field as null, and FieldComparator's null-handling branch returns a constant, non-negative result. The on= mapping was honored by eq.test() but never by the comparison that drives the merge, so a non-matching pair was never recognized as "streamA's value is less", and instead of advancing streamB it discarded it, one tuple at a time. The first streamA value absent from streamB therefore drained streamB to EOF, after which every remaining streamA tuple hit the "streamB is EOF" branch: complement() emitted its entire input and intersect() emitted nothing. Fix: reuse the equalitor-derived comparator that innerJoin/leftOuterJoin/ fullOuterJoin already build correctly (BiJoinStream.createIterationComparator, now promoted to StreamEqualitor.deriveComparator so BiJoinStream, Complement- Stream, and IntersectStream share one implementation). Also: - Added StreamEqualitor.isDerivedFromLeft/isDerivedFromRight so Complement/ IntersectStream's precondition check validates each stream against the correct side of an asymmetric on=, instead of the old isDerivedFrom(), which OR's the two field checks together and can't detect an asymmetric mismatch. - ComplementStream/IntersectStream deduped streamB with the full (asymmetric) equalitor, which compared streamB tuples using streamA's field name and so never matched; fixed via StreamEqualitor.deriveRightEqualitor(eq). - Added StreamEqualitor.assertFieldsPresent(), called at the cross-stream comparison site, to fail loudly if an on= field is entirely absent from a tuple (a wiring bug) rather than silently treating it as null. - Added regression tests using an asymmetric on= where streamA's first value is absent from streamB (the condition that drains streamB), asserting exact membership plus the |complement| + |intersect| == |streamA| invariant, and a focused test for the streamB dedup fix. - Updated the complement/intersect ref guide sections with the sort/on= precondition and the sanity-check invariant. https://issues.apache.org/jira/browse/SOLR-18418 _diagnosed and solution/fix written entirely with AI_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
