[
https://issues.apache.org/jira/browse/FLINK-41327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137707#comment-18137707
]
Luo Zhengjie edited comment on FLINK-41327 at 11/4/26, 2:57 AM:
----------------------------------------------------------------
Verified against community master ce373fe8b9e4131a0d756b739449e410318a74b5 with
JDK 17 and public Maven repositories.
Before the fix: ObjectHashSetTest reproduced 2 assertion failures in 3 tests;
BinaryInPredicateITCase reproduced 4 assertion failures in 10 SQL tests, with
no errors or skips. The SQL tests disable constant folding and cover
BINARY/VARBINARY IN/NOT IN, high-bit bytes and NULL semantics.
After a minimal content-key fix, ObjectHashSetTest (3), BinaryInPredicateITCase
(10) and existing ComparisonFunctionITCase (42) all pass with no failures,
errors or skips. The 55-module selected reactor also packages successfully.
Inserted binary keys are copied; lookup keys are not copied. Other
non-byte-array objects retain their existing behavior. UUID values also use
byte[] internally and take the same content-based path; UUID-specific SQL and
performance benchmarking are not claimed here.
Full clean verify stopped in the unchanged Runtime module at
TieredResultPartitionTest.testRequestBuffersAfterPoolSizeDecreased with a
storage-tier selection error. The local filesystem has 4.94% usable space,
below the disk tier's fixed 5% reserve. Runtime reported 10,249 tests, 0
assertion failures, 1 error and 384 skips; subsequent reactor modules were not
verified. No tests or checks were bypassed. Full verification and the Flink PR
remain pending until the build environment is corrected.
Generated-by: OpenAI Codex (GPT-5)
was (Author: JIRAUSER315273):
Verified against community master ce373fe8b9e4131a0d756b739449e410318a74b5 with
JDK 17 and public Maven repositories.
Before the fix: ObjectHashSetTest reproduced 2 assertion failures in 3 tests;
BinaryInPredicateITCase reproduced 4 assertion failures in 10 SQL tests, with
no errors or skips. The SQL tests disable constant folding and cover
BINARY/VARBINARY IN/NOT IN, high-bit bytes and NULL semantics.
After a minimal content-key fix, ObjectHashSetTest (3), BinaryInPredicateITCase
(10) and existing ComparisonFunctionITCase (42) all pass with no failures,
errors or skips. The 55-module selected reactor also packages successfully.
Inserted binary keys are copied; lookup keys are not copied. Other
non-byte-array objects retain their existing behavior. UUID values also use
byte[] internally and take the same content-based path; UUID-specific SQL and
performance benchmarking are not claimed here.
Full clean verify is still in progress; no full-suite success or Flink PR is
claimed yet.
Generated-by: OpenAI Codex (GPT-5)
> Binary IN/NOT IN predicates compare byte arrays by identity in SEARCH point
> sets
> --------------------------------------------------------------------------------
>
> Key: FLINK-41327
> URL: https://issues.apache.org/jira/browse/FLINK-41327
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Reporter: Luo Zhengjie
> Priority: Major
>
> The runtime ObjectHashSet used for SEARCH point sets delegates to HashSet.
> SQL BINARY/VARBINARY values are represented as byte[], whose Java equality is
> identity-based. A separately allocated value containing the same bytes as an
> IN-list literal can therefore be reported as absent; NOT IN can produce the
> opposite incorrect result.
> h3. Minimal runtime reproduction
> {code:java}
> ObjectHashSet<byte[]> set = new ObjectHashSet<>();
> set.add(new byte[] {1});
> boolean found = set.contains(new byte[] {1});
> // Expected: true. Actual: false.
> {code}
> Reproduced on community master ce373fe8b9e4131a0d756b739449e410318a74b5
> (2.5-SNAPSHOT), using Java 17 and a source reactor build. New runtime
> regression tests run: 3; failures: 2; errors: 0. The failures cover
> equal-content membership and isolation from mutation of the inserted byte
> array.
> CodeGeneratorContext.addReusableHashSet selects ObjectHashSet for binary
> values. SQL coverage must use non-folded input and enough point literals to
> exercise the SEARCH hash-set path rather than a short OR chain. SQL
> integration validation is still in progress; no full-suite success is claimed.
> h3. Proposed fix
> Use content-based equality and hashing for byte-array keys, retain normal
> equality for other key types, and copy inserted byte-array keys so later
> caller mutation cannot invalidate membership. No dependency or storage-format
> changes.
> Prepared with assistance from OpenAI Codex (GPT-5).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
