FrankChen021 commented on PR #19972: URL: https://github.com/apache/druid/pull/19972#issuecomment-5249249136
Closing this exact head (`571f3ca12642198f092f94a3cb849bce67600585`) as **WONT_FIX_LARGE_CHANGE / CLOSED_HIGH_EFFORT**. This is not superseded: current upstream `master` is `2b02e7a28aebc0f7ce6d211d2ca1ebbbbced4139` and still uses 2.2.0. All 22 failed checks have the same PR-caused root, across CodeQL, every Java/QTest shard, packaging, static checks, strict compilation, OpenRewrite, web checks, and validate-dist: Maven stops in `druid-processing` because it cannot resolve a **jar** at `org.apache.datasketches:datasketches-memory:7.0.0`. No test workload actually starts. The upstream 7.0.0 release changed its artifact layout: * `org.apache.datasketches:datasketches-memory:7.0.0` is now an aggregator with `<packaging>pom</packaging>`, so there is intentionally no jar at this coordinate. * The released jars are `datasketches-memory-java11` and `datasketches-memory-java17_25`. Druid is on Java 25, so only the latter is relevant. * That jar requires both `--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED` and `--add-exports=java.base/sun.nio.ch=ALL-UNNAMED`. Druid has the first, but currently only **opens**, rather than exports, `sun.nio.ch`; the production launch scripts/docs and task-runner propagation would need coordinated updates. * The new artifact remains Apache-2.0 and has no runtime transitive dependencies, but Druid's artifact/license metadata must change. Changing only the artifact ID is unsafe: * `datasketches-java:4.2.0` is compiled against and transitively depends on the old `datasketches-memory:2.2.0` coordinate. Since 7.0.0 uses a new coordinate, Maven would retain 2.2.0 and place two jars containing the same `org.apache.datasketches.memory` packages on the classpath unless exclusions and packaging are redesigned. * The 2.2.0→7.0.0 public API/ABI is not compatible. 7.0.0 removes `BaseState`, `BaseBuffer`, `Utf8CodingException`, map/handle types, boolean-array and UTF-8 methods, atomic long methods, and changes resource lifecycle/positional interfaces. `datasketches-java:4.2.0` bytecode itself references removed `BaseBuffer`; for example, KLL code invokes `WritableBuffer.incrementPosition(long): BaseBuffer`, while 7.0.0 returns `Positional`. * Druid directly imports the Memory API in 157 Java files (129 production files). Its custom `SafeWritableBase`, `SafeWritableMemory`, and `SafeWritableBuffer` implementations directly implement removed interfaces and methods, so this requires a broad source migration rather than a dependency-only fix. A coordinated Java 25 migration is larger still. DataSketches Java 9.0.0 is the Java 25 line, but its release notes explicitly describe incompatible refactoring across the library: it replaces DataSketches Memory usage with FFM and renames public sketch classes. Druid's sketch extensions and external/plugin SPI consumers would need corresponding source and binary-compatibility work. Upstream states that serialized sketch images remain backward-readable across these software versions, but Druid still needs cross-version/golden verification for persisted Theta/HLL/KLL/quantiles columns, query-result interchange, ingestion rollup, mixed-version clusters, MSQ frames/durable shuffle, frame files, and third-party extensions. The Java 17/25 Memory artifact also removes creation/modification of memory-mapped files, which requires explicit coverage even though Druid primarily wraps its own mapped buffers. No GitHub repository or global advisory-database entries were found for this package, so there is no security emergency that justifies bypassing those checks. Authoritative release details: * https://github.com/apache/datasketches-memory/releases/tag/7.0.0 * https://github.com/apache/datasketches-java/releases/tag/9.0.0 This should be handled as a dedicated, coordinated DataSketches/Memory migration with compatibility fixtures and focused rollout testing, not as a one-line Dependabot bump. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
