allthingssecurity opened a new pull request, #18764: URL: https://github.com/apache/iceberg/pull/18764
`ValueWriters.variants()` returned a JVM-wide singleton whose metadata and value writers serialize through a reused buffer, so two threads writing Avro files with variant columns could overwrite each other's bytes before they were encoded, silently writing another row's value or a corrupt one. This returns a new writer per call; callers already build one writer tree per file, so each file writer gets its own buffers. The reader side is stateless and unchanged. Spark and Flink use their own variant writers and are not affected; the generic data writers (for example in Kafka Connect) are. Test plan: new `TestVariantWriterConcurrency` writes distinct variants from two threads, each with its own writer from `ValueWriters.variants()`, and checks that every value reads back as written. On main it failed 3 of 3 runs (a value from the other thread appears); with this change it passed 3 of 3. `checkstyleMain`, `checkstyleTest` and `:iceberg-core:test` pass, except one failure of `TestFindFiles#testAsOfTimestamp` (format v3) that is a same-millisecond timing race in that test; it passed 5 of 5 re-runs. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: fully reviewed - Prompt Summary: Find concurrency bugs in Iceberg from an inventory of shared mutable state, reproduce them against released jars with stress tests, and prepare a minimal fix with a regression test. The fix, the test and this description are generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
