flyrain edited a comment on pull request #3287: URL: https://github.com/apache/iceberg/pull/3287#issuecomment-962267333
The Spark 3.0 compiling issue is due to a sbt behavior, which reads all public methods in dependencies and analysis if there is any dependency missing. Spark 3.0 depends on iceberg-core, which exposes the RoaringBitmap by the method ``` public Roaring64Bitmap deletedRowPositions() ``` The workaround is to wrap Roaring64Bitmap with an interface, so that we don't expose RoaringBitmap in a public method. It has the following benefits: 1. Solve Spark 3.0 compiling issue. 2. Decouple the implementation with interface. For example, we can implement it with other data structures(e.g. set) if needed. 3. More descriptive method names from the new interface. The related sbt code: https://github.com/sbt/zinc/blob/ce0f0a4427fed23a509221a7e68d8f0ea7b88bb3/zinc/src/main/scala/sbt/internal/inc/javac/AnalyzingJavaCompiler.scala#L170 Many thanks to @aokolnychyi and @RussellSpitzer for helping debugging and solution suggestion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
