cshuo commented on PR #19866: URL: https://github.com/apache/hudi/pull/19866#issuecomment-5597629019
@vinothchandar The increase is expected: Lance’s native libraries account for ~155 MiB. The default bundle remains ~37 MiB; the larger variant requires opting into the `flink-lance` profile. External runtime dependencies are possible, but users would currently need to assemble the required JARs: the Lance community’s Flink connector `lance-flink`, supports only Flink 1.18–1.20, and it's not a drop-in replacement: it relocates Arrow classes into its own namespace, while Hudi references the original Arrow packages. It also uses Lance 7.0.0, whereas Hudi uses 4.0.0. With the default Hudi Flink bundle, if we wanna make lance a runtime dependency, users would need to manually add these dependencies, using versions compatible with Hudi: - `org.lance:lance-core` - `org.apache.arrow:arrow-vector` - `org.apache.arrow:arrow-format` - `org.apache.arrow:arrow-memory-core` - `org.apache.arrow:arrow-memory-netty` - `org.apache.arrow:arrow-memory-netty-buffer-patch` - `org.apache.arrow:arrow-c-data` - `org.questdb:jar-jni` - `com.google.flatbuffers:flatbuffers-java` - `io.netty:netty-buffer` - `io.netty:netty-common` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
