Looking at the packaging perspective the efforts should be minimal. At present, Hive and Spark are the only Bigtop components that can support iceberg, and only a single jar is required to be packaged into the respective component lib directories.
I guess the versions of Flink, Spark and Hive matters. https://github.com/apache/iceberg/blob/apache-iceberg-1.3.1/settings.gradle If we package the Iceberg, the jars should be installed under /usr/lib/iceburg. The jar should be added to the classpaths by settings of depending products. We need Puppet manifests for that to deploy testing environment. Smoke tests should make it certain that the combination works and the jar in the classpath does not bring conflicting dependencies. On 2023/08/09 16:28, Muralikrishna Dmmkr wrote:
Hi All, I noticed that there have been various discussions within Bigtop and Ambari communities regarding the addition of Iceberg support, Unfortunately these have not reached a conclusion yet. So, I am initiating this thread to discuss on the support for iceberg. Looking at the packaging perspective the efforts should be minimal. At present, Hive and Spark are the only Bigtop components that can support iceberg, and only a single jar is required to be packaged into the respective component lib directories. Please share your thoughts on this. Thanks Murali Krishna
