sfc-gh-pbennes opened a new issue #3958: URL: https://github.com/apache/iceberg/issues/3958
Greetings, I run the [Google Linkage Checker](https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/Linkage-Checker-Enforcer-Rule) to look for [linkage errors](https://jlbp.dev/glossary.html#linkage-error) as part of my Java builds and noticed that iceberg has some when adding iceberg-api, iceberg-core, iceberg-hive-metastore to a project: ``` [INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-linkage-checker) @ example --- [ERROR] Linkage Checker rule found 53 reachable errors: Class org.apache.hadoop.conf.Configurable is not found; referenced by 6 class files org.apache.iceberg.CatalogUtil (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopCatalog (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopConfigurable (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopTables (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.jdbc.JdbcCatalog (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hive.HiveCatalog (org.apache.iceberg:iceberg-hive-metastore:0.12.1) Cause: Unknown Class org.apache.hadoop.conf.Configuration is not found; referenced by 13 class files org.apache.iceberg.CatalogUtil (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.SerializableTable (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopCatalog (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopFileIO (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopInputFile (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.HadoopTables (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.SerializableConfiguration (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hadoop.Util (org.apache.iceberg:iceberg-core:0.12.1) org.apache.iceberg.hive.CachedClientPool (org.apache.iceberg:iceberg-hive-metastore:0.12.1) org.apache.iceberg.hive.HiveCatalog (org.apache.iceberg:iceberg-hive-metastore:0.12.1) org.apache.iceberg.hive.HiveCatalogs (org.apache.iceberg:iceberg-hive-metastore:0.12.1) org.apache.iceberg.hive.HiveClientPool (org.apache.iceberg:iceberg-hive-metastore:0.12.1) org.apache.iceberg.hive.HiveTableOperations (org.apache.iceberg:iceberg-hive-metastore:0.12.1) Cause: Unknown ... ``` How to reproduce: Create this pom.xml and run `mvn verify`: ``` <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> <modelVersion>4.0.0</modelVersion> <groupId>dev.null</groupId> <artifactId>example</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>example</name> <dependencies> <dependency> <groupId>org.apache.iceberg</groupId> <artifactId>iceberg-core</artifactId> <version>0.12.1</version> </dependency> <dependency> <groupId>org.apache.iceberg</groupId> <artifactId>iceberg-api</artifactId> <version>0.12.1</version> </dependency> <dependency> <groupId>org.apache.iceberg</groupId> <artifactId>iceberg-hive-metastore</artifactId> <version>0.12.1</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>3.0.0-M3</version> <dependencies> <dependency> <groupId>com.google.cloud.tools</groupId> <artifactId>linkage-checker-enforcer-rules</artifactId> <version>1.5.12</version> </dependency> </dependencies> <executions> <execution> <id>enforce-linkage-checker</id> <phase>verify</phase> <goals> <goal>enforce</goal> </goals> <configuration> <rules> <LinkageCheckerRule implementation="com.google.cloud.tools.dependencies.enforcer.LinkageCheckerRule"> <reportOnlyReachable>true</reportOnlyReachable> </LinkageCheckerRule> </rules> </configuration> </execution> </executions> </plugin> </plugins> </build> </project> ``` As best I can tell, it's [because hadoop-common is listed as a compileOnly](https://github.com/apache/iceberg/blob/master/build.gradle#L236) dependency which prevents it from ending up in [the public pom](https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-core/0.12.1/iceberg-core-0.12.1.pom). When there are optional dependencies it's normal to have these linkage errors, though it's not obvious this is a case of that. Either hadoop-common etc. should be marked as optional dependencies if they're only needed for functionality related to hadoop, or just included as regular dependencies so that the build system can correctly pull in the needed classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
