voonhous commented on code in PR #19596:
URL: https://github.com/apache/hudi/pull/19596#discussion_r3772927936
##########
hudi-common/src/main/java/org/apache/hudi/common/avro/HoodieAvroUtils.java:
##########
@@ -124,6 +125,29 @@ public class HoodieAvroUtils {
private static final Properties PROPERTIES = new Properties();
+ /**
+ * Resolves the Avro library version, preferring Maven's generated
pom.properties over
+ * {@link Package#getImplementationVersion()}. The latter reads the jar
manifest, which is
+ * present for a standalone avro-*.jar but is usually dropped for the avro
package once its
+ * classes are merged into a shaded/fat jar.
+ */
+ private static String resolveAvroVersion() {
+ String avroPomPropertiesPath =
"META-INF/maven/org.apache.avro/avro/pom.properties";
+ try (InputStream inputStream =
Schema.class.getClassLoader().getResourceAsStream(avroPomPropertiesPath)) {
+ if (inputStream != null) {
+ Properties avroProperties = new Properties();
+ avroProperties.load(inputStream);
+ String version = avroProperties.getProperty("version");
+ if (version != null) {
+ return version;
+ }
+ }
+ } catch (Exception ignored) {
+ // Ignoring the exception and falling back to the manifest-based version
below
+ }
+ return Schema.class.getPackage().getImplementationVersion();
Review Comment:
Following up on this thread with more evidence, because the residual null is
not just a narrower NPE - it poisons a static initializer.
The consumer named in the reporter's stack trace is
`HoodieSchemaUtils.java:66`:
```java
public static final HoodieSchema RECORD_KEY_SCHEMA = initRecordKeySchema();
```
So a null `AVRO_VERSION` gives `ExceptionInInitializerError` on first touch
and then a permanent `NoClassDefFoundError: Could not initialize class
HoodieSchemaUtils` for the rest of the JVM's life, with the real cause visible
only in the very first stack trace. That is the same failure mode
`2092890af292` ("fix: ProtoConversionUtil$AvroSupport static init under Avro
1.12", #18571) was written to fix in this module.
@Rajeev-01 your point about needing to know when the version could not be
resolved is fair, and the ask is not to hide it - it is to fail in a way that
names the problem. The repo already has both idioms:
`HoodieSchema.java:1069-1071` guards `pkg != null` and returns `"unknown"`, and
`HoodieVersion.get()` falls back to a default.
Concretely, pick one of:
- null-guard the three helpers, e.g. `return AVRO_VERSION != null &&
StringUtils.compareVersions(AVRO_VERSION, "1.12") >= 0;`, and log a warning
once at resolution time; or
- have `resolveAvroVersion()` throw explicitly when both sources miss, with
a message naming avro and the classpath.
Either is better than the current outcome, which surfaces as a
`ComparableVersion` NPE that mentions neither avro nor the classpath.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
