Gatsby-Lee opened a new issue, #11783:
URL: https://github.com/apache/hudi/issues/11783
**Describe the problem you faced**
* Hudi 0.14.1
* Enabled Metadata Table + Enabled Bloom filter Indexing
* When enabling "hoodie.bloom.index.use.metadata=true" to use the Bloom
filter Indexing in Metadata Table, it started failing.
**To Reproduce**
Steps to reproduce the behavior:
1. Enable Metadata Table + Bloom Filter Indexing
2. Enable "hoodie.bloom.index.use.metadata=true"
**Expected behavior**
Once "hoodie.bloom.index.use.metadata=true" is enabled, it has to work with
no issue
**Environment Description**
* Hudi version : 0.14.1-amazon-0 ( EMR on EKS )
* Spark version : Spark 3.5 ( EMR 7.1 )
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```
2024-08-15 01:25:00
at
com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
at
org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:279)
at
org.apache.spark.broadcast.TorrentBroadcast$.$anonfun$blockifyObject$4(TorrentBroadcast.scala:365)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:95)
at
org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:367)
at
org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:161)
at
org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:99)
at
org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:38)
at
org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:78)
at
org.apache.spark.SparkContext.broadcastInternal(SparkContext.scala:1670)
at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1652)
at
org.apache.spark.api.java.JavaSparkContext.broadcast(JavaSparkContext.scala:546)
at
org.apache.hudi.index.bloom.SparkHoodieBloomIndexHelper.findMatchingFilesForRecordKeys(SparkHoodieBloomIndexHelper.java:113)
at
org.apache.hudi.index.bloom.HoodieBloomIndex.lookupIndex(HoodieBloomIndex.java:135)
at
org.apache.hudi.index.bloom.HoodieBloomIndex.tagLocation(HoodieBloomIndex.java:91)
at
org.apache.hudi.table.action.commit.HoodieWriteHelper.tag(HoodieWriteHelper.java:59)
at
org.apache.hudi.table.action.commit.HoodieWriteHelper.tag(HoodieWriteHelper.java:41)
at
org.apache.hudi.table.action.commit.BaseWriteHelper.write(BaseWriteHelper.java:59)
... 95 more
Caused by: java.lang.IllegalArgumentException: Unable to create serializer
"com.esotericsoftware.kryo.serializers.FieldSerializer" for class:
java.util.concurrent.locks.ReentrantReadWriteLock
at
com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:65)
at
com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:43)
at com.esotericsoftware.kryo.Kryo.newDefaultSerializer(Kryo.java:396)
at com.twitter.chill.KryoBase.newDefaultSerializer(KryoBase.scala:62)
at com.esotericsoftware.kryo.Kryo.getDefaultSerializer(Kryo.java:380)
at
com.esotericsoftware.kryo.util.DefaultClassResolver.registerImplicit(DefaultClassResolver.java:74)
at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:508)
at
com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97)
at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:540)
at
com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:75)
... 116 more
Caused by: java.lang.reflect.InvocationTargetException
at
jdk.internal.reflect.GeneratedConstructorAccessor112.newInstance(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at
com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:51)
... 125 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
field private final java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock
java.util.concurrent.locks.ReentrantReadWriteLock.readerLock accessible: module
java.base does not "opens java.util.concurrent.locks" to unnamed module
@75db5df9
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at
java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.buildValidFields(FieldSerializer.java:283)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.rebuildCachedFields(FieldSerializer.java:216)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.rebuildCachedFields(FieldSerializer.java:157)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.<init>(FieldSerializer.java:150)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.<init>(FieldSerializer.java:134)
at
com.esotericsoftware.kryo.serializers.FieldSerializer.<init>(FieldSerializer.java:130)
... 130 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
