[
https://issues.apache.org/jira/browse/PHOENIX-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853889#comment-17853889
]
Istvan Toth edited comment on PHOENIX-7132 at 6/11/24 3:47 AM:
---------------------------------------------------------------
Report of similar issue when using --jars:
It is reported fixed when switching to *.extraclasspath.*
{noformat}
(REDACTED executor 2): org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ClassNotFoundException:
org.apache.phoenix.filter.ColumnProjectionFilter
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1368)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:961)
at
org.apache.phoenix.mapreduce.PhoenixInputSplit.readFields(PhoenixInputSplit.java:91)
at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:77)
at
org.apache.spark.SerializableWritable.$anonfun$readObject$1(SerializableWritable.scala:45)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1471)
at
org.apache.spark.SerializableWritable.readObject(SerializableWritable.scala:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1184)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2321)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:502)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:460)
at
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:87)
at
org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:129)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:510)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException:
org.apache.phoenix.filter.ColumnProjectionFilter
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at
org.apache.hadoop.hbase.util.DynamicClassLoader.loadClass(DynamicClassLoader.java:135)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1359)
... 32 more
{noformat}
was (Author: stoty):
Report of similar issue when using --jars:
{noformat}
(REDACTED executor 2): org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.ClassNotFoundException:
org.apache.phoenix.filter.ColumnProjectionFilter
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1368)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:961)
at
org.apache.phoenix.mapreduce.PhoenixInputSplit.readFields(PhoenixInputSplit.java:91)
at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:77)
at
org.apache.spark.SerializableWritable.$anonfun$readObject$1(SerializableWritable.scala:45)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1471)
at
org.apache.spark.SerializableWritable.readObject(SerializableWritable.scala:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1184)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2321)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:502)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:460)
at
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:87)
at
org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:129)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:510)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException:
org.apache.phoenix.filter.ColumnProjectionFilter
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at
org.apache.hadoop.hbase.util.DynamicClassLoader.loadClass(DynamicClassLoader.java:135)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1359)
... 32 more
{noformat}
> HBase cannot load ClientRpcControllerFactory when adding connector with the
> --jar option to Spark
> -------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-7132
> URL: https://issues.apache.org/jira/browse/PHOENIX-7132
> Project: Phoenix
> Issue Type: Bug
> Components: connectors, core
> Affects Versions: connectors-6.0.0, 5.2.0
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Major
>
> I have noticed this today when working with the shaded spark connector jar:
> {noformat}
> 23/12/01 06:01:34 WARN ipc.RpcControllerFactory: Cannot load configured
> "hbase.rpc.controllerfactory.class"
> (org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory) from
> hbase-site.xml, falling back to use default RpcControllerFactory
> {noformat}
> -We should be able to avoid this by not relocating these classes at all.-
> -This is only a problem for shaded artifacts that do not include HBase, like
> the shaded connectors and the planned phoenix-client-byo-hbase variant.-
> -In the full-fat shaded clients phoenix and HBase has the same shading, and
> HBase is able to find the shaded class.-
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
