Hello Team,
I am facing a strange dependency issue while trying to access *parseFrom()
*method of the auto generated protobuf implementation. And the issue is not
happening in my local IDE.
*Usecase*: I need to deserialize the protobuf object from databricks
environment. So i had to build a custom protobuf project which contains the
accessor method which intern calls the parseForm() method to deserialise
the passed object.
So i had built the fat jar(contains all the protobuf
dependent classes) of my protobuf project and added as an external
dependency in databricks cluster. Now while accessing the accessor method,
it throws the below error.
But if i use the same fat jar in my local IDE, it's working fine.
*Error Details:*
*NoSuchMethodError:
com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom([Ljava/lang/String;[Lcom/google/protobuf/Descriptors$FileDescriptor;)Lcom/google/protobuf/Descriptors$FileDescriptor;*
*Function call from databricks:*
ProtobufSerializer.parse(Array[Byte](10, 15, 97, 115, 100, 97, 114, 51,
101, 119, 45, 97, 115, 100, 97, 115, 100, 26, 11, 101, 120, 116, 101, 114,
110, 97, 108, 95, 105, 100, 34, 8, 50, 48, 50, 50, 48, 54, 48, 51),"eiu")
*ProtobufSerializer* is a my custom class where *parse* is the accessor
method that internally calls the *parseFrom*();
This looks like a protobuf version problem which i am trying to figure it
out.
Please find my *env details *and also attached the *pom.xml and call stack *for
better understanding
*What version of protobuf and what language are you using?*
Version: 3.19.4
protoc version : 3.19.4
Language: Java 1.8
*What operating system (Linux, Windows, ...) and version?*
macOS 12.0.1, Apple M1 chip
*What runtime / compiler are you using *
I'm using the compiler published as an artifact in a Maven repo.
I am looking for some pointers to resolve the issue.
Please let me know if you need more information
Thanks in advance
Thanks,
Lokesh
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/86aba3c2-52fe-4ff1-99cc-a51b5aea0d32n%40googlegroups.com.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";>
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>events-protobuf</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<protoc.version>3.19.4</protoc.version>
<!-- required for jdk9 -->
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<maven-assembly-plugin.version>3.1.1</maven-assembly-plugin.version>
<java.version>1.8</java.version>
</properties>
<!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java -->
<dependencies>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.19.4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java-util -->
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java-util</artifactId>
<version>3.19.4</version>
</dependency>
</dependencies>
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.5.0.Final</version>
</extension>
</extensions>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>${maven-assembly-plugin.version}</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.5.1</version>
<configuration>
<protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>at StateChangeEventsProtos.<clinit>(StateChangeEventsProtos.java:1054)
at
StateChangeEventsProtos$ExternalIdUpdated.internalGetFieldAccessorTable(StateChangeEventsProtos.java:167)
at
com.google.protobuf.GeneratedMessageV3.getAllFieldsMutable(GeneratedMessageV3.java:162)
at
com.google.protobuf.GeneratedMessageV3.getAllFields(GeneratedMessageV3.java:238)
at com.google.protobuf.TextFormat$Printer.print(TextFormat.java:298)
at
com.google.protobuf.TextFormat$Printer.access$400(TextFormat.java:273)
at com.google.protobuf.TextFormat.print(TextFormat.java:76)
at com.google.protobuf.TextFormat.printToString(TextFormat.java:143)
at com.google.protobuf.AbstractMessage.toString(AbstractMessage.java:79)
at scala.runtime.ScalaRunTime$.inner$1(ScalaRunTime.scala:272)
at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:277)
at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:285)
at
$line507c0bd312ca4196a2e6ba535134474a27.$eval$.$print$lzycompute(<notebook>:12)
at $line507c0bd312ca4196a2e6ba535134474a27.$eval$.$print(<notebook>:6)
at $line507c0bd312ca4196a2e6ba535134474a27.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)
at
scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)
at
scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:568)
at
scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
at
scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
at
scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
at
com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:219)
at
com.databricks.backend.daemon.driver.ScalaDriverLocal.$anonfun$repl$1(ScalaDriverLocal.scala:225)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:971)
at
com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:924)
at
com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:225)
at
com.databricks.backend.daemon.driver.DriverLocal.$anonfun$execute$11(DriverLocal.scala:605)
at
com.databricks.logging.Log4jUsageLoggingShim$.$anonfun$withAttributionContext$1(Log4jUsageLoggingShim.scala:28)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:94)
at
com.databricks.logging.Log4jUsageLoggingShim$.withAttributionContext(Log4jUsageLoggingShim.scala:26)
at
com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:205)
at
com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:204)
at
com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:60)
at
com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:240)
at
com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:225)
at
com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:60)
at
com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:582)
at
com.databricks.backend.daemon.driver.DriverWrapper.$anonfun$tryExecutingCommand$1(DriverWrapper.scala:615)
at scala.util.Try$.apply(Try.scala:213)
at
com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:607)
at
com.databricks.backend.daemon.driver.DriverWrapper.executeCommandAndGetError(DriverWrapper.scala:526)
at
com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:561)
at
com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:431)
at
com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:374)
at
com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:225)
at java.lang.Thread.run(Thread.java:748)
