carl-mastrangelo opened a new issue, #27700: URL: https://github.com/apache/beam/issues/27700
### What happened? I am trying to make some HTTP calls from my Java Beam (2.48.0) job. I am using OkHTTP for my HTTP dependency (4.11.0) which has a dependency on Okio (3.2.0). The Jar `beam-sdks-java-extensions-sql-jdbc` includes a pretty old version of Okio (1.6.0), which happens to conflict with the version needed by OkHTTP. As a result, the later stages of my Beam job fail with class loader errors since the beam jar is earlier on the classpath (error included below). ``` org.apache.beam.sdk.util.UserCodeException: java.lang.NoClassDefFoundError: Could not initialize class okhttp3.internal.concurrent.TaskRunner at org.apache.beam.sdk.util.UserCodeException.wrap ( org/apache.beam.sdk.util/UserCodeException.java:39 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.Filter$1.processElement ( org/apache.beam.sdk.transforms/Filter.java:211 ) at org.apache.beam.sdk.transforms.Filter$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn.process ( org/apache.beam.sdk.io/FileIO.java:872 ) at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.MapElements$2.processElement ( org/apache.beam.sdk.transforms/MapElements.java:151 ) at org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.outputWithTimestamp ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2523 ) at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1.processElement ( org/apache.beam.sdk.transforms/Reify.java:206 ) at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1.process ( org/apache.beam.sdk.transforms/ReifyTimestamps.java:70 ) at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.Reshuffle$1.processElement ( org/apache.beam.sdk.transforms/Reshuffle.java:101 ) at org.apache.beam.sdk.transforms.Reshuffle$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer ( org/apache.beam.fn.harness/BeamFnDataReadRunner.java:213 ) at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements ( org/apache.beam.sdk.fn.data/BeamFnDataInboundObserver.java:158 ) at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion ( org/apache.beam.sdk.fn.data/BeamFnDataInboundObserver.java:123 ) at org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle ( org/apache.beam.fn.harness.control/ProcessBundleHandler.java:550 ) at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType ( org/apache.beam.fn.harness.control/BeamFnControlClient.java:151 ) at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0 ( org/apache.beam.fn.harness.control/BeamFnControlClient.java:116 ) at java.util.concurrent.Executors$RunnableAdapter.call ( java/util.concurrent/Executors.java:539 ) at java.util.concurrent.FutureTask.run ( java/util.concurrent/FutureTask.java:264 ) at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run ( org/apache.beam.sdk.util/UnboundedScheduledExecutorService.java:163 ) at java.util.concurrent.ThreadPoolExecutor.runWorker ( java/util.concurrent/ThreadPoolExecutor.java:1136 ) at java.util.concurrent.ThreadPoolExecutor$Worker.run ( java/util.concurrent/ThreadPoolExecutor.java:635 ) at java.lang.Thread.run ( java/lang/Thread.java:833 ) Caused by: java.lang.NoClassDefFoundError at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:41 ) at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:47 ) at okhttp3.OkHttpClient$Builder.<init> ( OkHttpClient.kt:471 ) Caused by: java.lang.ExceptionInInitializerError at okhttp3.internal.Util.<clinit> ( Util.kt:71 ) at okhttp3.internal.concurrent.TaskRunner.<clinit> ( TaskRunner.kt:309 ) at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:41 ) at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:47 ) at okhttp3.OkHttpClient$Builder.<init> ( OkHttpClient.kt:471 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.Filter$1.processElement ( Filter.java:211 ) at org.apache.beam.sdk.transforms.Filter$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn.process ( FileIO.java:872 ) at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.MapElements$2.processElement ( MapElements.java:151 ) at org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.outputWithTimestamp ( FnApiDoFnRunner.java:2523 ) at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1.processElement ( Reify.java:206 ) at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1.process ( ReifyTimestamps.java:70 ) at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 ) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 ) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 ) at org.apache.beam.sdk.transforms.Reshuffle$1.processElement ( Reshuffle.java:101 ) at org.apache.beam.sdk.transforms.Reshuffle$1$DoFnInvoker.invokeProcessElement at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 ) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 ) at org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer ( BeamFnDataReadRunner.java:213 ) at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements ( BeamFnDataInboundObserver.java:158 ) at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion ( BeamFnDataInboundObserver.java:123 ) at org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle ( ProcessBundleHandler.java:550 ) at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType ( BeamFnControlClient.java:151 ) at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0 ( BeamFnControlClient.java:116 ) at java.util.concurrent.Executors$RunnableAdapter.call ( Executors.java:539 ) at java.util.concurrent.FutureTask.run ( FutureTask.java:264 ) at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run ( UnboundedScheduledExecutorService.java:163 ) at java.util.concurrent.ThreadPoolExecutor.runWorker ( ThreadPoolExecutor.java:1136 ) at java.util.concurrent.ThreadPoolExecutor$Worker.run ( ThreadPoolExecutor.java:635 ) at java.lang.Thread.run ( Thread.java:833 ) ``` ### Issue Priority Priority: 3 (minor) ### Issue Components - [ ] Component: Python SDK - [X] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [ ] Component: IO connector - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Samza Runner - [ ] Component: Twister2 Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
