[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438370=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438370 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 28/May/20 17:50 Start Date: 28/May/20 17:50 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on pull request #11842: URL: https://github.com/apache/beam/pull/11842#issuecomment-635500201 PreCommit failures look unrelated, and Spark PVR batch failures are down to `testFlattenWithDifferentInputAndOutputCoders2` as expected. I'll go ahead and merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 438370) Time Spent: 2h 40m (was: 2.5h) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438371=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438371 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 28/May/20 17:50 Start Date: 28/May/20 17:50 Worklog Time Spent: 10m Work Description: TheNeuralBit merged pull request #11842: URL: https://github.com/apache/beam/pull/11842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 438371) Time Spent: 2h 50m (was: 2h 40m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at > sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122) >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438312=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438312 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 28/May/20 15:31 Start Date: 28/May/20 15:31 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on pull request #11842: URL: https://github.com/apache/beam/pull/11842#issuecomment-635422568 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 438312) Time Spent: 2.5h (was: 2h 20m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438311=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438311 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 28/May/20 15:29 Start Date: 28/May/20 15:29 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on pull request #11842: URL: https://github.com/apache/beam/pull/11842#issuecomment-635421424 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 438311) Time Spent: 2h 20m (was: 2h 10m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438088=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438088 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 23:22 Start Date: 27/May/20 23:22 Worklog Time Spent: 10m Work Description: TheNeuralBit opened a new pull request #11842: URL: https://github.com/apache/beam/pull/11842 Cherry pick #11784 into 2.22.0 R: @ibzib Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438089=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438089 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 23:22 Start Date: 27/May/20 23:22 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on pull request #11842: URL: https://github.com/apache/beam/pull/11842#issuecomment-634995178 Run Java Spark PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 438089) Time Spent: 2h 10m (was: 2h) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=438078=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438078 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 22:49 Start Date: 27/May/20 22:49 Worklog Time Spent: 10m Work Description: lukecwik merged pull request #11784: URL: https://github.com/apache/beam/pull/11784 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 438078) Time Spent: 1h 50m (was: 1h 40m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at > sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122) >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437940=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437940 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 18:07 Start Date: 27/May/20 18:07 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634844712 An LGTM from either Luke or Robert should be fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437940) Time Spent: 1h 40m (was: 1.5h) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437871=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437871 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 15:57 Start Date: 27/May/20 15:57 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634760176 > Sure as long as someone commits to the follow-up. Adding a TODO comment would be helpful. I added a TODO and assigned BEAM-10109 to myself. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437871) Time Spent: 1.5h (was: 1h 20m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[
https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437870=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437870
]
ASF GitHub Bot logged work on BEAM-9971:
Author: ASF GitHub Bot
Created on: 27/May/20 15:55
Start Date: 27/May/20 15:55
Worklog Time Spent: 10m
Work Description: ibzib commented on a change in pull request #11784:
URL: https://github.com/apache/beam/pull/11784#discussion_r431252300
##
File path:
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/NativeTransforms.java
##
@@ -46,7 +46,9 @@
* Returns true if an only if the Runner understands this transform and can
handle it directly.
*/
public static boolean isNative(RunnerApi.PTransform pTransform) {
-Iterator matchers =
ServiceLoader.load(IsNativeTransform.class).iterator();
+Iterator matchers =
Review comment:
```suggestion
// TODO(BEAM-10109) Use default (context) classloader.
Iterator matchers =
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 437870)
Time Spent: 1h 20m (was: 1h 10m)
> beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
> --
>
> Key: BEAM-9971
> URL: https://issues.apache.org/jira/browse/BEAM-9971
> Project: Beam
> Issue Type: Bug
> Components: test-failures
>Reporter: Kyle Weaver
>Assignee: Kyle Weaver
>Priority: P2
> Labels: portability-spark
> Fix For: 2.22.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This happens sporadically. One time the issue affected 14 tests; another time
> it affected 112 tests.
> It looks like the ClassLoader is sometimes contaminated with jars from
> /tmp/spark-*, which have already been deleted.
> 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation:
> Error during job invocation
> pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5.
> java.util.ServiceConfigurationError:
> org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform:
> Error reading configuration file
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.parse(ServiceLoader.java:309)
> at java.util.ServiceLoader.access$200(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357)
> at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
> at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
> at
> org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50)
> at
> org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189)
> at
> org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137)
> at
> org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90)
> at
> org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67)
> at
> org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90)
> at
> org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94)
> at
> org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
> at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.FileNotFoundException:
> /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437865=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437865 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 15:47 Start Date: 27/May/20 15:47 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634753120 > Is the approach in this PR acceptable for the short-term (and for 2.22.0 release branch)? Sure as long as someone commits to the follow-up. Adding a TODO comment would be helpful. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437865) Time Spent: 1h 10m (was: 1h) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437861=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437861 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 15:38 Start Date: 27/May/20 15:38 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634747637 Is the approach in this PR acceptable for the short-term (and for 2.22.0 release branch)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437861) Time Spent: 1h (was: 50m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 1h > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437477 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 03:54 Start Date: 27/May/20 03:54 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634118385 There's a ReflectHelpers class in core with some utility functions for loading services and finding class loaders: https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/util/common/ReflectHelpers.java#L247 Sorry if this is just noise, I don't have much context here, just thought I'd point it out in case its helpful. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437477) Time Spent: 50m (was: 40m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 50m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437366=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437366 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 03:40 Start Date: 27/May/20 03:40 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634315982 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437366) Time Spent: 40m (was: 0.5h) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 40m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=437314=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437314 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 27/May/20 03:33 Start Date: 27/May/20 03:33 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-634157424 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 437314) Time Spent: 0.5h (was: 20m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=436293=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436293 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 21/May/20 23:55 Start Date: 21/May/20 23:55 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #11784: URL: https://github.com/apache/beam/pull/11784#issuecomment-632403660 Run Java Spark PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 436293) Time Spent: 20m (was: 10m) > beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file) > -- > > Key: BEAM-9971 > URL: https://issues.apache.org/jira/browse/BEAM-9971 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: P2 > Labels: portability-spark > Fix For: 2.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This happens sporadically. One time the issue affected 14 tests; another time > it affected 112 tests. > It looks like the ClassLoader is sometimes contaminated with jars from > /tmp/spark-*, which have already been deleted. > 20/05/21 13:54:27 ERROR org.apache.beam.runners.jobsubmission.JobInvocation: > Error during job invocation > pipelinetest0testidentitytransform-kcweaver-0521205426-f4de06c4_51aced77-c171-4842-be1f-6c79226872e5. > java.util.ServiceConfigurationError: > org.apache.beam.runners.core.construction.NativeTransforms$IsNativeTransform: > Error reading configuration file > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.parse(ServiceLoader.java:309) > at java.util.ServiceLoader.access$200(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357) > at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) > at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) > at > org.apache.beam.runners.core.construction.NativeTransforms.isNative(NativeTransforms.java:50) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.isPrimitiveTransform(QueryablePipeline.java:189) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.getPrimitiveTransformIds(QueryablePipeline.java:137) > at > org.apache.beam.runners.core.construction.graph.QueryablePipeline.forPrimitivesIn(QueryablePipeline.java:90) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.(GreedyPipelineFuser.java:67) > at > org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:90) > at > org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:94) > at > org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:83) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /tmp/spark-5e8a8a9a-22d6-48d5-b398-1a4f5582d954/userFiles-ec74cac1-21b5-4127-b764-540636b733d0/beam-runners-core-construction-java-2.22.0-SNAPSHOT-tests.jar > (No such file or directory) > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.(ZipFile.java:230) > at java.util.zip.ZipFile.(ZipFile.java:155) > at java.util.jar.JarFile.(JarFile.java:167) > at java.util.jar.JarFile.(JarFile.java:104) > at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93) > at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69) > at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) > at >
[jira] [Work logged] (BEAM-9971) beam_PostCommit_Java_PVR_Spark_Batch flakes (no such file)
[ https://issues.apache.org/jira/browse/BEAM-9971?focusedWorklogId=436291=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436291 ] ASF GitHub Bot logged work on BEAM-9971: Author: ASF GitHub Bot Created on: 21/May/20 23:54 Start Date: 21/May/20 23:54 Worklog Time Spent: 10m Work Description: ibzib opened a new pull request #11784: URL: https://github.com/apache/beam/pull/11784 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[