[
https://issues.apache.org/jira/browse/BEAM-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Hulette updated BEAM-12207:
---------------------------------
Description:
The first run in the Python 3.8 suite with this particular failure appears to
be https://ci-beam.apache.org/job/beam_PostCommit_Python38/1104
{code}
06:26:36 OK (SKIP=6)
06:26:36
06:26:38 FAILURE: Build failed with an exception.
06:26:38
06:26:38 * What went wrong:
06:26:38 Execution failed for task
':sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch'.
06:26:38 > Process 'command 'sh'' finished with non-zero exit value 1
06:26:38
06:26:38 * Try:
06:26:38 Run with --stacktrace option to get the stack trace. Run with --info
or --debug option to get more log output. Run with --scan to get full insights.
06:26:38
06:26:38 * Get more help at https://help.gradle.org
06:26:38
06:26:38 Deprecated Gradle features were used in this build, making it
incompatible with Gradle 7.0.
06:26:38 Use '--warning-mode all' to show the individual deprecation warnings.
06:26:38 See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
06:26:38
06:26:38 BUILD FAILED in 1h 26m 11s
{code}
{code}
05:25:34 > Task
:sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch FAILED
05:25:34 INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers
at localhost:42005
05:25:34 WARNING:root:Make sure that locally built Python SDK docker image has
Python 3.8 interpreter.
05:25:34 INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.30.0.dev
05:25:34
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function lift_combiners at 0x7f71dcdf5a60> ====================
05:25:34
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function sort_stages at 0x7f71dcdfa1f0> ====================
05:25:34 INFO:apache_beam.utils.subprocess_server:Starting service with ['java'
'-jar'
'/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar'
'--spark-master-url' 'local[4]' '--artifacts-dir'
'/tmp/beam-tempgetxx5hw/artifactsw4djif3r' '--job-port' '42069'
'--artifact-port' '0' '--expansion-port' '0']
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:29 INFO
org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService
started on localhost:46287'
05:25:34 WARNING:root:Waiting for grpc channel to be ready at localhost:42069.
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO
org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService
started on localhost:41861'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO
org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on
localhost:42069'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO
org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running,
terminate with Ctrl+C'
05:25:34 WARNING:apache_beam.options.pipeline_options:Discarding unparseable
args: ['--parallelism=2']
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging
artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving
artifacts for
job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.ref_Environment_default_environment_1.'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1
artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.null.'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts
fully staged for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:32 INFO
org.apache.beam.runners.spark.SparkJobInvoker: Invoking job
BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:32 INFO
org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation
BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23'
05:25:34 INFO:apache_beam.runners.portability.portable_runner:Environment
"LOOPBACK" has started a component necessary for the execution. Be sure to run
the pipeline using
05:25:34 with Pipeline() as p:
05:25:34 p.apply(..)
05:25:34 This ensures that the pipeline finishes before this program exits.
05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed
to STOPPED
05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed
to STARTING
05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed
to RUNNING
05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:33 ERROR
org.apache.beam.runners.jobsubmission.JobInvocation: Error during job
invocation
BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23.'
05:25:34
INFO:apache_beam.utils.subprocess_server:b'java.lang.NullPointerException'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)'
05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
java.lang.Thread.run(Thread.java:748)'
05:25:34 ERROR:root:java.lang.NullPointerException
05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state changed
to FAILED
05:25:34 Traceback (most recent call last):
05:25:34 File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
05:25:34 return _run_code(code, main_globals, None,
05:25:34 File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
05:25:34 exec(code, run_globals)
05:25:34 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/examples/wordcount.py",
line 94, in <module>
05:25:34 run()
05:25:34 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/examples/wordcount.py",
line 89, in run
05:25:34 output | 'Write' >> WriteToText(known_args.output)
05:25:34 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/pipeline.py",
line 580, in __exit__
05:25:34 self.result.wait_until_finish()
05:25:34 File
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/runners/portability/portable_runner.py",
line 602, in wait_until_finish
05:25:34 raise self._runtime_exception
05:25:34 RuntimeError: Pipeline
BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23 failed
in state FAILED: java.lang.NullPointerException
{code}
was:
The first run in the Python 3.8 suite with this particular failure appears to
be https://ci-beam.apache.org/job/beam_PostCommit_Python38/1104
{code}
06:26:36 OK (SKIP=6)
06:26:36
06:26:38 FAILURE: Build failed with an exception.
06:26:38
06:26:38 * What went wrong:
06:26:38 Execution failed for task
':sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch'.
06:26:38 > Process 'command 'sh'' finished with non-zero exit value 1
06:26:38
06:26:38 * Try:
06:26:38 Run with --stacktrace option to get the stack trace. Run with --info
or --debug option to get more log output. Run with --scan to get full insights.
06:26:38
06:26:38 * Get more help at https://help.gradle.org
06:26:38
06:26:38 Deprecated Gradle features were used in this build, making it
incompatible with Gradle 7.0.
06:26:38 Use '--warning-mode all' to show the individual deprecation warnings.
06:26:38 See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
06:26:38
06:26:38 BUILD FAILED in 1h 26m 11s
{code}
> Python PostCommits failing portableWordCountSparkRunnerBatch
> ------------------------------------------------------------
>
> Key: BEAM-12207
> URL: https://issues.apache.org/jira/browse/BEAM-12207
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
> Reporter: Brian Hulette
> Priority: P1
> Labels: currently-failing
>
> The first run in the Python 3.8 suite with this particular failure appears to
> be https://ci-beam.apache.org/job/beam_PostCommit_Python38/1104
> {code}
> 06:26:36 OK (SKIP=6)
> 06:26:36
> 06:26:38 FAILURE: Build failed with an exception.
> 06:26:38
> 06:26:38 * What went wrong:
> 06:26:38 Execution failed for task
> ':sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch'.
> 06:26:38 > Process 'command 'sh'' finished with non-zero exit value 1
> 06:26:38
> 06:26:38 * Try:
> 06:26:38 Run with --stacktrace option to get the stack trace. Run with --info
> or --debug option to get more log output. Run with --scan to get full
> insights.
> 06:26:38
> 06:26:38 * Get more help at https://help.gradle.org
> 06:26:38
> 06:26:38 Deprecated Gradle features were used in this build, making it
> incompatible with Gradle 7.0.
> 06:26:38 Use '--warning-mode all' to show the individual deprecation warnings.
> 06:26:38 See
> https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
> 06:26:38
> 06:26:38 BUILD FAILED in 1h 26m 11s
> {code}
> {code}
> 05:25:34 > Task
> :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch
> FAILED
> 05:25:34 INFO:apache_beam.runners.worker.worker_pool_main:Listening for
> workers at localhost:42005
> 05:25:34 WARNING:root:Make sure that locally built Python SDK docker image
> has Python 3.8 interpreter.
> 05:25:34 INFO:root:Default Python SDK image for environment is
> apache/beam_python3.8_sdk:2.30.0.dev
> 05:25:34
> INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
> <function lift_combiners at 0x7f71dcdf5a60> ====================
> 05:25:34
> INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
> <function sort_stages at 0x7f71dcdfa1f0> ====================
> 05:25:34 INFO:apache_beam.utils.subprocess_server:Starting service with
> ['java' '-jar'
> '/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.30.0-SNAPSHOT.jar'
> '--spark-master-url' 'local[4]' '--artifacts-dir'
> '/tmp/beam-tempgetxx5hw/artifactsw4djif3r' '--job-port' '42069'
> '--artifact-port' '0' '--expansion-port' '0']
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:29 INFO
> org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService
> started on localhost:46287'
> 05:25:34 WARNING:root:Waiting for grpc channel to be ready at localhost:42069.
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO
> org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService
> started on localhost:41861'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO
> org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on
> localhost:42069'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:30 INFO
> org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now
> running, terminate with Ctrl+C'
> 05:25:34 WARNING:apache_beam.options.pipeline_options:Discarding unparseable
> args: ['--parallelism=2']
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
> org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging
> artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
> org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService:
> Resolving artifacts for
> job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.ref_Environment_default_environment_1.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
> org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting
> 1 artifacts for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.null.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:31 INFO
> org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService:
> Artifacts fully staged for job_7c9a3a39-f855-455c-8f36-bcd6bfd8152e.'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:32 INFO
> org.apache.beam.runners.spark.SparkJobInvoker: Invoking job
> BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:32 INFO
> org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation
> BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23'
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Environment
> "LOOPBACK" has started a component necessary for the execution. Be sure to
> run the pipeline using
> 05:25:34 with Pipeline() as p:
> 05:25:34 p.apply(..)
> 05:25:34 This ensures that the pipeline finishes before this program exits.
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state
> changed to STOPPED
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state
> changed to STARTING
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state
> changed to RUNNING
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'21/04/16 12:25:33 ERROR
> org.apache.beam.runners.jobsubmission.JobInvocation: Error during job
> invocation
> BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23.'
> 05:25:34
> INFO:apache_beam.utils.subprocess_server:b'java.lang.NullPointerException'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:120)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)'
> 05:25:34 INFO:apache_beam.utils.subprocess_server:b'\tat
> java.lang.Thread.run(Thread.java:748)'
> 05:25:34 ERROR:root:java.lang.NullPointerException
> 05:25:34 INFO:apache_beam.runners.portability.portable_runner:Job state
> changed to FAILED
> 05:25:34 Traceback (most recent call last):
> 05:25:34 File "/usr/lib/python3.8/runpy.py", line 194, in
> _run_module_as_main
> 05:25:34 return _run_code(code, main_globals, None,
> 05:25:34 File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
> 05:25:34 exec(code, run_globals)
> 05:25:34 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/examples/wordcount.py",
> line 94, in <module>
> 05:25:34 run()
> 05:25:34 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/examples/wordcount.py",
> line 89, in run
> 05:25:34 output | 'Write' >> WriteToText(known_args.output)
> 05:25:34 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/pipeline.py",
> line 580, in __exit__
> 05:25:34 self.result.wait_until_finish()
> 05:25:34 File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python38/src/sdks/python/apache_beam/runners/portability/portable_runner.py",
> line 602, in wait_until_finish
> 05:25:34 raise self._runtime_exception
> 05:25:34 RuntimeError: Pipeline
> BeamApp-jenkins-0416122532-da1b901c_d29eb552-6c06-4531-b0bd-fd234973ff23
> failed in state FAILED: java.lang.NullPointerException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
