See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/1012/display/redirect?page=changes>
Changes: [noreply] Merge pull request #23510: Vortex multiplexing streams [noreply] Io jms fix ack message checkpoint (#22932) ------------------------------------------ [...truncated 16.87 MB...] INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING: Unable to retrieve the requested metadata. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM org.apache.beam.runners.jobsubmission.JobServerDriver createArtifactStagingService INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: ArtifactStagingService started on localhost:42523 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM org.apache.beam.runners.jobsubmission.JobServerDriver createExpansionService INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Java ExpansionService started on localhost:44899 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM org.apache.beam.runners.jobsubmission.JobServerDriver createJobServer INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: JobService started on localhost:42085 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM org.apache.beam.runners.jobsubmission.JobServerDriver run INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Job server now running, terminate with Ctrl+C INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 onNext INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Staging artifacts for job_bb18ffc5-6e57-4904-8d00-ef7754a170c8. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:40 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 resolveNextEnvironment INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Resolving artifacts for job_bb18ffc5-6e57-4904-8d00-ef7754a170c8.ref_Environment_default_environment_1. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:41 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 onNext INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Getting 0 artifacts for job_bb18ffc5-6e57-4904-8d00-ef7754a170c8.null. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:41 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 finishStaging INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Artifacts fully staged for job_bb18ffc5-6e57-4904-8d00-ef7754a170c8. > Task :sdks:python:test-suites:portable:py310:flinkExamples FAILED [ 56%] apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it ERROR [ 56%] apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it -------------------------------- live log call --------------------------------- INFO apache_beam.runners.worker.worker_pool_main:worker_pool_main.py:93 Listening for workers at localhost:38895 INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.10_sdk:2.43.0.dev INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function pack_combiners at 0x7f40b37ed000> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function lift_combiners at 0x7f40b37ed090> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sort_stages at 0x7f40b37ed7e0> ==================== INFO apache_beam.utils.subprocess_server:subprocess_server.py:116 Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.43.0-SNAPSHOT.jar'> '--flink-master' '[auto]' '--artifacts-dir' '/tmp/beam-tempv__wpnof/artifacts65emoncr' '--job-port' '41549' '--artifact-port' '0' '--expansion-port' '0'] INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:43 AM software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING: Unable to retrieve the requested metadata. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.jobsubmission.JobServerDriver createArtifactStagingService INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: ArtifactStagingService started on localhost:46283 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.jobsubmission.JobServerDriver createExpansionService INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Java ExpansionService started on localhost:34439 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.jobsubmission.JobServerDriver createJobServer INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: JobService started on localhost:41549 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.jobsubmission.JobServerDriver run INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Job server now running, terminate with Ctrl+C INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 onNext INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Staging artifacts for job_7ee1bd24-ae0b-41fa-b885-e040aa305991. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 resolveNextEnvironment INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Resolving artifacts for job_7ee1bd24-ae0b-41fa-b885-e040aa305991.ref_Environment_default_environment_1. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 onNext INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Getting 0 artifacts for job_7ee1bd24-ae0b-41fa-b885-e040aa305991.null. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:44 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 finishStaging INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Artifacts fully staged for job_7ee1bd24-ae0b-41fa-b885-e040aa305991. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:45 AM org.apache.beam.runners.flink.FlinkJobInvoker invokeWithExecutor INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Invoking job BeamApp-jenkins-1019094344-8d367dc_a5df4eb9-ec32-4155-aa59-c1890e2b00d5 with pipeline runner org.apache.beam.runners.flink.FlinkPipelineRunner@1974b45a INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:45 AM org.apache.beam.runners.jobsubmission.JobInvocation start INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Starting job invocation BeamApp-jenkins-1019094344-8d367dc_a5df4eb9-ec32-4155-aa59-c1890e2b00d5 INFO apache_beam.runners.portability.portable_runner:portable_runner.py:452 Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using with Pipeline() as p: p.apply(..) This ensures that the pipeline finishes before this program exits. INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581 Job state changed to STOPPED INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581 Job state changed to STARTING INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581 Job state changed to RUNNING INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:45 AM org.apache.beam.runners.flink.FlinkPipelineRunner runPipelineWithTranslator INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Translating pipeline to Flink program. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:45 AM org.apache.beam.runners.jobsubmission.JobInvocation$1 onFailure INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 SEVERE: Error during job invocation BeamApp-jenkins-1019094344-8d367dc_a5df4eb9-ec32-4155-aa59-c1890e2b00d5. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 java.lang.IllegalArgumentException: PCollectionNodes [PCollectionNode{id=ref_PCollection_PCollection_52, PCollection=unique_name: "61Write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables.None" INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 coder_id: "ref_Coder_FastPrimitivesCoder_3" INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 is_bounded: BOUNDED INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 windowing_strategy_id: "ref_Windowing_Windowing_1" INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 }] were consumed but never produced INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:440) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.core.construction.graph.QueryablePipeline.buildNetwork(QueryablePipeline.java:234) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.core.construction.graph.QueryablePipeline.<init>(QueryablePipeline.java:127) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.core.construction.graph.QueryablePipeline.forTransforms(QueryablePipeline.java:107) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.core.construction.graph.FusedPipeline.toPipeline(FusedPipeline.java:81) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:112) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:85) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.runners.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:86) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 at java.lang.Thread.run(Thread.java:750) INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 ERROR root:portable_runner.py:574 java.lang.IllegalArgumentException: PCollectionNodes [PCollectionNode{id=ref_PCollection_PCollection_52, PCollection=unique_name: "61Write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables.None" coder_id: "ref_Coder_FastPrimitivesCoder_3" is_bounded: BOUNDED windowing_strategy_id: "ref_Windowing_Windowing_1" }] were consumed but never produced INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581 Job state changed to FAILED INFO apache_beam.io.gcp.tests.utils:utils.py:93 Clean up a BigQuery table with project: apache-beam-testing, dataset: BigQueryTornadoesIT, table: monthly_tornadoes_1666172621178. FAILED [ 60%] apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it ERROR [ 60%] apache_beam/examples/cookbook/coders_it_test.py::CodersIT::test_coders_output_files_on_small_input -------------------------------- live log call --------------------------------- INFO root:coders_it_test.py:48 Creating file: gs://temp-storage-for-end-to-end-tests/py-it-cloud/input/e6796d06-a958-40fc-b14d-824013105694/input.txt ERROR apache_beam.io.gcp.gcsio:gcsio.py:767 Error in _start_upload while inserting file gs://temp-storage-for-end-to-end-tests/py-it-cloud/input/e6796d06-a958-40fc-b14d-824013105694/input.txt: Traceback (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/ws/src/sdks/python/apache_beam/io/gcp/gcsio.py";,> line 763, in _start_upload self._client.objects.Insert(self._insert_request, upload=self._upload) File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/ws/src/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py";,> line 1152, in Insert return self._RunMethod( File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/ws/src/build/gradleenv/-1720702907/lib/python3.10/site-packages/apitools/base/py/base_api.py";,> line 714, in _RunMethod http_response = upload.InitializeUpload( File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/ws/src/build/gradleenv/-1720702907/lib/python3.10/site-packages/apitools/base/py/transfer.py";,> line 896, in InitializeUpload raise exceptions.HttpError.FromResponse(http_response) apitools.base.py.exceptions.HttpUnauthorizedError: HttpError accessing <https://www.googleapis.com/resumable/upload/storage/v1/b/temp-storage-for-end-to-end-tests/o?alt=json&name=py-it-cloud%2Finput%2Fe6796d06-a958-40fc-b14d-824013105694%2Finput.txt&uploadType=resumable>: response: <{'x-guploader-uploadid': 'ADPycdstTfOvuoyhQQ2xm5xdzRaVrm9vnZ9CU_ehApnUHrzIQ5o15XhHdiFeChnBl_7PTMxrmJAWCao_4kEJ4bUxVyP8WJ_Mfyn3', 'date': 'Wed, 19 Oct 2022 09:43:46 GMT', 'vary': 'Origin, X-Origin', 'www-authenticate': 'Bearer realm="https://accounts.google.com/";', 'cache-control': 'no-cache, no-store, max-age=0, must-revalidate', 'expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'pragma': 'no-cache', 'content-length': '438', 'server': 'UploadServer', 'content-type': 'text/html; charset=UTF-8', 'status': '401'}>, content <{ "error": { "code": 401, "message": "Anonymous caller does not have storage.objects.create access to the Google Cloud Storage object.", "errors": [ { "message": "Anonymous caller does not have storage.objects.create access to the Google Cloud Storage object.", "domain": "global", "reason": "required", "locationType": "header", "location": "Authorization" } ] } } > FAILED [ 65%] apache_beam/examples/cookbook/custom_ptransform_test.py::CustomCountTest::test_custom_ptransform_output_files_on_small_input -------------------------------- live log call --------------------------------- INFO root:custom_ptransform_test.py:41 Creating temp file: /tmp/tmp23yjz5f8/input.txt INFO root:custom_ptransform.py:48 Running first pipeline INFO root:pipeline.py:185 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.10_sdk:2.43.0.dev INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function annotate_downstream_side_inputs at 0x7f40b37ec9d0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function fix_side_input_pcoll_coders at 0x7f40b37ecaf0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function pack_combiners at 0x7f40b37ed000> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function lift_combiners at 0x7f40b37ed090> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function expand_sdf at 0x7f40b37ed240> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function expand_gbk at 0x7f40b37ed2d0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sink_flattens at 0x7f40b37ed3f0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function greedily_fuse at 0x7f40b37ed480> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function read_to_impulse at 0x7f40b37ed510> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function impulse_to_input at 0x7f40b37ed5a0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sort_stages at 0x7f40b37ed7e0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function add_impulse_to_dangling_transforms at 0x7f40b37ed900> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function setup_timer_mapping at 0x7f40b37ed750> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function populate_data_channel_coders at 0x7f40b37ed870> ==================== INFO apache_beam.runners.worker.statecache:statecache.py:234 Creating state cache with size 104857600 INFO apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:903 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f407be4ffa0> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'') INFO apache_beam.io.filebasedsink:filebasedsink.py:310 Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1 INFO apache_beam.io.filebasedsink:filebasedsink.py:358 Renamed 1 shards in 0.00 seconds. INFO root:custom_ptransform.py:67 Running second pipeline INFO root:pipeline.py:185 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.10_sdk:2.43.0.dev INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function annotate_downstream_side_inputs at 0x7f40b37ec9d0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function fix_side_input_pcoll_coders at 0x7f40b37ecaf0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function pack_combiners at 0x7f40b37ed000> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function lift_combiners at 0x7f40b37ed090> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function expand_sdf at 0x7f40b37ed240> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function expand_gbk at 0x7f40b37ed2d0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sink_flattens at 0x7f40b37ed3f0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function greedily_fuse at 0x7f40b37ed480> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function read_to_impulse at 0x7f40b37ed510> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function impulse_to_input at 0x7f40b37ed5a0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sort_stages at 0x7f40b37ed7e0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function add_impulse_to_dangling_transforms at 0x7f40b37ed900> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function setup_timer_mapping at 0x7f40b37ed750> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function populate_data_channel_coders at 0x7f40b37ed870> ==================== INFO apache_beam.runners.worker.statecache:statecache.py:234 Creating state cache with size 104857600 INFO apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:903 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f4068363f10> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'') WARNING apache_beam.io.filebasedsink:filebasedsink.py:230 Deleting 1 existing files in target path matching: -*-of-%(num_shards)05d INFO apache_beam.io.filebasedsink:filebasedsink.py:310 Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1 INFO apache_beam.io.filebasedsink:filebasedsink.py:358 Renamed 1 shards in 0.00 seconds. INFO root:custom_ptransform.py:94 Running third pipeline INFO root:pipeline.py:185 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.10_sdk:2.43.0.dev INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function annotate_downstream_side_inputs at 0x7f40b37ec9d0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function fix_side_input_pcoll_coders at 0x7f40b37ecaf0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function pack_combiners at 0x7f40b37ed000> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function lift_combiners at 0x7f40b37ed090> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function expand_sdf at 0x7f40b37ed240> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function expand_gbk at 0x7f40b37ed2d0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sink_flattens at 0x7f40b37ed3f0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function greedily_fuse at 0x7f40b37ed480> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function read_to_impulse at 0x7f40b37ed510> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function impulse_to_input at 0x7f40b37ed5a0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sort_stages at 0x7f40b37ed7e0> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function add_impulse_to_dangling_transforms at 0x7f40b37ed900> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function setup_timer_mapping at 0x7f40b37ed750> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function populate_data_channel_coders at 0x7f40b37ed870> ==================== INFO apache_beam.runners.worker.statecache:statecache.py:234 Creating state cache with size 104857600 INFO apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:903 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f4080f502b0> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'') WARNING apache_beam.io.filebasedsink:filebasedsink.py:230 Deleting 1 existing files in target path matching: -*-of-%(num_shards)05d INFO apache_beam.io.filebasedsink:filebasedsink.py:310 Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1 INFO apache_beam.io.filebasedsink:filebasedsink.py:358 Renamed 1 shards in 0.00 seconds. PASSED [ 69%] apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher -------------------------------- live log call --------------------------------- INFO apache_beam.runners.worker.worker_pool_main:worker_pool_main.py:93 Listening for workers at localhost:36681 INFO root:environments.py:376 Default Python SDK image for environment is apache/beam_python3.10_sdk:2.43.0.dev INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function pack_combiners at 0x7f40b37ed000> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function lift_combiners at 0x7f40b37ed090> ==================== INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:714 ==================== <function sort_stages at 0x7f40b37ed7e0> ==================== INFO apache_beam.utils.subprocess_server:subprocess_server.py:116 Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.43.0-SNAPSHOT.jar'> '--flink-master' '[auto]' '--artifacts-dir' '/tmp/beam-tempeuq5svax/artifactssojse9rc' '--job-port' '54121' '--artifact-port' '0' '--expansion-port' '0'] INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:50 AM software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING: Unable to retrieve the requested metadata. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:51 AM org.apache.beam.runners.jobsubmission.JobServerDriver createArtifactStagingService INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: ArtifactStagingService started on localhost:38059 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:51 AM org.apache.beam.runners.jobsubmission.JobServerDriver createExpansionService INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Java ExpansionService started on localhost:34501 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:51 AM org.apache.beam.runners.jobsubmission.JobServerDriver createJobServer INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: JobService started on localhost:54121 INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:51 AM org.apache.beam.runners.jobsubmission.JobServerDriver run INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Job server now running, terminate with Ctrl+C WARNING root:subprocess_server.py:95 Waiting for grpc channel to be ready at localhost:54121. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:53 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 onNext INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Staging artifacts for job_4d31bc41-54aa-40d0-979b-60b2c0427470. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:53 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 resolveNextEnvironment INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Resolving artifacts for job_4d31bc41-54aa-40d0-979b-60b2c0427470.ref_Environment_default_environment_1. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:53 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 onNext INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Getting 0 artifacts for job_4d31bc41-54aa-40d0-979b-60b2c0427470.null. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:53 AM org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2 finishStaging INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Artifacts fully staged for job_4d31bc41-54aa-40d0-979b-60b2c0427470. > Task :sdks:python:test-suites:portable:py37:flinkExamples INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:41 AM org.apache.beam.runners.flink.FlinkJobInvoker invokeWithExecutor INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Invoking job BeamApp-jenkins-1019094341-27599187_4cb52bdd-a4e1-4e3c-8954-d5e9cc922bb8 with pipeline runner org.apache.beam.runners.flink.FlinkPipelineRunner@1d25a27f INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:41 AM org.apache.beam.runners.jobsubmission.JobInvocation start INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Starting job invocation BeamApp-jenkins-1019094341-27599187_4cb52bdd-a4e1-4e3c-8954-d5e9cc922bb8 INFO apache_beam.runners.portability.portable_runner:portable_runner.py:458 Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using with Pipeline() as p: p.apply(..) This ensures that the pipeline finishes before this program exits. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:41 AM org.apache.beam.runners.flink.FlinkPipelineRunner runPipelineWithTranslator INFO apache_beam.runners.portability.portable_runner:portable_runner.py:583 Job state changed to STOPPED INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Translating pipeline to Flink program. INFO apache_beam.runners.portability.portable_runner:portable_runner.py:583 Job state changed to STARTING INFO apache_beam.runners.portability.portable_runner:portable_runner.py:583 Job state changed to RUNNING INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM org.apache.beam.runners.flink.FlinkExecutionEnvironments createBatchExecutionEnvironment INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: Creating a Batch Execution Environment. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING: Unable to retrieve the requested metadata. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM org.apache.flink.api.java.utils.PlanGenerator logTypeRegistrationDetails INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: The job has 0 registered types and 0 default Kryo serializers INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils setConfigOptionToDefaultIfNotSet INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: The configuration option taskmanager.cpu.cores required for local execution is not set, setting it to the maximal possible value. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils setConfigOptionToDefaultIfNotSet INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: The configuration option taskmanager.memory.task.heap.size required for local execution is not set, setting it to the maximal possible value. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils setConfigOptionToDefaultIfNotSet INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: The configuration option taskmanager.memory.task.off-heap.size required for local execution is not set, setting it to the maximal possible value. INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Oct 19, 2022 9:43:42 AM org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils setConfigOptionToDefaultIfNotSet java.lang.OutOfMemoryError: GC overhead limit exceeded Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
