See
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/1973/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16364 from [BEAM-13182] Add diagrams to backend
[noreply] [BEAM-13811] Fix save_main_session arg in tests examples (#16709)
[Kiley Sok] Update beam-master version
[noreply] [BEAM-13015] Calculate exception for closing
BeamFnDataInboundObserver2
[noreply] Minor doc tweaks for validating vendoring. (#16747)
[noreply] [BEAM-13686] OOM while logging a large pipeline even when logging
level
[noreply] [BEAM-13629] Update URL artifact type for Dataflow Go (#16490)
[noreply] [BEAM-13832] Add automated expansion service start-up to JDBCio
(#16739)
[noreply] [BEAM-13831] Add automated expansion service infra into Debezium
Read()
[noreply] [BEAM-13821] Add automated expansion service start-up to KafkaIO
------------------------------------------
[...truncated 204.08 KB...]
"urn": "beam:metric:ptransform_execution_time:total_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "ref_AppliedPTransform_stateful_15"
}
}, {
"urn": "beam:metric:statecache:put_total",
"type": "beam:metrics:sum_int64:v1",
"payload": "Cg=="
}, {
"urn": "beam:metric:statecache:evict_total",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA=="
}, {
"urn": "beam:metric:data_channel:read_index:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "bg==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_9:0"
}
}, {
"urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "ref_AppliedPTransform_stateful_15"
}
}, {
"urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "ref_AppliedPTransform_stateful_15"
}
}, {
"urn": "beam:metric:ptransform_execution_time:total_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_9:0"
}
}, {
"urn": "beam:metric:user:sum_int64:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "bg==",
"labels": {
"NAMESPACE": "apache_beam.runners.portability.flink_runner_test.DoFn",
"PTRANSFORM": "ref_AppliedPTransform_stateful_15",
"NAME": "elem_counter"
}
}, {
"urn": "beam:metric:statecache:clear_total",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA=="
}, {
"urn": "beam:metric:sampled_byte_size:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "KbkFERE=",
"labels": {
"PCOLLECTION": "ref_PCollection_PCollection_9"
}
}, {
"urn": "beam:metric:sampled_byte_size:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "AAD//////////3+AgICAgICAgIAB",
"labels": {
"PCOLLECTION": "ref_PCollection_PCollection_10"
}
}, {
"urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM":
"ref_AppliedPTransform_create-FlatMap-lambda-at-core-py-3228-_4"
}
}, {
"urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM":
"ref_AppliedPTransform_create-FlatMap-lambda-at-core-py-3228-_4"
}
}, {
"urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_1:0"
}
}, {
"urn": "beam:metric:pardo_execution_time:start_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM":
"ref_AppliedPTransform_create-FlatMap-lambda-at-core-py-3228-_4"
}
}, {
"urn": "beam:metric:pardo_execution_time:start_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_1:0"
}
}, {
"urn": "beam:metric:pardo_execution_time:process_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/write/ref_PCollection_PCollection_2:0"
}
}, {
"urn": "beam:metric:data_channel:read_index:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AQ==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_1:0"
}
}, {
"urn": "beam:metric:pardo_execution_time:start_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/write/ref_PCollection_PCollection_2:0"
}
}, {
"urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/write/ref_PCollection_PCollection_2:0"
}
}, {
"urn": "beam:metric:ptransform_execution_time:total_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/write/ref_PCollection_PCollection_2:0"
}
}, {
"urn": "beam:metric:ptransform_execution_time:total_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_1:0"
}
}, {
"urn": "beam:metric:element_count:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "bg==",
"labels": {
"PCOLLECTION": "ref_PCollection_PCollection_2"
}
}, {
"urn": "beam:metric:ptransform_execution_time:total_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM":
"ref_AppliedPTransform_create-FlatMap-lambda-at-core-py-3228-_4"
}
}, {
"urn": "beam:metric:pardo_execution_time:finish_bundle_msecs:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AA==",
"labels": {
"PTRANSFORM": "fn/read/ref_PCollection_PCollection_1:0"
}
}, {
"urn": "beam:metric:element_count:v1",
"type": "beam:metrics:sum_int64:v1",
"payload": "AQ==",
"labels": {
"PCOLLECTION": "ref_PCollection_PCollection_1"
}
}, {
"urn": "beam:metric:sampled_byte_size:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "KswEDg4=",
"labels": {
"PCOLLECTION": "ref_PCollection_PCollection_2"
}
}, {
"urn": "beam:metric:sampled_byte_size:v1",
"type": "beam:metrics:distribution_int64:v1",
"payload": "AQ0NDQ==",
"labels": {
"PCOLLECTION": "ref_PCollection_PCollection_1"
}
}]
}
}
Feb 05, 2022 12:34:40 AM
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal
INFO: Free slot TaskSlot(index:0, state:ALLOCATED, resource profile:
ResourceProfile{taskHeapMemory=1024.000gb (1099511627776 bytes),
taskOffHeapMemory=1024.000gb (1099511627776 bytes), managedMemory=9.067gb
(9735473152 bytes), networkMemory=64.000mb (67108864 bytes)}, allocationId:
cad25bdb5b6d95559ebfdbe094d3961d, jobId: 9caa2dc816cb260295efc222417c5c8c).
Feb 05, 2022 12:34:40 AM
org.apache.flink.runtime.resourcemanager.ResourceManager
closeTaskManagerConnection
INFO: Closing TaskExecutor connection 970b278d-b56b-4922-9c74-ab2488ca8218
because: The TaskExecutor is shutting down.
Feb 05, 2022 12:34:40 AM org.apache.flink.runtime.jobmaster.JobMaster onStop
INFO: Stopping the JobMaster for job
test_flink_metrics_1644021273.3523726(9caa2dc816cb260295efc222417c5c8c).
Feb 05, 2022 12:34:40 AM
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool
releaseSlots
INFO: Releasing slot [cad25bdb5b6d95559ebfdbe094d3961d].
Feb 05, 2022 12:34:40 AM org.apache.flink.runtime.jobmaster.JobMaster
dissolveResourceManagerConnection
INFO: Close ResourceManager connection 10f2b4348a724d57bc4be9b3de05ca08:
Stopping JobMaster for job
test_flink_metrics_1644021273.3523726(9caa2dc816cb260295efc222417c5c8c)..
Feb 05, 2022 12:34:40 AM
org.apache.flink.runtime.resourcemanager.ResourceManager
closeJobManagerConnection
INFO: Disconnect job manager
a737747589a8399c77245c213e594a2a@akka://flink/user/rpc/jobmanager_47 for job
9caa2dc816cb260295efc222417c5c8c from the resource manager.
Feb 05, 2022 12:34:40 AM
org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop
INFO: Stop job leader service.
Feb 05, 2022 12:34:40 AM
org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager shutdown
INFO: Shutting down TaskExecutorLocalStateStoresManager.
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK
docker image has Python 3.6 interpreter.
WARNING apache_beam.options.pipeline_options:pipeline_options.py:324
Discarding invalid overrides: {'max_bundle_time_millis': 95130590130}
- generated xml file:
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
============== 1 failed, 130 passed, 40 skipped in 741.64 seconds ==============
[31mERROR: InvocationError for command
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test apache_beam/runners/portability/flink_runner_test.py
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/runners/flink/1.13/job-server/build/libs/beam-runners-flink-1.13-job-server-2.37.0-SNAPSHOT.jar>
--environment_type=LOOPBACK' (exited with code 1)
[0m[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[31mERROR: flink-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py36:flinkCompatibilityMatrixLOOPBACK
> FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py36:flinkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 16m 47s
147 actionable tasks: 101 executed, 42 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/oo4henxlfh2ee
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
