See
<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/6901/display/redirect?page=changes>
Changes:
[noreply] Bump github.com/proullon/ramsql from 0.0.1 to 0.1.1 in /sdks (#28089)
------------------------------------------
[...truncated 870.16 KB...]
# through properly, global_var will be undefined and the pipeline will fail.
global_var = 1
pcoll = (pipeline
| Create([0, 1, 2])
| Map(lambda x: x + global_var))
else:
pcoll = (pipeline
| Create([0, 1, 2])
| Map(lambda x: x + 1))
assert_that(pcoll, equal_to([1, 2, 3]))
result = pipeline.run()
result.wait_until_finish()
"
if [[ "$SAVE_MAIN_SESSION" -eq 0 ]]; then
(python -c "$PIPELINE_PY" \
--runner FlinkRunner \
--flink_job_server_jar "$FLINK_JOB_SERVER_JAR" \
--parallelism 1 \
--environment_type DOCKER \
--environment_options "docker_container_image=$PYTHON_CONTAINER_IMAGE" \
--flink_master "localhost:$FLINK_PORT" \
--flink_submit_uber_jar \
) || TEST_EXIT_CODE=$? # don't fail fast here; clean up before exiting
else
(python -c "$PIPELINE_PY" \
--runner FlinkRunner \
--flink_job_server_jar "$FLINK_JOB_SERVER_JAR" \
--parallelism 1 \
--environment_type DOCKER \
--environment_options "docker_container_image=$PYTHON_CONTAINER_IMAGE" \
--flink_master "localhost:$FLINK_PORT" \
--flink_submit_uber_jar \
--save_main_session
) || TEST_EXIT_CODE=$? # don't fail fast here; clean up before exiting
fi
Starting Flink mini cluster listening on port 35715
Aug 23, 2023 6:11:55 PM
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils
setConfigOptionToDefaultIfNotSet
INFO: The configuration option taskmanager.cpu.cores required for local
execution is not set, setting it to the maximal possible value.
Aug 23, 2023 6:11:55 PM
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils
setConfigOptionToDefaultIfNotSet
INFO: The configuration option taskmanager.memory.task.heap.size required for
local execution is not set, setting it to the maximal possible value.
Aug 23, 2023 6:11:55 PM
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils
setConfigOptionToDefaultIfNotSet
INFO: The configuration option taskmanager.memory.task.off-heap.size required
for local execution is not set, setting it to the maximal possible value.
Aug 23, 2023 6:11:55 PM
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils
setConfigOptionToDefaultIfNotSet
INFO: The configuration option taskmanager.memory.network.min required for
local execution is not set, setting it to its default value 64 mb.
Aug 23, 2023 6:11:55 PM
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils
setConfigOptionToDefaultIfNotSet
INFO: The configuration option taskmanager.memory.network.max required for
local execution is not set, setting it to its default value 64 mb.
Aug 23, 2023 6:11:55 PM
org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils
setConfigOptionToDefaultIfNotSet
INFO: The configuration option taskmanager.memory.managed.size required for
local execution is not set, setting it to its default value 128 mb.
Aug 23, 2023 6:11:55 PM org.apache.flink.runtime.minicluster.MiniCluster start
INFO: Starting Flink Mini Cluster
INFO:root:Using provided Python SDK container image:
apache/beam_python3.8_sdk:2.51.0.dev
INFO:root:Python SDK container image set to
"apache/beam_python3.8_sdk:2.51.0.dev" for Docker environment
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function pack_combiners at 0x7fbdf46570d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function lift_combiners at 0x7fbdf4657160> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function sort_stages at 0x7fbdf46578b0> ====================
INFO:apache_beam.runners.portability.flink_runner:Adding HTTP protocol scheme
to flink_master parameter: http://localhost:35715
Aug 23, 2023 6:11:56 PM org.apache.flink.runtime.minicluster.MiniCluster start
INFO: Starting Metrics Registry
Aug 23, 2023 6:11:56 PM org.apache.flink.runtime.metrics.MetricRegistryImpl
<init>
INFO: No metrics reporter configured, no metrics will be exposed/reported.
Aug 23, 2023 6:11:56 PM org.apache.flink.runtime.minicluster.MiniCluster start
INFO: Starting RPC Service(s)
Aug 23, 2023 6:11:56 PM org.apache.flink.runtime.rpc.akka.AkkaBootstrapTools
startLocalActorSystem
INFO: Trying to start local actor system
Aug 23, 2023 6:11:58 PM akka.event.slf4j.Slf4jLogger$$anonfun$receive$1
applyOrElse
INFO: Slf4jLogger started
Aug 23, 2023 6:11:58 PM org.apache.flink.runtime.rpc.akka.AkkaBootstrapTools
startActorSystem
INFO: Actor system started at akka://flink
Aug 23, 2023 6:11:58 PM org.apache.flink.runtime.rpc.akka.AkkaBootstrapTools
startLocalActorSystem
INFO: Trying to start local actor system
Aug 23, 2023 6:11:58 PM akka.event.slf4j.Slf4jLogger$$anonfun$receive$1
applyOrElse
INFO: Slf4jLogger started
Aug 23, 2023 6:11:58 PM org.apache.flink.runtime.rpc.akka.AkkaBootstrapTools
startActorSystem
INFO: Actor system started at akka://flink-metrics
Aug 23, 2023 6:11:58 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService
startServer
INFO: Starting RPC endpoint for
org.apache.flink.runtime.metrics.dump.MetricQueryService at
akka://flink-metrics/user/rpc/MetricQueryService .
INFO:apache_beam.runners.portability.abstract_job_service:Artifact server
started on port 44751
INFO:apache_beam.runners.portability.abstract_job_service:Running job
'job-d2c84c5a-a4fe-4e48-ac18-e2dbe21749a9'
Aug 23, 2023 6:11:59 PM org.apache.flink.runtime.blob.BlobServer <init>
INFO: Created BLOB server storage directory
/tmp/minicluster_9e5724cbdd3d1579a7e3f969e3630b9d/blobStorage
Aug 23, 2023 6:11:59 PM org.apache.flink.runtime.blob.BlobServer <init>
INFO: Started BLOB server at 0.0.0.0:42069 - max concurrent requests: 50 - max
backlog: 1000
Aug 23, 2023 6:11:59 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Aug 23, 2023 6:11:59 PM
org.apache.flink.runtime.security.token.KerberosDelegationTokenManagerFactory
create
INFO: Cannot use kerberos delegation token manager no valid kerberos
credentials provided.
Aug 23, 2023 6:11:59 PM org.apache.flink.runtime.blob.AbstractBlobCache <init>
INFO: Created BLOB cache storage directory
/tmp/minicluster_9e5724cbdd3d1579a7e3f969e3630b9d/blobStorage
Aug 23, 2023 6:11:59 PM org.apache.flink.runtime.blob.AbstractBlobCache <init>
INFO: Created BLOB cache storage directory
/tmp/minicluster_9e5724cbdd3d1579a7e3f969e3630b9d/blobStorage
Aug 23, 2023 6:11:59 PM org.apache.flink.runtime.minicluster.MiniCluster
startTaskManagers
INFO: Starting 1 TaskManager(s)
Aug 23, 2023 6:11:59 PM org.apache.flink.runtime.taskexecutor.TaskManagerRunner
startTaskManager
INFO: Starting TaskManager with ResourceID: db1b4d55-9b12-4b7d-b17c-057a960d47f6
Aug 23, 2023 6:12:00 PM
org.apache.flink.runtime.taskexecutor.TaskManagerServices checkTempDirs
INFO: Temporary file directory '/tmp': total 969 GB, usable 713 GB (73.58%
usable)
Aug 23, 2023 6:12:00 PM org.apache.flink.runtime.io.disk.iomanager.IOManager
<init>
INFO: Created a new FileChannelManager for spilling of task related data to
disk (joins, sorting, ...). Used directories:
/tmp/flink-io-96d077c9-908c-43a6-b8cc-d136ffe81bb5
Aug 23, 2023 6:12:00 PM
org.apache.flink.runtime.io.network.NettyShuffleServiceFactory
createNettyShuffleEnvironment
INFO: Created a new FileChannelManager for storing result partitions of
BLOCKING shuffles. Used directories:
/tmp/flink-netty-shuffle-e03c804a-6824-45f2-ad14-7e0a2b0d6299
Aug 23, 2023 6:12:00 PM
org.apache.flink.runtime.io.network.buffer.NetworkBufferPool <init>
INFO: Allocated 64 MB for network buffer pool (number of memory segments: 2048,
bytes per segment: 32768).
Aug 23, 2023 6:12:00 PM
org.apache.flink.runtime.io.network.NettyShuffleEnvironment start
INFO: Starting the network environment and its components.
Aug 23, 2023 6:12:00 PM org.apache.flink.runtime.taskexecutor.KvStateService
start
INFO: Starting the kvState service and its components.
Aug 23, 2023 6:12:00 PM org.apache.flink.configuration.Configuration
loggingFallback
INFO: Config uses fallback configuration key 'akka.ask.timeout' instead of key
'taskmanager.slot.timeout'
Aug 23, 2023 6:12:00 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService
startServer
INFO: Starting RPC endpoint for
org.apache.flink.runtime.taskexecutor.TaskExecutor at
akka://flink/user/rpc/taskmanager_0 .
Aug 23, 2023 6:12:00 PM
org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService start
INFO: Start job leader service.
Aug 23, 2023 6:12:00 PM org.apache.flink.runtime.filecache.FileCache <init>
INFO: User file cache uses directory
/tmp/flink-dist-cache-b0352a8e-ff43-409c-b94b-c0051d445e3d
Aug 23, 2023 6:12:00 PM org.apache.flink.configuration.Configuration
loggingFallback
INFO: Config uses fallback configuration key 'rest.port' instead of key
'rest.bind-port'
Aug 23, 2023 6:12:00 PM org.apache.flink.runtime.rest.RestServerEndpoint start
INFO: Starting rest endpoint.
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connection.py";,>
line 203, in _new_conn
sock = connection.create_connection(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/util/connection.py";,>
line 85, in create_connection
raise err
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/util/connection.py";,>
line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connectionpool.py";,>
line 790, in urlopen
response = self._make_request(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connectionpool.py";,>
line 496, in _make_request
conn.request(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connection.py";,>
line 395, in request
self.endheaders()
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connection.py";,>
line 243, in connect
self.sock = self._new_conn()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connection.py";,>
line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection
object at 0x7fbdeef87610>: Failed to establish a new connection: [Errno 111]
Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/requests/adapters.py";,>
line 486, in send
resp = conn.urlopen(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/connectionpool.py";,>
line 844, in urlopen
retries = retries.increment(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/urllib3/util/retry.py";,>
line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type:
ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost',
port=35715): Max retries exceeded with url: /v1/jars/upload (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7fbdeef87610>: Failed to establish a new connection: [Errno 111] Connection
refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 31, in <module>
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 577, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/runner.py";,>
line 190, in run_pipeline
return self.run_portable_pipeline(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,>
line 409, in run_portable_pipeline
job_id, message_stream, state_stream = job_service_handle.submit(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,>
line 114, in submit
return self.run(prepare_response.preparation_id)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,>
line 241, in run
run_response = self.job_service.Run(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/abstract_job_service.py";,>
line 120, in Run
self._jobs[job_id].run()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/flink_uber_jar_job_server.py";,>
line 152, in run
self._flink_jar_id = self.post(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/flink_uber_jar_job_server.py";,>
line 142, in post
return self.request(requests.post, path, **kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/apache_beam/runners/portability/flink_uber_jar_job_server.py";,>
line 130, in request
response = method(url, **kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/requests/api.py";,>
line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/requests/api.py";,>
line 59, in request
return session.request(method=method, url=url, **kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/requests/sessions.py";,>
line 589, in request
resp = self.send(prep, **send_kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/requests/sessions.py";,>
line 703, in send
r = adapter.send(request, **kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages/requests/adapters.py";,>
line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost',
port=35715): Max retries exceeded with url: /v1/jars/upload (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7fbdeef87610>: Failed to establish a new connection: [Errno 111] Connection
refused'))
Aug 23, 2023 6:12:01 PM
org.apache.flink.runtime.webmonitor.WebMonitorUtils$LogFileLocation find
WARNING: Log file environment variable 'log.file' is not set.
Aug 23, 2023 6:12:01 PM
org.apache.flink.runtime.webmonitor.WebMonitorUtils$LogFileLocation find
WARNING: JobManager log files are unavailable in the web dashboard. Log file
location not found in environment variable 'log.file' or configuration key
'web.log.path'.
kill %1 || echo "Failed to shut down Flink mini cluster"
rm -rf "$ENV_DIR"
Aug 23, 2023 6:12:01 PM org.apache.flink.runtime.blob.AbstractBlobCache close
INFO: Shutting down BLOB cache
Aug 23, 2023 6:12:01 PM
org.apache.flink.runtime.state.TaskExecutorStateChangelogStoragesManager
shutdown
INFO: Shutting down TaskExecutorStateChangelogStoragesManager.
>>> FAILURE
if [[ "$TEST_EXIT_CODE" -eq 0 ]]; then
echo ">>> SUCCESS"
else
echo ">>> FAILURE"
fi
exit $TEST_EXIT_CODE
> Task
> :sdks:python:test-suites:portable:py38:testUberJarFlinkRunnerSaveMainSession
> FAILED
FAILURE: Build failed with an exception.
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_PortableJar_Flink/ws/src/sdks/python/test-suites/portable/common.gradle'>
line: 411
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py38:testUberJarFlinkRunnerSaveMainSession'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 35s
149 actionable tasks: 99 executed, 46 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/ugrnocqnhb2q2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
