See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/2804/display/redirect>
Changes:
------------------------------------------
[...truncated 864.91 KB...]
To get a full path (which begins with top) to a file or directory in
dirpath, do os.path.join(dirpath, name).
If optional arg 'topdown' is true or not specified, the triple for a
directory is generated before the triples for any of its subdirectories
(directories are generated top down). If topdown is false, the triple
for a directory is generated after the triples for all of its
subdirectories (directories are generated bottom up).
When topdown is true, the caller can modify the dirnames list in-place
(e.g., via del or slice assignment), and walk will only recurse into the
subdirectories whose names remain in dirnames; this can be used to
prune the
search, or to impose a specific order of visiting. Modifying dirnames
when
topdown is false has no effect on the behavior of os.walk(), since the
directories in dirnames have already been generated by the time dirnames
itself is generated. No matter the value of topdown, the list of
subdirectories is retrieved before the tuples for the directory and its
subdirectories are generated.
By default errors from the os.scandir() call are ignored. If
optional arg 'onerror' is specified, it should be a function; it
will be called with one argument, an OSError instance. It can
report the error to continue with the walk, or raise the exception
to abort the walk. Note that the filename is available as the
filename attribute of the exception object.
By default, os.walk does not follow symbolic links to subdirectories on
systems that support them. In order to get this functionality, set the
optional argument 'followlinks' to true.
Caution: if you pass a relative pathname for top, don't change the
current working directory between resumptions of walk. walk never
changes the current directory, and assumes that the client doesn't
either.
Example:
import os
from os.path import join, getsize
for root, dirs, files in os.walk('python/Lib/email'):
print(root, "consumes", end="")
print(sum(getsize(join(root, name)) for name in files), end="")
print("bytes in", len(files), "non-directory files")
if 'CVS' in dirs:
dirs.remove('CVS') # don't visit CVS directories
"""
top = fspath(top)
dirs = []
nondirs = []
walk_dirs = []
# We may not have read permission for top, in which case we can't
# get a list of the files the directory contains. os.walk
# always suppressed the exception then, rather than blow up for a
# minor reason when (say) a thousand readable directories are still
# left to visit. That logic is copied here.
try:
# Note that scandir is global in this module due
# to earlier import-*.
scandir_it = scandir(top)
except OSError as error:
if onerror is not None:
onerror(error)
return
with scandir_it:
while True:
try:
try:
> entry = next(scandir_it)
E Failed: Timeout >600.0s [while running
'WriteToText/Write/WriteImpl/PreFinalize']
/usr/lib/python3.8/os.py:362: Failed
----------------------------- Captured stderr call -----------------------------
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++
~~~~~~~~~~~~~~~~~~~~ Stack of Thread-160 (140480008881920) ~~~~~~~~~~~~~~~~~~~~~
File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py";,>
line 224, in run
while not self._finished.wait(next_call - time.time()):
File "/usr/lib/python3.8/threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.8/threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++
------------------------------ Captured log call -------------------------------
INFO root:pipeline.py:196 Missing pipeline option (runner). Executing
pipeline using the default runner: DirectRunner.
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function annotate_downstream_side_inputs at
0x7fc3fac61940> ====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function fix_side_input_pcoll_coders at 0x7fc3fac61a60>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function pack_combiners at 0x7fc3fac61f70>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function lift_combiners at 0x7fc3fac62040>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function expand_sdf at 0x7fc3fac621f0>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function expand_gbk at 0x7fc3fac62280>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function sink_flattens at 0x7fc3fac623a0>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function greedily_fuse at 0x7fc3fac62430>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function read_to_impulse at 0x7fc3fac624c0>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function impulse_to_input at 0x7fc3fac62550>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function sort_stages at 0x7fc3fac62790>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function add_impulse_to_dangling_transforms at
0x7fc3fac628b0> ====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function setup_timer_mapping at 0x7fc3fac62700>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function populate_data_channel_coders at 0x7fc3fac62820>
====================
INFO apache_beam.runners.worker.statecache:statecache.py:234 Creating state
cache with size 104857600
INFO
apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:922
Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler
object at 0x7fc41c518a90> for environment
ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
ERROR apache_beam.runners.common:common.py:1546 Timeout >600.0s [while
running 'WriteToText/Write/WriteImpl/PreFinalize']
Traceback (most recent call last):
File "/usr/lib/python3.8/os.py", line 362, in walk
entry = next(scandir_it)
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/runners/common.py";,>
line 1435, in process
return self.do_fn_invoker.invoke_process(windowed_value)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/runners/common.py";,>
line 851, in invoke_process
self._invoke_process_per_window(
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/runners/common.py";,>
line 997, in _invoke_process_per_window
self.process_method(*args_for_process, **kwargs_for_process),
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/iobase.py";,>
line 1226, in _pre_finalize
return sink.pre_finalize(init_result, write_results)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filebasedsink.py";,>
line 225, in pre_finalize
file_metadata.path for mr in FileSystems.match([dst_glob])
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems.py";,>
line 204, in match
return filesystem.match(patterns, limits)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystem.py";,>
line 799, in match
result.append(_match(pattern, limit))
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystem.py";,>
line 788, in _match
for file_metadata in self.match_files(file_metadatas, pattern):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystem.py";,>
line 669, in match_files
for file_metadata in file_metas:
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/localfilesystem.py";,>
line 122, in _list
for f in list_files(dir_or_prefix):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/localfilesystem.py";,>
line 117, in list_files
for dirpath, _, files in os.walk(root):
File "/usr/lib/python3.8/os.py", line 413, in walk
yield from walk(new_path, topdown, onerror, followlinks)
File "/usr/lib/python3.8/os.py", line 362, in walk
entry = next(scandir_it)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/build/gradleenv/1398941892/lib/python3.8/site-packages/pytest_timeout.py";,>
line 264, in handler
timeout_sigalrm(item, settings)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/build/gradleenv/1398941892/lib/python3.8/site-packages/pytest_timeout.py";,>
line 448, in timeout_sigalrm
pytest.fail("Timeout >%ss" % settings.timeout)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/build/gradleenv/1398941892/lib/python3.8/site-packages/_pytest/outcomes.py";,>
line 198, in fail
raise Failed(msg=reason, pytrace=pytrace)
Failed: Timeout >600.0s
=============================== warnings summary ===============================
apache_beam/io/gcp/bigquery.py:2603
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2603:
DeprecationWarning: invalid escape sequence \#
"""Read data from BigQuery.
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63:
PendingDeprecationWarning: Client.dataset is deprecated and will be removed in
a future version. Use a string like 'my_project.my_dataset' or a
cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100:
PendingDeprecationWarning: Client.dataset is deprecated and will be removed in
a future version. Use a string like 'my_project.my_dataset' or a
cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py38.xml>
-
=========================== short test summary info ============================
FAILED
apache_beam/examples/complete/juliaset/juliaset/juliaset_test.py::JuliaSetTest::test_output_file_format
- Failed: Timeout >600.0s [while running
'WriteToText/Write/WriteImpl/PreFinalize']
= 1 failed, 22 passed, 18 skipped, 7264 deselected, 4 warnings in 835.30s
(0:13:55) =
> Task :sdks:python:test-suites:direct:py38:examples FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 93
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py311:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 93
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py310:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 93
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py39:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'>
line: 93
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
For more on this, please refer to
https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings
in the Gradle documentation.
BUILD FAILED in 21m 24s
18 actionable tasks: 17 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/7iyo3tdk6rtw2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
