See
<https://builds.apache.org/job/beam_PostCommit_Python36/878/display/redirect?page=changes>
Changes:
[jeff] [BEAM-8540] Fix CSVSink example in FileIO docs
------------------------------------------
[...truncated 79.35 KB...]
warnings.warn('Datastore IO will support Python 3 after replacing '
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/vcfio.py>:47:
UserWarning: VCF IO will support Python 3 after migration to Nucleus, see:
BEAM-5628.
warnings.warn("VCF IO will support Python 3 after migration to Nucleus, "
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/transforms/trigger_test.py>:509:
YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as
the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full
details.
for spec in yaml.load_all(open(transcript_filename)):
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
> Task :sdks:python:test-suites:direct:py36:postCommitIT FAILED
test_bigquery_read_1M_python
(apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT) ... ok
test_datastore_write_limit
(apache_beam.io.gcp.datastore.v1new.datastore_write_it_test.DatastoreWriteIT)
... ok
test_big_query_read
(apache_beam.io.gcp.bigquery_read_it_test.BigQueryReadIntegrationTests) ... ok
test_big_query_read_new_types
(apache_beam.io.gcp.bigquery_read_it_test.BigQueryReadIntegrationTests) ... ok
test_streaming_data_only
(apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_streaming_with_attributes
(apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_big_query_legacy_sql
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... ok
test_big_query_new_types
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... FAIL
test_big_query_standard_sql
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... ok
test_big_query_standard_sql_kms_key_native
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
... SKIP: This test doesn't work on DirectRunner.
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_big_query_write
(apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_new_types
(apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_schema_autodetect
(apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_without_schema
(apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
======================================================================
FAIL: test_big_query_new_types
(apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_it_test.py";,>
line 217, in test_big_query_new_types
big_query_query_to_table_pipeline.run_bq_pipeline(options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py";,>
line 82, in run_bq_pipeline
result = p.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/testing/test_pipeline.py";,>
line 112, in run
else test_runner_api))
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 406, in run
self._options).run(False)
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 419, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/direct/test_direct_runner.py";,>
line 51, in run_pipeline
hc_assert_that(self.result, pickler.loads(on_success_matcher))
AssertionError:
Expected: (Test pipeline expected terminated in state: DONE and Expected
checksum is 24de460c4d344a4b77ccc4cc1acb7b7ffc11a214)
but: Expected checksum is 24de460c4d344a4b77ccc4cc1acb7b7ffc11a214 Actual
checksum is da39a3ee5e6b4b0d3255bfef95601890afd80709
-------------------- >> begin captured logging << --------------------
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Union[]
root: DEBUG: Unhandled type_constraint: Any
root: INFO: Running pipeline with DirectRunner.
root: INFO: Using location 'US' from table <TableReference
datasetId: 'python_query_to_table_15726232624583'
projectId: 'apache-beam-testing'
tableId: 'python_new_types_table'> referenced by query SELECT bytes, date,
time FROM [python_query_to_table_15726232624583.python_new_types_table]
root: WARNING: Dataset
apache-beam-testing:temp_dataset_dc55c8d26ffd4f6e91cb6b58e376fa95 does not
exist so we will create it as temporary with location=US
root: DEBUG: Creating or getting table <TableReference
datasetId: 'python_query_to_table_15726232624583'
projectId: 'apache-beam-testing'
tableId: 'output_table'> with schema {'fields': [{'name': 'bytes', 'type':
'BYTES', 'mode': 'NULLABLE'}, {'name': 'date', 'type': 'DATE', 'mode':
'NULLABLE'}, {'name': 'time', 'type': 'TIME', 'mode': 'NULLABLE'}]}.
root: DEBUG: Created the table with id output_table
root: INFO: Created table
apache-beam-testing.python_query_to_table_15726232624583.output_table with
schema <TableSchema
fields: [<TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'bytes'
type: 'BYTES'>, <TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'date'
type: 'DATE'>, <TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'time'
type: 'TIME'>]>. Result: <Table
creationTime: 1572623268778
etag: 'NxEBWov+WiiC35YWzpIA8g=='
id: 'apache-beam-testing:python_query_to_table_15726232624583.output_table'
kind: 'bigquery#table'
lastModifiedTime: 1572623268832
location: 'US'
numBytes: 0
numLongTermBytes: 0
numRows: 0
schema: <TableSchema
fields: [<TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'bytes'
type: 'BYTES'>, <TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'date'
type: 'DATE'>, <TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'time'
type: 'TIME'>]>
selfLink:
'https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/python_query_to_table_15726232624583/tables/output_table'
tableReference: <TableReference
datasetId: 'python_query_to_table_15726232624583'
projectId: 'apache-beam-testing'
tableId: 'output_table'>
type: 'TABLE'>.
root: DEBUG: Attempting to flush to all destinations. Total buffered: 4
root: DEBUG: Flushing data to
apache-beam-testing:python_query_to_table_15726232624583.output_table. Total 4
rows.
root: DEBUG: Passed: True. Errors are []
root: INFO: Attempting to perform query SELECT bytes, date, time FROM
`python_query_to_table_15726232624583.output_table`; to BQ
google.auth.transport._http_client: DEBUG: Making request: GET
http://169.254.169.254
google.auth.transport._http_client: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/project/project-id
urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3,
connect=None, read=None, redirect=None, status=None)
google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
metadata.google.internal:80
urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1"
200 144
google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token
urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/[email protected]/token
HTTP/1.1" 200 181
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
www.googleapis.com:443
urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST
/bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/queries/7d427e36-ce89-4b1e-9515-81a84ec6aabf?maxResults=0&timeoutMs=10000&location=US
HTTP/1.1" 200 None
urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/jobs/7d427e36-ce89-4b1e-9515-81a84ec6aabf?location=US
HTTP/1.1" 200 None
urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon048c823c0a769b32cf584b279f5c691663fa39b4/data
HTTP/1.1" 200 None
root: INFO: Read from given query (SELECT bytes, date, time FROM
`python_query_to_table_15726232624583.output_table`;), total rows 0
root: INFO: Generate checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709
--------------------- >> end captured logging << ---------------------
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1208:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:1208:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:795:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
----------------------------------------------------------------------
XML: nosetests-postCommitIT-direct-py36.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 15 tests in 24.563s
FAILED (SKIP=1, failures=1)
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
(unset)
test_datastore_wordcount_it
(apache_beam.examples.cookbook.datastore_wordcount_it_test.DatastoreWordCountIT)
... SKIP: This test still needs to be fixed on Python 3TODO: BEAM-4543
test_avro_it (apache_beam.examples.fastavro_it_test.FastavroIT) ... SKIP: Due
to a known issue in avro-python3 package, thistest is skipped until BEAM-6522
is addressed.
test_bigquery_tornadoes_it
(apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT)
... ok
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ok
test_streaming_wordcount_it
(apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT) ... ok
test_autocomplete_it
(apache_beam.examples.complete.autocomplete_test.AutocompleteTest) ... ok
test_leader_board_it
(apache_beam.examples.complete.game.leader_board_it_test.LeaderBoardIT) ... ok
test_game_stats_it
(apache_beam.examples.complete.game.game_stats_it_test.GameStatsIT) ... ok
Traceback (most recent call last):
File "setup.py", line 251, in <module>
'test': generate_protos_first(test),
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/setuptools/__init__.py";,>
line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/commands.py";,>
line 158, in run
TestProgram(argv=argv, config=self.__config)
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/core.py";,>
line 121, in __init__
**extra_args)
File "/usr/lib/python3.6/unittest/main.py", line 95, in __init__
self.runTests()
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/core.py";,>
line 207, in runTests
result = self.testRunner.run(self.test)
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/plugins/multiprocess.py";,>
line 396, in run
timeout=nexttimeout)
File "<string>", line 2, in get
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in
_callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in
_recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Terminated
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=f608f287-54a6-44aa-a6bd-30b9ff99f517,
currentDir=<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 11457
log file: /home/jenkins/.gradle/daemon/5.2.1/daemon-11457.out.log
----- Last 20 lines from daemon log file - daemon-11457.out.log -----
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/core.py";,>
line 121, in __init__
**extra_args)
File "/usr/lib/python3.6/unittest/main.py", line 95, in __init__
self.runTests()
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/core.py";,>
line 207, in runTests
result = self.testRunner.run(self.test)
File
"<https://builds.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/nose/plugins/multiprocess.py";,>
line 396, in run
timeout=nexttimeout)
File "<string>", line 2, in get
File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in
_callmethod
kind, result = conn.recv()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in
_recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Terminated
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
