See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/105/display/redirect>
Changes:
------------------------------------------
[...truncated 99.01 KB...]
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_9"
transform_id:
"WriteToBigQuery/StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/Convert/Convert
to message"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1682152718
nanos: 580000000
}
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_9"
transform_id:
"WriteToBigQuery/StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Write
Records"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1682152719
nanos: 264000000
}
message: "Reconnecting for
stream:projects/apache-beam-testing/datasets/python_xlang_storage_write1682152707d62f1c/tables/python_storage_write_nested_records_and_lists/streams/Cig2N2QzNjA3Yy0wMDAwLTI2MWUtYmYyMy04ODNkMjRmZmZiYjA6czEx
id: eb79820f-ed72-4608-9d53-fd43e6bf1567"
log_location: "com.google.cloud.bigquery.storage.v1.StreamWriter"
thread: "48"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1682152719
nanos: 270000000
}
message: "Reconnect done for
stream:projects/apache-beam-testing/datasets/python_xlang_storage_write1682152707d62f1c/tables/python_storage_write_nested_records_and_lists/streams/Cig2N2QzNjA3Yy0wMDAwLTI2MWUtYmYyMy04ODNkMjRmZmZiYjA6czEx
id: eb79820f-ed72-4608-9d53-fd43e6bf1567"
log_location: "com.google.cloud.bigquery.storage.v1.StreamWriter"
thread: "48"
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1682152719
nanos: 693000000
}
message: "Application name is not set. Call Builder#setApplicationName."
instruction_id: "bundle_11"
transform_id:
"WriteToBigQuery/StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Finalize
writes/ParMultiDo(StorageApiFinalizeWrites)"
log_location: "com.google.api.client.googleapis.services.AbstractGoogleClient"
thread: "30"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1682152719
nanos: 819000000
}
message: "Finalize of stream
projects/apache-beam-testing/datasets/python_xlang_storage_write1682152707d62f1c/tables/python_storage_write_nested_records_and_lists/streams/Cig2N2QzNjA3Yy0wMDAwLTI2MWUtYmYyMy04ODNkMjRmZmZiYjA6czEx
finished with row_count: 1\n"
instruction_id: "bundle_11"
transform_id:
"WriteToBigQuery/StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Finalize
writes/ParMultiDo(StorageApiFinalizeWrites)"
log_location: "org.apache.beam.sdk.io.gcp.bigquery.StorageApiFinalizeWritesDoFn"
thread: "30"
[32mINFO [0m root:worker_handlers.py:415 severity: INFO
timestamp {
seconds: 1682152720
nanos: 268000000
}
message: "BatchCommit succeeded for tableId
projects/apache-beam-testing/datasets/python_xlang_storage_write1682152707d62f1c/tables/python_storage_write_nested_records_and_lists
response commit_time {\n seconds: 1682152720\n nanos: 254447000\n}\n"
instruction_id: "bundle_11"
transform_id:
"WriteToBigQuery/StorageWriteToBigQuery/SchemaAwareExternalTransform/ExternalTransform(beam:expansion:payload:schematransform:v1)/BigQueryIO.Write/StorageApiLoads/StorageApiWriteUnsharded/Finalize
writes/ParMultiDo(StorageApiFinalizeWrites)"
log_location: "org.apache.beam.sdk.io.gcp.bigquery.StorageApiFinalizeWritesDoFn"
thread: "30"
[32mINFO [0m root:environments.py:376 Default Python SDK image for
environment is apache/beam_python3.8_sdk:2.48.0.dev
[32mINFO [0m root:environments.py:376 Default Python SDK image for
environment is apache/beam_python3.8_sdk:2.48.0.dev
[33mWARNING [0m root:worker_handlers.py:415 severity: WARN
timestamp {
seconds: 1682152720
nanos: 304000000
}
message: "Hanged up for url: \"localhost:37271\"\n."
log_location: "org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer"
thread: "23"
aa77a59950ea1e46bea312912fdd0d49fb7f83b71a87a296cd67d09b22c5986c
[32mINFO [0m
apache_beam.io.gcp.tests.bigquery_matcher:bigquery_matcher.py:121 Attempting to
perform query SELECT * FROM
python_xlang_storage_write1682152707d62f1c.python_storage_write_nested_records_and_lists
to BQ
[32mINFO [0m
apache_beam.io.gcp.tests.bigquery_matcher:bigquery_matcher.py:158 Result of
query is: [([1, 2, 3], {'nested_int': 1, 'nested_str': 'a'},
[{'nested_numeric': Decimal('1.23'), 'nested_bytes': b'a'}, {'nested_numeric':
Decimal('3.21'), 'nested_bytes': b'aa'}])]
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:117
Deleting dataset python_xlang_storage_write1682152707d62f1c in project
apache-beam-testing
[32mPASSED[0m
=================================== FAILURES ===================================
[31m[1m___________ BigQueryXlangStorageWriteIT.test_storage_write_beam_rows
___________[0m
self =
<apache_beam.io.external.xlang_bigqueryio_it_test.BigQueryXlangStorageWriteIT
testMethod=test_storage_write_beam_rows>
def test_storage_write_beam_rows(self):
table_id = '{}:{}.python_xlang_storage_write_beam_rows'.format(
self.project, self.dataset_id)
row_elements = [
beam.Row(
my_int=e['int'],
my_float=e['float'],
my_numeric=e['numeric'],
my_string=e['str'],
my_bool=e['bool'],
my_bytes=e['bytes'],
my_timestamp=e['timestamp']) for e in self.ELEMENTS
]
bq_matcher = BigqueryFullResultMatcher(
project=self.project,
query="SELECT * FROM %s" %
'{}.python_xlang_storage_write_beam_rows'.format(self.dataset_id),
data=self.parse_expected_data(self.ELEMENTS))
with beam.Pipeline(argv=self.args) as p:
_ = (
p
| beam.Create(row_elements)
> | beam.io.StorageWriteToBigQuery(
table=table_id, expansion_service=self.expansion_service))
[1m[31mE AttributeError: module 'apache_beam.io' has no attribute
'StorageWriteToBigQuery'[0m
[1m[31mapache_beam/io/external/xlang_bigqueryio_it_test.py[0m:232:
AttributeError
------------------------------ Captured log call -------------------------------
[32mINFO [0m apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:816
Dataset apache-beam-testing:python_xlang_storage_write1682152706d38b95 does not
exist so we will create it as temporary with location=None
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:107
Created dataset python_xlang_storage_write1682152706d38b95 in project
apache-beam-testing
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:110
expansion port: 37013
[32mINFO [0m
apache_beam.io.external.xlang_bigqueryio_it_test:xlang_bigqueryio_it_test.py:117
Deleting dataset python_xlang_storage_write1682152706d38b95 in project
apache-beam-testing
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/1922375555/lib/python3.8/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:121
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py>:121:
DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2870:
18 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2870:
13 warnings
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2349
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2349
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py>:2349:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py:2870
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/pkg_resources/__init__.py>:2870:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.iam')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
../../build/gradleenv/1922375555/lib/python3.8/site-packages/google/rpc/__init__.py:20
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/google/rpc/__init__.py>:20:
DeprecationWarning: Deprecated call to
`pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is
preferred to `pkg_resources.declare_namespace`. See
https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
apache_beam/typehints/pandas_type_compatibility_test.py:67
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:67:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:91
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:91:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_storage_write_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_storage_write_nested_records_and_lists
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2029:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_storage_write_all_types
apache_beam/io/external/xlang_bigqueryio_it_test.py::BigQueryXlangStorageWriteIT::test_storage_write_nested_records_and_lists
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2035:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/ws/src/sdks/python/pytest_gcpCrossLanguage.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mFAILED[0m
apache_beam/io/external/xlang_bigqueryio_it_test.py::[1mBigQueryXlangStorageWriteIT::test_storage_write_beam_rows[0m
- AttributeError: module 'apache_beam.io' has no attribute
'StorageWriteToBigQuery'
[31m==== [31m[1m1 failed[0m, [32m2 passed[0m, [33m9 skipped[0m,
[33m6883 deselected[0m, [33m47 warnings[0m[31m in 44.83s[0m[31m =====[0m
> Task :sdks:python:test-suites:direct:py37:gcpCrossLanguagePythonUsingJava
> FAILED
> Task :sdks:python:test-suites:direct:py37:gcpCrossLanguageCleanup
Stopping expansion service pid: 3849658.
Skipping invalid pid: 3849659.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 3849710
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:py37:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 2m 19s
108 actionable tasks: 74 executed, 30 from cache, 4 up-to-date
Build scan background action failed.
java.lang.IllegalArgumentException:
com.gradle.enterprise.gradleplugin.internal.extension.a is not an interface
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:590)
at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:557)
at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:230)
at java.lang.reflect.WeakCache.get(WeakCache.java:127)
at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:719)
at
com.gradle.ProxyFactory$ProxyingInvocationHandler.createLocalProxy(ProxyFactory.java:64)
at
com.gradle.ProxyFactory$ProxyingInvocationHandler.lambda$adaptActionArg$1(ProxyFactory.java:59)
at
com.gradle.enterprise.gradleplugin.internal.extension.b$3.run(SourceFile:100)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Publishing build scan...
https://gradle.com/s/bw6tyu7njinj4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
