See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/2254/display/redirect>
Changes:
------------------------------------------
[...truncated 882.72 KB...]
expanding external transform
expanding transform with ExpansionRequest:
components:{pcollections:{key:"n2" value:{unique_name:"n2"
coder_id:"c0@ejDeUBJzJt" is_bounded:BOUNDED
windowing_strategy_id:"w0@ejDeUBJzJt"}}
windowing_strategies:{key:"w0@ejDeUBJzJt"
value:{window_fn:{urn:"beam:window_fn:global_windows:v1"}
merge_status:NON_MERGING window_coder_id:"c1@ejDeUBJzJt" trigger:{default:{}}
accumulation_mode:DISCARDING output_time:END_OF_WINDOW
closing_behavior:EMIT_IF_NONEMPTY OnTimeBehavior:FIRE_IF_NONEMPTY
environment_id:"go"}} coders:{key:"c0@ejDeUBJzJt"
value:{spec:{urn:"beam:coder:row:v1"
payload:"\n\r\n\x07role_id\x1a\x02\x10\x03\x12$abc5e222-ab8f-4f6f-800a-a58bb0f45675"}}}
coders:{key:"c1@ejDeUBJzJt" value:{spec:{urn:"beam:coder:global_window:v1"}}}
environments:{key:"go" value:{}}} transform:{unique_name:"External"
spec:{urn:"beam:transform:org.apache.beam:schemaio_jdbc_write:v1"
payload:"\nX\n\x0e\n\x08location\x1a\x02\x10\x07\n\x0c\n\x06config\x1a\x02\x10\t\n\x12\n\ndataSchema\x1a\x04\x08\x01\x10\t\x12$853bc92e-cc52-429e-bf30-f534bebcedce\x12`\x03\x01\x04\x05rolesV\n\x02\xf0\x03\x15org.postgresql.Driver*jdbc:postgresql://localhost:33293/postjdbc\x07newuser\x08password"}
inputs:{key:"i0" value:"n2"} environment_id:"go"} namespace:"ejDeUBJzJt"
expansion failed
caused by:
java.lang.UnsupportedOperationException: Unknown urn:
beam:transform:org.apache.beam:schemaio_jdbc_write:v1
at
org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:506)
at
org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:591)
at
org.apache.beam.model.expansion.v1.ExpansionServiceGrpc$MethodHandlers.invoke(ExpansionServiceGrpc.java:220)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:340)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:866)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[recovered]
panic: tried cross-language for
beam:transform:org.apache.beam:schemaio_jdbc_write:v1 against localhost:57059
and failed
expanding external transform
expanding transform with ExpansionRequest:
components:{pcollections:{key:"n2" value:{unique_name:"n2"
coder_id:"c0@ejDeUBJzJt" is_bounded:BOUNDED
windowing_strategy_id:"w0@ejDeUBJzJt"}}
windowing_strategies:{key:"w0@ejDeUBJzJt"
value:{window_fn:{urn:"beam:window_fn:global_windows:v1"}
merge_status:NON_MERGING window_coder_id:"c1@ejDeUBJzJt" trigger:{default:{}}
accumulation_mode:DISCARDING output_time:END_OF_WINDOW
closing_behavior:EMIT_IF_NONEMPTY OnTimeBehavior:FIRE_IF_NONEMPTY
environment_id:"go"}} coders:{key:"c0@ejDeUBJzJt"
value:{spec:{urn:"beam:coder:row:v1"
payload:"\n\r\n\x07role_id\x1a\x02\x10\x03\x12$abc5e222-ab8f-4f6f-800a-a58bb0f45675"}}}
coders:{key:"c1@ejDeUBJzJt" value:{spec:{urn:"beam:coder:global_window:v1"}}}
environments:{key:"go" value:{}}} transform:{unique_name:"External"
spec:{urn:"beam:transform:org.apache.beam:schemaio_jdbc_write:v1"
payload:"\nX\n\x0e\n\x08location\x1a\x02\x10\x07\n\x0c\n\x06config\x1a\x02\x10\t\n\x12\n\ndataSchema\x1a\x04\x08\x01\x10\t\x12$853bc92e-cc52-429e-bf30-f534bebcedce\x12`\x03\x01\x04\x05rolesV\n\x02\xf0\x03\x15org.postgresql.Driver*jdbc:postgresql://localhost:33293/postjdbc\x07newuser\x08password"}
inputs:{key:"i0" value:"n2"} environment_id:"go"} namespace:"ejDeUBJzJt"
expansion failed
caused by:
java.lang.UnsupportedOperationException: Unknown urn:
beam:transform:org.apache.beam:schemaio_jdbc_write:v1
at
org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:506)
at
org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:591)
at
org.apache.beam.model.expansion.v1.ExpansionServiceGrpc$MethodHandlers.invoke(ExpansionServiceGrpc.java:220)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:340)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:866)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
goroutine 50 [running]:
testing.tRunner.func1.2(0xe57600, 0xc00011c640)
/home/jenkins/sdk/go1.16.12/src/testing/testing.go:1143 +0x332
testing.tRunner.func1(0xc000103b00)
/home/jenkins/sdk/go1.16.12/src/testing/testing.go:1146 +0x4b6
panic(0xe57600, 0xc00011c640)
/home/jenkins/sdk/go1.16.12/src/runtime/panic.go:965 +0x1b9
github.com/apache/beam/sdks/v2/go/pkg/beam.CrossLanguage(0xc0003e6ba0,
0xc000624820, 0xfa25f3, 0x35, 0xc00048c000, 0xbc, 0xbc, 0x7ffc86b0942e, 0xf,
0xc00029fb38, ...)
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/pkg/beam/xlang.go>:162
+0x1f7
github.com/apache/beam/sdks/v2/go/pkg/beam/io/xlang/jdbcio.Write(0xc0003e6b80,
0xc000624820, 0x7ffc86b0942e, 0xf, 0xf74760, 0x5, 0xf84bb9, 0x15, 0xc00019e4e0,
0x2a, ...)
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/pkg/beam/io/xlang/jdbcio/jdbc.go>:129
+0x4c7
github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/jdbc.writeRows(0xc0003e6ae0,
0xc000624820, 0x7ffc86b0942e, 0xf, 0xf74760, 0x5, 0xf84bb9, 0x15,
0xc00019e4e0, 0x2a, ...)
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/test/integration/io/xlang/jdbc/jdbc.go>:39
+0x1be
github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/jdbc.WritePipeline(0x7ffc86b0942e,
0xf, 0xf74760, 0x5, 0xf84bb9, 0x15, 0xc00019e4e0, 0x2a, 0xf76d45, 0x7, ...)
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/test/integration/io/xlang/jdbc/jdbc.go>:49
+0x20d
github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/jdbc.TestJDBCIO_BasicReadWrite(0xc000103b00)
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/test/integration/io/xlang/jdbc/jdbc_test.go>:104
+0x227
testing.tRunner(0xc000103b00, 0xfb89f0)
/home/jenkins/sdk/go1.16.12/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
/home/jenkins/sdk/go1.16.12/src/testing/testing.go:1238 +0x2b3
FAIL github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/jdbc
21.851s
=== RUN TestKafkaIO_BasicReadWrite
integration.go:184: Test TestKafkaIO_BasicReadWrite is currently filtered
for runner portable
--- SKIP: TestKafkaIO_BasicReadWrite (0.00s)
PASS
ok github.com/apache/beam/sdks/v2/go/test/integration/io/xlang/kafka
3.031s
FAIL
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 1
$ exit 1
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerGoUsingJava
> FAILED
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:39249 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 5179 items / 5169 deselected / 10 selected
apache_beam/io/external/generate_sequence_test.py .
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingJava
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPython
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPythonOnly
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingJava
.. [ 30%]
apache_beam/transforms/validate_runner_xlang_test.py ....... [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/pytest_xlangValidateRunner.xml>
-
=========== 10 passed, 5169 deselected, 2 warnings in 123.52 seconds ===========
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingPython
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:39249 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 5179 items / 5169 deselected / 10 selected
apache_beam/io/external/generate_sequence_test.py .ss [ 30%]
apache_beam/transforms/validate_runner_xlang_test.py ....... [100%]
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/pytest_xlangValidateRunner.xml>
-
============ 8 passed, 2 skipped, 5169 deselected in 61.65 seconds =============
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingSql
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:39249 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_sql_expansion_service
============================= test session starts ==============================
platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 5179 items / 5170 deselected / 9 selected
apache_beam/transforms/sql_test.py ..F...... [100%]
=================================== FAILURES ===================================
_____________________ SqlTransformTest.test_generate_data ______________________
self = <apache_beam.transforms.sql_test.SqlTransformTest
testMethod=test_generate_data>
def test_generate_data(self):
with TestPipeline() as p:
out = p | SqlTransform(
"""SELECT
CAST(1 AS INT) AS `id`,
CAST('foo' AS VARCHAR) AS `str`,
> CAST(3.14 AS DOUBLE) AS `flt`""")
apache_beam/transforms/sql_test.py:86:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/transforms/ptransform.py:614: in __ror__
result = p.apply(self, pvalueish, label)
apache_beam/pipeline.py:708: in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
apache_beam/runners/runner.py:185: in apply
return m(transform, input, options)
apache_beam/runners/runner.py:215: in apply_PTransform
return transform.expand(input)
apache_beam/transforms/external.py:509: in expand
response = service.Expand(request)
../../build/gradleenv/1922375555/lib/python3.6/site-packages/grpc/_channel.py:946:
in __call__
return _end_unary_response_blocking(state, call, False, None)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
state = <grpc._channel._RPCState object at 0x7fad04715198>
call = <grpc._cython.cygrpc.SegregatedCall object at 0x7fad04773ac8>
with_call = False, deadline = None
def _end_unary_response_blocking(state, call, with_call, deadline):
if state.code is grpc.StatusCode.OK:
if with_call:
rendezvous = _MultiThreadedRendezvous(state, call, None,
deadline)
return state.response, rendezvous
else:
return state.response
else:
> raise _InactiveRpcError(state)
E grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that
terminated with:
E status = StatusCode.UNIMPLEMENTED
E details = "Method not found:
org.apache.beam.model.expansion.v1.ExpansionService/Expand"
E debug_error_string =
"{"created":"@1643740784.647194851","description":"Error received from peer
ipv6:[::1]:46309","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Method
not found:
org.apache.beam.model.expansion.v1.ExpansionService/Expand","grpc_status":12}"
E >
../../build/gradleenv/1922375555/lib/python3.6/site-packages/grpc/_channel.py:849:
_InactiveRpcError
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/pytest_xlangSqlValidateRunner.xml>
-
============ 1 failed, 8 passed, 5170 deselected in 253.16 seconds =============
> Task
> :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingSql
> FAILED
> Task :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 235562.
Stopping expansion service pid: 235563.
> Task :sdks:python:test-suites:direct:xlang:fnApiJobServerCleanup
Killing process at 169342
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Build file
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/go/test/build.gradle'>
line: 170
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerGoUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerPythonUsingSql'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 39m 53s
200 actionable tasks: 139 executed, 52 from cache, 9 up-to-date
Publishing build scan...
https://gradle.com/s/v4mphzuyfooni
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
