[jira] [Commented] (BEAM-9137) PR10338 breaks beam_PostCommit_Py_ValCont

2020-01-17 Thread Boyuan Zhang (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018452#comment-17018452
 ] 

Boyuan Zhang commented on BEAM-9137:


I'll drop it from release blocker since it doesn't make sense for now.

> PR10338 breaks beam_PostCommit_Py_ValCont
> -
>
> Key: BEAM-9137
> URL: https://issues.apache.org/jira/browse/BEAM-9137
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Affects Versions: 2.19.0
>Reporter: Boyuan Zhang
>Priority: Blocker
> Fix For: 2.19.0
>
>
> For the first failure, please refer to 
> https://builds.apache.org/job/beam_PostCommit_Py_ValCont/5172/#showFailuresLink



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (BEAM-9137) PR10338 breaks beam_PostCommit_Py_ValCont

2020-01-17 Thread Boyuan Zhang (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018451#comment-17018451
 ] 

Boyuan Zhang commented on BEAM-9137:


Turns out the root cause should be the python ValidationContainer tests are 
kind of mis-configured. The test is running with dataflow runner, but not 
provides a dataflow_worker_jar option, which forces dataflow service pulls a 
pre-built runner image. So the mismatch between sdk(built with latest code) and 
runner(maybe last release ver) will break this test.

> PR10338 breaks beam_PostCommit_Py_ValCont
> -
>
> Key: BEAM-9137
> URL: https://issues.apache.org/jira/browse/BEAM-9137
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Affects Versions: 2.19.0
>Reporter: Boyuan Zhang
>Priority: Blocker
> Fix For: 2.19.0
>
>
> For the first failure, please refer to 
> https://builds.apache.org/job/beam_PostCommit_Py_ValCont/5172/#showFailuresLink



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (BEAM-9137) PR10338 breaks beam_PostCommit_Py_ValCont

2020-01-17 Thread Boyuan Zhang (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018354#comment-17018354
 ] 

Boyuan Zhang commented on BEAM-9137:


Thanks for your confirmation and update! Turns out this test target only fails 
on Py2 and even the last successful commit also failed on the same error:

Error processing instruction -77. Original traceback is Traceback (most recent 
call last): File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
 line 161, in _execute response = task() File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
 line 217, in  lambda: self.create_worker().do_instruction(request), 
request) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
 line 346, in do_instruction request.instruction_id) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
 line 374, in process_bundle instruction_id, 
request.process_bundle_descriptor_id) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
 line 282, in get self.data_channel_factory) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 717, in __init__ self.ops = 
self.create_execution_tree(self.process_bundle_descriptor) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 766, in create_execution_tree descriptor.transforms, 
key=topological_height, reverse=True)]) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 680, in wrapper result = cache[args] = func(*args) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 748, in get_operation in 
descriptor.transforms[transform_id].outputs.items() File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 747, in  for tag, pcoll_id File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 680, in wrapper result = cache[args] = func(*args) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 748, in get_operation in 
descriptor.transforms[transform_id].outputs.items() File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 747, in  for tag, pcoll_id File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 680, in wrapper result = cache[args] = func(*args) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 748, in get_operation in 
descriptor.transforms[transform_id].outputs.items() File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 747, in  for tag, pcoll_id File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 680, in wrapper result = cache[args] = func(*args) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 751, in get_operation transform_id, transform_consumers) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1053, in create_operation return creator(self, transform_id, 
transform_proto, payload, consumers) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1168, in create output_coder = 
factory.get_only_input_coder(transform_proto) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1099, in get_only_input_coder return 
only_element(list(self.get_input_coders(transform_proto).values())) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1094, in get_input_coders for tag, pcoll_id in 
transform_proto.inputs.items() File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1094, in  for tag, pcoll_id in transform_proto.inputs.items() 
File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1069, in get_windowed_coder coder = 
self.get_coder(self.descriptor.pcollections[pcoll_id].coder_id) File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py",
 line 1065, in get_coder json.loads(coder_proto.spec.payload.decode('utf-8'))) 
File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/operation_specs.py",
 line 331, in get_coder_from_spec get_coder_from_spec(c) for c in 
coder_spec['component_encodings']] File 
"/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/operation_specs.py",
 line 349, in get_coder_from_spec 
get_coder_from_spec(coder_spec['component_encodings'][0])) File 

[jira] [Commented] (BEAM-9137) PR10338 breaks beam_PostCommit_Py_ValCont

2020-01-16 Thread sunjincheng (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017754#comment-17017754
 ] 

sunjincheng commented on BEAM-9137:
---

I have a quick look at of this issue and it seems that the 
beam_PostCommit_Py_ValCont was broken on 20 Dec, 2019.

We can see that it succeed on [19 Dec 2019 
|http://mail-archives.apache.org/mod_mbox/beam-builds/201912.mbox/%3c100795298.3239.1576775433670.JavaMail.jenkins@jenkins02%3e]and
 failed on [20 Dec, 
2019|http://mail-archives.apache.org/mod_mbox/beam-builds/201912.mbox/%3c165865149.3447.1576823413985.JavaMail.jenkins@jenkins02%3e].

> PR10338 breaks beam_PostCommit_Py_ValCont
> -
>
> Key: BEAM-9137
> URL: https://issues.apache.org/jira/browse/BEAM-9137
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Affects Versions: 2.19.0
>Reporter: Boyuan Zhang
>Priority: Blocker
> Fix For: 2.19.0
>
>
> For the first failure, please refer to 
> https://builds.apache.org/job/beam_PostCommit_Py_ValCont/5172/#showFailuresLink



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (BEAM-9137) PR10338 breaks beam_PostCommit_Py_ValCont

2020-01-16 Thread sunjincheng (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017747#comment-17017747
 ] 

sunjincheng commented on BEAM-9137:
---

It seems that the test error is not caused by[1], more detail log can be found 
in [2]. [~boyuanz] 

[1] [https://github.com/apache/beam/pull/10338]

[2] [https://github.com/apache/beam/pull/10338#issuecomment-575482269] 

> PR10338 breaks beam_PostCommit_Py_ValCont
> -
>
> Key: BEAM-9137
> URL: https://issues.apache.org/jira/browse/BEAM-9137
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Affects Versions: 2.19.0
>Reporter: Boyuan Zhang
>Assignee: sunjincheng
>Priority: Blocker
> Fix For: 2.19.0
>
>
> For the first failure, please refer to 
> https://builds.apache.org/job/beam_PostCommit_Py_ValCont/5172/#showFailuresLink



--
This message was sent by Atlassian Jira
(v8.3.4#803005)