[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=320189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320189 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 29/Sep/19 10:48 Start Date: 29/Sep/19 10:48 Worklog Time Spent: 10m Work Description: sunjincheng121 commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-536282790 Hi @mxm, thanks for your reply. I am glad to share my thoughts as follows: 1) We can create a JIRA if there is no JIRA number. This is also mentioned by @dmvk. Actually I find that there are many commits without JIRA numbers in Beam, so I don't think this is a strong reason to revert a commit. (BTW, this PR is helping @dmvk to fix BEAM-7730's PR issue: https://github.com/apache/beam/pull/9296#issuecomment-525262607). 2) Regarding to breaking the Python part, the tests already passed when merging this PR and so we didn't find this problem.Besides, as this commit doesn't break the master tests and will not block other contributors, so we have enough time to fix the Python part if we find that it is broken. So we have other better choices instead of reverting the commit. 3) Regarding to the changes to fix the Python problem on top of this PR, I think less changes are needed and only need add `encode_nested` and `decode_nested` in `coder.py` and `bundle_processor.py` for using the `encode_nested`(It's very clear in your PR). In this way, we can not only solve this problem, but also encourage contributors and committers. BTW, not all the new contributors know the rules of Beam well. We should have a better way to correct the mistakes of new contributors. Reverting is not the only way, nor the best way. I believe we have the same goal: to find a better way to make contributors happy to contribute to the Beam community. So I just share my thoughts not only for this PR but also for how to deal with same kinds of problems in the future. Great thanks for affirmation of my contribution. Feel free to correct me if there are anything incorrect. Best, Jincheng This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 320189) Time Spent: 1h 50m (was: 1h 40m) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[
https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=319494=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319494
]
ASF GitHub Bot logged work on BEAM-8162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 11:32
Start Date: 27/Sep/19 11:32
Worklog Time Spent: 10m
Work Description: mxm commented on issue #9464: [BEAM-8162] Encode keys
as NESTED for flink keyselector
URL: https://github.com/apache/beam/pull/9464#issuecomment-535902308
I very much appreciate your work on this @sunjincheng121. Let it be said,
your contributions are recognized. I also have to repeat that your solution in
this PR was incomplete. Not only did it not adhere to our contribution standard
because it was originally without a JIRA ("hotfix"), but it was also missing
the Python part which would have broken the Python support, as our tests
showed. It required careful investigation to find out the root of the problem
and to not break any other SDK as the result of a quick fix. We could not
iterate on your PR because it was already merged, so reverting it was the only
option.
I know that getting one's commit reverted is not a good feeling. It happens
to everyone eventually, and it is not to diminish your contributors in any way.
Keep up the great contributions. I know that everyone in the Beam community
greatly appreciates your work.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319494)
Time Spent: 1h 40m (was: 1.5h)
> NPE error when add flink 1.9 runner
> ---
>
> Key: BEAM-8162
> URL: https://issues.apache.org/jira/browse/BEAM-8162
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
>Reporter: sunjincheng
>Assignee: sunjincheng
>Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we
> find an NPE error when run the `PortableTimersExecutionTest`.
> the detail can be found here:
> https://github.com/apache/beam/pull/9296#issuecomment-525262607
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=311286=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311286 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 12/Sep/19 09:03 Start Date: 12/Sep/19 09:03 Worklog Time Spent: 10m Work Description: dmvk commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-530735523 @sunjincheng121 https://github.com/apache/beam/pull/9484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311286) Time Spent: 1h 20m (was: 1h 10m) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=311276=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311276 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 12/Sep/19 08:57 Start Date: 12/Sep/19 08:57 Worklog Time Spent: 10m Work Description: sunjincheng121 commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-530733113 Is there any progress here? @mxm This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311276) Time Spent: 1h 10m (was: 1h) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=308096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308096 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 06/Sep/19 19:05 Start Date: 06/Sep/19 19:05 Worklog Time Spent: 10m Work Description: sunjincheng121 commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-528977118 > > I can see you already do more effort for that PR. > > I don't consider a response time of one day to be long. Please keep in mind that people may be occupied with other tasks. Sorry, maybe I didn't express my meaning correctly. I meant that, we already do our best to fix it. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308096) Time Spent: 1h (was: 50m) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=308025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308025 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 06/Sep/19 17:19 Start Date: 06/Sep/19 17:19 Worklog Time Spent: 10m Work Description: sunjincheng121 commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-528938882 Hi @mxm @tweise I think you are much familiar with this code. So I just share my thoughts, and looking forward your decision :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308025) Time Spent: 50m (was: 40m) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=308001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-308001 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 06/Sep/19 16:25 Start Date: 06/Sep/19 16:25 Worklog Time Spent: 10m Work Description: mxm commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-528921688 >I can see you already do more effort for that PR. I don't consider a response time of one day to be long. Please keep in mind that people may be occupied with other tasks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 308001) Time Spent: 40m (was: 0.5h) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=307998=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307998 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 06/Sep/19 16:22 Start Date: 06/Sep/19 16:22 Worklog Time Spent: 10m Work Description: mxm commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-528920697 To comment on your summary, only approach `#2` and `#3` are possible because we cannot change the SDKs internal encoding. The encoding should be transparent to the Runner and independent of the key encoding a Runner chooses. The state request handler which caused the issues happens to leak the internal encoding of the SDK. I agree that it would be convenient to just use NESTED context. I'm not against it. However, I think it is also valid to keep the existing behavior, as the state requests is the only interface that exposes the raw key. In any case, it makes sense to think about this, test this, and ask the people who are most familiar with the code. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307998) Time Spent: 0.5h (was: 20m) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=307983=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307983 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 06/Sep/19 16:06 Start Date: 06/Sep/19 16:06 Worklog Time Spent: 10m Work Description: sunjincheng121 commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-528915026 Yes @mxm we already do your best try to make it available. and I can see you already do more effort for that PR. Best, Jincheng This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307983) Time Spent: 20m (was: 10m) > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (BEAM-8162) NPE error when add flink 1.9 runner
[ https://issues.apache.org/jira/browse/BEAM-8162?focusedWorklogId=307976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307976 ] ASF GitHub Bot logged work on BEAM-8162: Author: ASF GitHub Bot Created on: 06/Sep/19 15:58 Start Date: 06/Sep/19 15:58 Worklog Time Spent: 10m Work Description: mxm commented on issue #9464: [BEAM-8162] Encode keys as NESTED for flink keyselector URL: https://github.com/apache/beam/pull/9464#issuecomment-528912481 >(After all, @dmvk 's PR has been opened for a long time.） It may have been open for a long time but it was only considered ready-to-review with tests passing yesterday. I too want this to be part of 1.9 but we can't force it. Things need enough time to be reviewed to ensure the quality of the next Beam release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307976) Remaining Estimate: 0h Time Spent: 10m > NPE error when add flink 1.9 runner > --- > > Key: BEAM-8162 > URL: https://issues.apache.org/jira/browse/BEAM-8162 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: sunjincheng >Assignee: sunjincheng >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When add flink 1.9 runner in https://github.com/apache/beam/pull/9296, we > find an NPE error when run the `PortableTimersExecutionTest`. > the detail can be found here: > https://github.com/apache/beam/pull/9296#issuecomment-525262607 -- This message was sent by Atlassian Jira (v8.3.2#803003)
