[
https://issues.apache.org/jira/browse/BEAM-6077?focusedWorklogId=171106&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-171106
]
ASF GitHub Bot logged work on BEAM-6077:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Nov/18 15:37
Start Date: 30/Nov/18 15:37
Worklog Time Spent: 10m
Work Description: JozoVilcek commented on issue #7128: [BEAM-6077] If
available, use max_parallelism for splitting unbounded source
URL: https://github.com/apache/beam/pull/7128#issuecomment-443241807
I have updated test pipeline options and use of max-parallelism for bounded
source wrapper.
As for requested tests, I find it quite difficult right now. Code around
translators is quite tight and private. I am struggling on how to create
PTransform<> which would be hit by
`FlinkStreamingTransformTranslators.getTranslator()` via URN match and can call
`translator.translateNode()`. Transforms must be quite real otherwise it fails
on `toProto`->`fromProto` exercise. Also types. Mocking started to be quite
complex here.
I did not have a success to find similar tests to learn from. Any other
alternative approach on how translation layer should be approached for testing?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 171106)
Time Spent: 50m (was: 40m)
> Make UnboundedSource state rescale friendly
> -------------------------------------------
>
> Key: BEAM-6077
> URL: https://issues.apache.org/jira/browse/BEAM-6077
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Affects Versions: 2.8.0
> Reporter: Jozef Vilcek
> Assignee: Jozef Vilcek
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> FlinkRunner's UnboundedSourceWrapper currently does not rescale well it's
> state when job changes parallelism.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
