[
https://issues.apache.org/jira/browse/BEAM-5760?focusedWorklogId=155458&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-155458
]
ASF GitHub Bot logged work on BEAM-5760:
----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Oct/18 15:25
Start Date: 17/Oct/18 15:25
Worklog Time Spent: 10m
Work Description: robertwb commented on issue #6723: [WIP] [BEAM-5760]
Add support for multi-element bundles to portable Flink runner.
URL: https://github.com/apache/beam/pull/6723#issuecomment-430673927
Thanks, I'll take a look at this. It should help a lot (as getting the
per-bundle overhead comparable to the time spent plumbing a single element
through would be difficult; just constructing and parsing the various
messages and channels would take time).
Is there a good, brief summary as to why we had the one-element-per-bundle
setup in the first place? I have in the back of my head it was something to
do with how checkpoints and/or watermarks are controlled in Flink, but
could be wrong.
On Wed, Oct 17, 2018 at 5:02 PM Thomas Weise <[email protected]>
wrote:
> @tweise <https://github.com/tweise> requested your review on: #6723
> <https://github.com/apache/beam/pull/6723> [WIP] [BEAM-5760] Add support
> for multi-element bundles to portable Flink runner..
>
> —
> You are receiving this because your review was requested.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/6723#event-1909825120>, or mute the
> thread
>
<https://github.com/notifications/unsubscribe-auth/AAdqgaxCvhq39_qMgsE8-tPAtaYPjeWvks5ul0aPgaJpZM4XkIxu>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 155458)
Time Spent: 20m (was: 10m)
> Portable Flink support for maxBundleSize/maxBundleMillis
> --------------------------------------------------------
>
> Key: BEAM-5760
> URL: https://issues.apache.org/jira/browse/BEAM-5760
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Affects Versions: 2.8.0
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Priority: Major
> Labels: portability-flink
> Fix For: 2.9.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The portable runner needs to support larger bundles in streaming mode.
> Currently every element is a separate bundle, which is very inefficient due
> to the per bundle SDK worker overhead. The old Java SDK runner already
> supports these parameters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
