[jira] [Assigned] (BEAM-1116) Support for new Timer API in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1116: -- Assignee: Jingsong Lee > Support for new Timer API in Flink runner >

[jira] [Commented] (BEAM-25) Add user-ready API for interacting with state

[ https://issues.apache.org/jira/browse/BEAM-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855665#comment-15855665 ] Jingsong Lee commented on BEAM-25: -- [~kenn] Thanks for reply. But I need scan all elements in BagState every

[jira] [Assigned] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1393: -- Assignee: Jingsong Lee > Update Flink Runner to Flink 1.2.0 >

[jira] [Commented] (BEAM-41) State for "Set" data, supporting efficient membership checks

[ https://issues.apache.org/jira/browse/BEAM-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857281#comment-15857281 ] Jingsong Lee commented on BEAM-41: -- Yes, `add()` in CombiningState has no return value, so I can't override

[jira] [Commented] (BEAM-42) State for "Map" data, supporting efficient key lookup, etc.

[ https://issues.apache.org/jira/browse/BEAM-42?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857350#comment-15857350 ] Jingsong Lee commented on BEAM-42: -- MapState maybe need more "sighted write"(contrary to "blind write").

[jira] [Commented] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856453#comment-15856453 ] Jingsong Lee commented on BEAM-1393: (y) But there is a problem with the 6 point.

[jira] [Assigned] (BEAM-1498) Use Flink-native side outputs

[ https://issues.apache.org/jira/browse/BEAM-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1498: -- Assignee: Jingsong Lee > Use Flink-native side outputs > - > >

[jira] [Commented] (BEAM-1498) Use Flink-native side outputs

[ https://issues.apache.org/jira/browse/BEAM-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876026#comment-15876026 ] Jingsong Lee commented on BEAM-1498: Great~ That will simplify the implementation of

[jira] [Commented] (BEAM-1116) Support for new Timer API in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871001#comment-15871001 ] Jingsong Lee commented on BEAM-1116: [~kenn] I found that {{onTimer()}} requires the {{BoundedWindow}}

[jira] [Commented] (BEAM-1476) Support MapState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882055#comment-15882055 ] Jingsong Lee commented on BEAM-1476: We can implement it by Flink MapState in Flink 1.3.

[jira] [Commented] (BEAM-1483) Support SetState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882056#comment-15882056 ] Jingsong Lee commented on BEAM-1483: We can implement it by Flink MapState in Flink 1.3, like HashSet

[jira] [Commented] (BEAM-1517) Garbage collect user state in Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876005#comment-15876005 ] Jingsong Lee commented on BEAM-1517: Is it appropriate for the user to do the work of GC? Just like

[jira] [Commented] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859138#comment-15859138 ] Jingsong Lee commented on BEAM-1393: About point 7, no matter what state we want, we just use the first

[jira] [Commented] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862290#comment-15862290 ] Jingsong Lee commented on BEAM-1393: Totally agree! {{AbstractStreamOperator}} will check the type of

[jira] [Commented] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862325#comment-15862325 ] Jingsong Lee commented on BEAM-1393: BTW, I think SPLIT_DISTRIBUTE state maybe need a new

[jira] [Comment Edited] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862325#comment-15862325 ] Jingsong Lee edited comment on BEAM-1393 at 2/11/17 9:08 AM: - BTW, I think

[jira] [Created] (BEAM-1456) Make UnboundedSourceWrapper snapshot to rescalable operator state in Flink Runner

Jingsong Lee created BEAM-1456: -- Summary: Make UnboundedSourceWrapper snapshot to rescalable operator state in Flink Runner Key: BEAM-1456 URL: https://issues.apache.org/jira/browse/BEAM-1456 Project:

[jira] [Commented] (BEAM-1393) Update Flink Runner to Flink 1.2.0

[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860992#comment-15860992 ] Jingsong Lee commented on BEAM-1393: Good point! The processing of pushed-back events is indeed a

[jira] [Created] (BEAM-1445) Use Flink broadcast state to store side-input data

Jingsong Lee created BEAM-1445: -- Summary: Use Flink broadcast state to store side-input data Key: BEAM-1445 URL: https://issues.apache.org/jira/browse/BEAM-1445 Project: Beam Issue Type:

[jira] [Commented] (BEAM-41) State for "Set" data, supporting efficient membership checks

[ https://issues.apache.org/jira/browse/BEAM-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825944#comment-15825944 ] Jingsong Lee commented on BEAM-41: -- Hi, Our project is great need of SetState. Is it blocked by some thing?

[jira] [Commented] (BEAM-25) Add user-ready API for interacting with state

[ https://issues.apache.org/jira/browse/BEAM-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831368#comment-15831368 ] Jingsong Lee commented on BEAM-25: -- Are there plans to support StatefulCombineFn? eg: select itemId,

[jira] [Commented] (BEAM-1116) Support for new Timer API in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887771#comment-15887771 ] Jingsong Lee commented on BEAM-1116: OK, I will finish it quickly. > Support for new Timer API in

[jira] [Commented] (BEAM-1517) Garbage collect user state in Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887550#comment-15887550 ] Jingsong Lee commented on BEAM-1517: [~kenn] Is it like {{LateDataDroppingDoFnRunner}} droping late

[jira] [Assigned] (BEAM-1476) Support MapState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1476: -- Assignee: Jingsong Lee > Support MapState in Flink runner > >

[jira] [Assigned] (BEAM-1483) Support SetState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1483: -- Assignee: Jingsong Lee > Support SetState in Flink runner > >

[jira] [Updated] (BEAM-1587) Use StringBuilder to stringKey of StateNamespace instead of String.format

[ https://issues.apache.org/jira/browse/BEAM-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1587: --- Description: In Flink Runner, each State visit will call the namespace stringKey once. Since

[jira] [Created] (BEAM-1588) Reuse StateNamespace.stringKey in Flink States

Jingsong Lee created BEAM-1588: -- Summary: Reuse StateNamespace.stringKey in Flink States Key: BEAM-1588 URL: https://issues.apache.org/jira/browse/BEAM-1588 Project: Beam Issue Type:

[jira] [Created] (BEAM-1587) Use StringBuilder to stringKey of StateNamespace instead of String.format

Jingsong Lee created BEAM-1587: -- Summary: Use StringBuilder to stringKey of StateNamespace instead of String.format Key: BEAM-1587 URL: https://issues.apache.org/jira/browse/BEAM-1587 Project: Beam

[jira] [Assigned] (BEAM-1587) Use StringBuilder to stringKey of StateNamespace instead of String.format

[ https://issues.apache.org/jira/browse/BEAM-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1587: -- Assignee: Jingsong Lee (was: Kenneth Knowles) > Use StringBuilder to stringKey of

[jira] [Assigned] (BEAM-1036) Support for new State API in FlinkRunner

[ https://issues.apache.org/jira/browse/BEAM-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1036: -- Assignee: Jingsong Lee (was: Aljoscha Krettek) > Support for new State API in FlinkRunner >

[jira] [Commented] (BEAM-1560) Use SimpleDoFnRunner to invoke and use new CombineFnRunner in batch mode of Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885493#comment-15885493 ] Jingsong Lee commented on BEAM-1560: [~aljoscha] Because it is independent of streaming mode, so I did

[jira] [Commented] (BEAM-1772) Support merging WindowFn other than IntervalWindow on Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-1772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938119#comment-15938119 ] Jingsong Lee commented on BEAM-1772: The problem is that

[jira] [Commented] (BEAM-1772) Support merging WindowFn other than IntervalWindow on Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-1772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942408#comment-15942408 ] Jingsong Lee commented on BEAM-1772: Consider the performance, I retained the combining with sorted

[jira] [Commented] (BEAM-773) Implement Metrics support for Flink runner

[ https://issues.apache.org/jira/browse/BEAM-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941542#comment-15941542 ] Jingsong Lee commented on BEAM-773: --- Best wishes for your vacation! Do you think Metrics is necessary to

[jira] [Commented] (BEAM-1727) Add setForNowAlign(period, offset) to Timer

[ https://issues.apache.org/jira/browse/BEAM-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925942#comment-15925942 ] Jingsong Lee commented on BEAM-1727: What do you think? [~kenn] > Add setForNowAlign(period, offset)

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931210#comment-15931210 ] Jingsong Lee commented on BEAM-1612: [~aljoscha] Yes, this will greatly improve performance in some

[jira] [Assigned] (BEAM-1727) Add setForNowAlign(period, offset) to Timer

[ https://issues.apache.org/jira/browse/BEAM-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1727: -- Assignee: Jingsong Lee > Add setForNowAlign(period, offset) to Timer >

[jira] [Updated] (BEAM-1727) Add setForNowAlign(period, offset) to Timer

[ https://issues.apache.org/jira/browse/BEAM-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1727: --- Fix Version/s: First stable release > Add setForNowAlign(period, offset) to Timer >

[jira] [Created] (BEAM-1727) Add setForNowAlign(period, offset) to Timer

Jingsong Lee created BEAM-1727: -- Summary: Add setForNowAlign(period, offset) to Timer Key: BEAM-1727 URL: https://issues.apache.org/jira/browse/BEAM-1727 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-1727) Add setForNowAlign(period, offset) to Timer

[ https://issues.apache.org/jira/browse/BEAM-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929606#comment-15929606 ] Jingsong Lee commented on BEAM-1727: Can we return true or false in verifyTargetTime to let the user

[jira] [Commented] (BEAM-1723) FlinkRunner should deduplicate when an UnboundedSource requires Deduping

[ https://issues.apache.org/jira/browse/BEAM-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962125#comment-15962125 ] Jingsong Lee commented on BEAM-1723: I think it is necessary to be configurable because the

[jira] [Commented] (BEAM-1723) FlinkRunner should deduplicate when an UnboundedSource requires Deduping

[ https://issues.apache.org/jira/browse/BEAM-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962077#comment-15962077 ] Jingsong Lee commented on BEAM-1723: I understand. The reason for the duplication is that {{PubSubIO}}

[jira] [Created] (BEAM-1862) SplittableDoFnOperator should close the ScheduledExecutorService

Jingsong Lee created BEAM-1862: -- Summary: SplittableDoFnOperator should close the ScheduledExecutorService Key: BEAM-1862 URL: https://issues.apache.org/jira/browse/BEAM-1862 Project: Beam

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955360#comment-15955360 ] Jingsong Lee commented on BEAM-1612: I think this is a good idea. No matter how small it is better than

[jira] [Created] (BEAM-1941) Add Source Watermark Metrics in Runners

Jingsong Lee created BEAM-1941: -- Summary: Add Source Watermark Metrics in Runners Key: BEAM-1941 URL: https://issues.apache.org/jira/browse/BEAM-1941 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-1942) Add Source Watermark Metrics in Flink Runner

Jingsong Lee created BEAM-1942: -- Summary: Add Source Watermark Metrics in Flink Runner Key: BEAM-1942 URL: https://issues.apache.org/jira/browse/BEAM-1942 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-1945) Add Source Watermark Metrics in Apex runner

Jingsong Lee created BEAM-1945: -- Summary: Add Source Watermark Metrics in Apex runner Key: BEAM-1945 URL: https://issues.apache.org/jira/browse/BEAM-1945 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-1943) Add Source Watermark Metrics in Dataflow runner

Jingsong Lee created BEAM-1943: -- Summary: Add Source Watermark Metrics in Dataflow runner Key: BEAM-1943 URL: https://issues.apache.org/jira/browse/BEAM-1943 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-1944) Add Source Watermark Metrics in Spark runner

Jingsong Lee created BEAM-1944: -- Summary: Add Source Watermark Metrics in Spark runner Key: BEAM-1944 URL: https://issues.apache.org/jira/browse/BEAM-1944 Project: Beam Issue Type: Sub-task

[jira] [Commented] (BEAM-1723) FlinkRunner should deduplicate when an UnboundedSource requires Deduping

[ https://issues.apache.org/jira/browse/BEAM-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959226#comment-15959226 ] Jingsong Lee commented on BEAM-1723: I see {{CachedIdDeduplicator}} in direct runner. It use

[jira] [Assigned] (BEAM-1723) FlinkRunner should deduplicate when an UnboundedSource requires Deduping

[ https://issues.apache.org/jira/browse/BEAM-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1723: -- Assignee: Jingsong Lee > FlinkRunner should deduplicate when an UnboundedSource requires

[jira] [Assigned] (BEAM-1886) Remove TextIO override in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1886: -- Assignee: Jingsong Lee > Remove TextIO override in Flink runner >

[jira] [Assigned] (BEAM-1942) Add Source Watermark Metrics in Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1942: -- Assignee: Jingsong Lee > Add Source Watermark Metrics in Flink Runner >

[jira] [Assigned] (BEAM-1941) Add Source Watermark Metrics in Runners

[ https://issues.apache.org/jira/browse/BEAM-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1941: -- Assignee: Jingsong Lee > Add Source Watermark Metrics in Runners >

[jira] [Assigned] (BEAM-1944) Add Source Watermark Metrics in Spark runner

[ https://issues.apache.org/jira/browse/BEAM-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1944: -- Assignee: Jingsong Lee > Add Source Watermark Metrics in Spark runner >

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894699#comment-15894699 ] Jingsong Lee commented on BEAM-1612: https://issues.apache.org/jira/browse/FLINK-2846 Does this issue

[jira] [Created] (BEAM-1612) Support real Bundle in Flink runner

Jingsong Lee created BEAM-1612: -- Summary: Support real Bundle in Flink runner Key: BEAM-1612 URL: https://issues.apache.org/jira/browse/BEAM-1612 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893814#comment-15893814 ] Jingsong Lee commented on BEAM-1612: What do you think [~aljoscha] ? > Support real Bundle in Flink

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893827#comment-15893827 ] Jingsong Lee commented on BEAM-1612: In a bundle, we can reuse ReduceFnRunner simply too. We do not

[jira] [Commented] (BEAM-1531) Support dynamic work rebalancing for HBaseIO

[ https://issues.apache.org/jira/browse/BEAM-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067629#comment-16067629 ] Jingsong Lee commented on BEAM-1531: Of course, the embedded HBase server version is better, it is a

[jira] [Commented] (BEAM-1531) Support dynamic work rebalancing for HBaseIO

[ https://issues.apache.org/jira/browse/BEAM-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067961#comment-16067961 ] Jingsong Lee commented on BEAM-1531: (y) It's Great! > Support dynamic work rebalancing for HBaseIO >

[jira] [Commented] (BEAM-2393) BoundedSource is not fault-tolerant in FlinkRunner Streaming mode

[ https://issues.apache.org/jira/browse/BEAM-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066882#comment-16066882 ] Jingsong Lee commented on BEAM-2393: Let's talk about it in BEAM-1531. > BoundedSource is not

[jira] [Commented] (BEAM-1531) Support dynamic work rebalancing for HBaseIO

[ https://issues.apache.org/jira/browse/BEAM-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066883#comment-16066883 ] Jingsong Lee commented on BEAM-1531: Hi [~iemejia]l, at BEAM-2393 we talked about accelerating

[jira] [Commented] (BEAM-2534) KafkaIO should allow gaps in message offsets

[ https://issues.apache.org/jira/browse/BEAM-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076928#comment-16076928 ] Jingsong Lee commented on BEAM-2534: Could be cherry-picked to 2.1.0 branch? > KafkaIO should allow

[jira] [Commented] (BEAM-1641) Support synchronized processing time in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988363#comment-15988363 ] Jingsong Lee commented on BEAM-1641: I think it is difficult to achieve, it requires Flink-runtime

[jira] [Commented] (BEAM-1641) Support synchronized processing time in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992110#comment-15992110 ] Jingsong Lee commented on BEAM-1641: There are some differences between the processing of event time

[jira] [Created] (BEAM-2248) KafkaIO support to use start read time to set start offset

Jingsong Lee created BEAM-2248: -- Summary: KafkaIO support to use start read time to set start offset Key: BEAM-2248 URL: https://issues.apache.org/jira/browse/BEAM-2248 Project: Beam Issue

[jira] [Commented] (BEAM-2140) Fix SplittableDoFn ValidatesRunner tests in FlinkRunner

[ https://issues.apache.org/jira/browse/BEAM-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003002#comment-16003002 ] Jingsong Lee commented on BEAM-2140: 1. {{SplittableParDo}} will clear its State, in

[jira] [Commented] (BEAM-2140) Fix SplittableDoFn ValidatesRunner tests in FlinkRunner

[ https://issues.apache.org/jira/browse/BEAM-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002039#comment-16002039 ] Jingsong Lee commented on BEAM-2140: First, {{SplittableParDo}} should not wrap {{StatefulDoFnRunner}}.

[jira] [Updated] (BEAM-1942) Add Watermark Metrics in Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1942: --- Summary: Add Watermark Metrics in Flink Runner (was: Add Source Watermark Metrics in Flink Runner) >

[jira] [Updated] (BEAM-1945) Add Watermark Metrics in Apex runner

[ https://issues.apache.org/jira/browse/BEAM-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1945: --- Summary: Add Watermark Metrics in Apex runner (was: Add Source Watermark Metrics in Apex runner) >

[jira] [Updated] (BEAM-1941) Add Watermark Metrics in Runners

[ https://issues.apache.org/jira/browse/BEAM-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1941: --- Summary: Add Watermark Metrics in Runners (was: Add Source Watermark Metrics in Runners) > Add

[jira] [Updated] (BEAM-1943) Add Watermark Metrics in Dataflow runner

[ https://issues.apache.org/jira/browse/BEAM-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1943: --- Summary: Add Watermark Metrics in Dataflow runner (was: Add Source Watermark Metrics in Dataflow

[jira] [Updated] (BEAM-1944) Add Watermark Metrics in Spark runner

[ https://issues.apache.org/jira/browse/BEAM-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1944: --- Summary: Add Watermark Metrics in Spark runner (was: Add Source Watermark Metrics in Spark runner) >

[jira] [Created] (BEAM-2486) Should throws some useful messages when statefulParDo use non-KV input

Jingsong Lee created BEAM-2486: -- Summary: Should throws some useful messages when statefulParDo use non-KV input Key: BEAM-2486 URL: https://issues.apache.org/jira/browse/BEAM-2486 Project: Beam

[jira] [Created] (BEAM-2487) Give a option to ignore the timer that is larger than END_OF_GLOBAL_WINDOW

Jingsong Lee created BEAM-2487: -- Summary: Give a option to ignore the timer that is larger than END_OF_GLOBAL_WINDOW Key: BEAM-2487 URL: https://issues.apache.org/jira/browse/BEAM-2487 Project: Beam

[jira] [Commented] (BEAM-2486) Should throws some useful messages when statefulParDo use non-KV input

[ https://issues.apache.org/jira/browse/BEAM-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056877#comment-16056877 ] Jingsong Lee commented on BEAM-2486: [~kenn] Can we do some validates before runner? > Should throws

[jira] [Created] (BEAM-2477) BeamAggregationRel should use Combine.perKey instead of GroupByKey

Jingsong Lee created BEAM-2477: -- Summary: BeamAggregationRel should use Combine.perKey instead of GroupByKey Key: BEAM-2477 URL: https://issues.apache.org/jira/browse/BEAM-2477 Project: Beam

[jira] [Created] (BEAM-2478) Distinct Aggregates

Jingsong Lee created BEAM-2478: -- Summary: Distinct Aggregates Key: BEAM-2478 URL: https://issues.apache.org/jira/browse/BEAM-2478 Project: Beam Issue Type: New Feature Components:

[jira] [Commented] (BEAM-2477) BeamAggregationRel should use Combine.perKey instead of GroupByKey

[ https://issues.apache.org/jira/browse/BEAM-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055416#comment-16055416 ] Jingsong Lee commented on BEAM-2477: *Local combine*: Cloud Dataflow/Flink Batch optimizes Combine

[jira] [Updated] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1612: --- Fix Version/s: 2.1.0 > Support real Bundle in Flink runner > --- > >

[jira] [Created] (BEAM-2508) Fix javaDoc of Stateful DoFn

Jingsong Lee created BEAM-2508: -- Summary: Fix javaDoc of Stateful DoFn Key: BEAM-2508 URL: https://issues.apache.org/jira/browse/BEAM-2508 Project: Beam Issue Type: Bug Components:

[jira] [Updated] (BEAM-2508) Fix javaDoc of Stateful DoFn

[ https://issues.apache.org/jira/browse/BEAM-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-2508: --- Description: StateSpec > StateSpec (was: StateSpec

[jira] [Updated] (BEAM-2508) Fix javaDoc of Stateful DoFn

[ https://issues.apache.org/jira/browse/BEAM-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-2508: --- Fix Version/s: 2.1.0 > Fix javaDoc of Stateful DoFn > > >

[jira] [Commented] (BEAM-2478) Distinct Aggregates

[ https://issues.apache.org/jira/browse/BEAM-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060353#comment-16060353 ] Jingsong Lee commented on BEAM-2478: Count(Distinct) is a very interesting function. It needs operator

[jira] [Commented] (BEAM-2140) Fix SplittableDoFn ValidatesRunner tests in FlinkRunner

[ https://issues.apache.org/jira/browse/BEAM-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064090#comment-16064090 ] Jingsong Lee commented on BEAM-2140: On 1: If the decision whether the window expired is output

[jira] [Commented] (BEAM-2478) Distinct Aggregates

[ https://issues.apache.org/jira/browse/BEAM-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062468#comment-16062468 ] Jingsong Lee commented on BEAM-2478: (y) You are right. Calcite native offers a lot of useful Rules,

[jira] [Resolved] (BEAM-2477) BeamAggregationRel should use Combine.perKey instead of GroupByKey

[ https://issues.apache.org/jira/browse/BEAM-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee resolved BEAM-2477. Resolution: Fixed Fix Version/s: Not applicable > BeamAggregationRel should use

[jira] [Closed] (BEAM-2477) BeamAggregationRel should use Combine.perKey instead of GroupByKey

[ https://issues.apache.org/jira/browse/BEAM-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee closed BEAM-2477. -- > BeamAggregationRel should use Combine.perKey instead of GroupByKey >

[jira] [Closed] (BEAM-2248) KafkaIO support to use start read time to set start offset

[ https://issues.apache.org/jira/browse/BEAM-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee closed BEAM-2248. -- Resolution: Fixed Fix Version/s: 2.1.0 > KafkaIO support to use start read time to set start

[jira] [Created] (BEAM-2393) BoundedSource is not fault-tolerant in FlinkRunner Streaming mode

Jingsong Lee created BEAM-2393: -- Summary: BoundedSource is not fault-tolerant in FlinkRunner Streaming mode Key: BEAM-2393 URL: https://issues.apache.org/jira/browse/BEAM-2393 Project: Beam

[jira] [Created] (BEAM-2401) Update Flink Runner to Flink 1.3.0

Jingsong Lee created BEAM-2401: -- Summary: Update Flink Runner to Flink 1.3.0 Key: BEAM-2401 URL: https://issues.apache.org/jira/browse/BEAM-2401 Project: Beam Issue Type: Improvement

[jira] [Assigned] (BEAM-1476) Support MapState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1476: -- Assignee: Jingsong Lee > Support MapState in Flink runner > >

[jira] [Assigned] (BEAM-1483) Support SetState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1483: -- Assignee: Jingsong Lee > Support SetState in Flink runner > >

[jira] [Assigned] (BEAM-1498) Use Flink-native side outputs

[ https://issues.apache.org/jira/browse/BEAM-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1498: -- Assignee: Jingsong Lee > Use Flink-native side outputs > - > >

[jira] [Created] (BEAM-2423) Abstract StateInternalsTest for the different state internals/Runners

Jingsong Lee created BEAM-2423: -- Summary: Abstract StateInternalsTest for the different state internals/Runners Key: BEAM-2423 URL: https://issues.apache.org/jira/browse/BEAM-2423 Project: Beam

[jira] [Assigned] (BEAM-1476) Support MapState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee reassigned BEAM-1476: -- Assignee: Pei He (was: Jingsong Lee) > Support MapState in Flink runner >

[jira] [Commented] (BEAM-1476) Support MapState in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040044#comment-16040044 ] Jingsong Lee commented on BEAM-1476: solved in https://github.com/apache/beam/pull/3289 > Support

[jira] [Commented] (BEAM-1612) Support real Bundle in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996603#comment-15996603 ] Jingsong Lee commented on BEAM-1612: Agree with [~aljoscha]. About the course grained, A little idea

[jira] [Updated] (BEAM-1641) Support synchronized processing time in Flink runner

[ https://issues.apache.org/jira/browse/BEAM-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingsong Lee updated BEAM-1641: --- Fix Version/s: (was: First stable release) > Support synchronized processing time in Flink runner

  1   2   >