[jira] [Closed] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-10-01 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi closed BEAM-4038. -- Resolution: Fixed Fix Version/s: Not applicable > Support Kafka Headers in KafkaIO >

[jira] [Commented] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-10-01 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634641#comment-16634641 ] Raghu Angadi commented on BEAM-4038: Sounds good. I will close this for now. We can ad

[jira] [Updated] (BEAM-5375) KafkaIO reader should handle runtime exceptions kafka client

2018-09-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-5375: --- Description: KafkaIO reader might stop reading from Kafka without any explicit error message if Kafka

[jira] [Commented] (BEAM-5408) (Java) Using Compression.GZIP with TFRecordIO

2018-09-17 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618351#comment-16618351 ] Raghu Angadi commented on BEAM-5408: The root cause for this is described in BEAM-5412

[jira] [Assigned] (BEAM-5412) TFRecordIO fails with records larger than 8K

2018-09-17 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-5412: -- Assignee: Chamikara Jayalath (was: Eugene Kirpichov) > TFRecordIO fails with records larger th

[jira] [Created] (BEAM-5412) TFRecordIO fails with records larger than 8K

2018-09-17 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-5412: -- Summary: TFRecordIO fails with records larger than 8K Key: BEAM-5412 URL: https://issues.apache.org/jira/browse/BEAM-5412 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-5375) KafkaIO reader should handle runtime exceptions kafka client

2018-09-12 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-5375: -- Summary: KafkaIO reader should handle runtime exceptions kafka client Key: BEAM-5375 URL: https://issues.apache.org/jira/browse/BEAM-5375 Project: Beam Issue Ty

[jira] [Commented] (BEAM-4803) Beam spark runner not working properly with kafka

2018-08-10 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576566#comment-16576566 ] Raghu Angadi commented on BEAM-4803: It is not configurable. It was chosen mainly as a

[jira] [Commented] (BEAM-4803) Beam spark runner not working properly with kafka

2018-08-08 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573312#comment-16573312 ] Raghu Angadi commented on BEAM-4803: FYI, Dataflow applies an inactivity timeout of 1

[jira] [Commented] (BEAM-4803) Beam spark runner not working properly with kafka

2018-07-17 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547075#comment-16547075 ] Raghu Angadi commented on BEAM-4803: [~aromanenko], you have looked into Spark runner

[jira] [Assigned] (BEAM-4803) Beam spark runner not working properly with kafka

2018-07-17 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4803: -- Assignee: Alexey Romanenko (was: Raghu Angadi) > Beam spark runner not working properly with k

[jira] [Commented] (BEAM-4192) Pubsub benchmarks in Nexmark

2018-07-13 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543462#comment-16543462 ] Raghu Angadi commented on BEAM-4192: Thanks. This is not urgent. For now I have remove

[jira] [Updated] (BEAM-4192) Pubsub benchmarks in Nexmark

2018-07-13 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-4192: --- Fix Version/s: (was: 2.7.0) > Pubsub benchmarks in Nexmark > > >

[jira] [Commented] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-07-12 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541977#comment-16541977 ] Raghu Angadi commented on BEAM-4038: It is not a blocker. I removed 'Fix Version'. The

[jira] [Updated] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-07-12 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-4038: --- Fix Version/s: (was: 2.6.0) > Support Kafka Headers in KafkaIO >

[jira] [Commented] (BEAM-4632) kafkIO should be the streaming mode over spark runner

2018-06-25 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522850#comment-16522850 ] Raghu Angadi commented on BEAM-4632: cc [~amitsela]. > kafkIO should be the streaming

[jira] [Commented] (BEAM-4632) kafkIO should be the streaming mode over spark runner

2018-06-25 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522533#comment-16522533 ] Raghu Angadi commented on BEAM-4632: Hi [~aromanenko], I am assigning this to you. Sin

[jira] [Assigned] (BEAM-4632) kafkIO should be the streaming mode over spark runner

2018-06-25 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4632: -- Assignee: Alexey Romanenko (was: Raghu Angadi) > kafkIO should be the streaming mode over spar

[jira] [Resolved] (BEAM-4631) kafkIO should run the streaming mode over spark runner

2018-06-25 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-4631. Resolution: Duplicate Filed twice by mistake. Closing this in favor of it's clone BEAM-4632. > kafk

[jira] [Commented] (BEAM-4521) Backlog metrics not showing up

2018-06-08 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506228#comment-16506228 ] Raghu Angadi commented on BEAM-4521: This is really a flink bug. All the Beam API meth

[jira] [Assigned] (BEAM-4521) Backlog metrics not showing up

2018-06-08 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4521: -- Assignee: Aljoscha Krettek (was: Raghu Angadi) > Backlog metrics not showing up >

[jira] [Updated] (BEAM-4521) Backlog metrics not showing up

2018-06-08 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-4521: --- Component/s: (was: io-java-kafka) runner-flink > Backlog metrics not showing up >

[jira] [Commented] (BEAM-4485) Incredibly difficult to use KakfaIO + TLS + DataflowRunner

2018-06-05 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502336#comment-16502336 ] Raghu Angadi commented on BEAM-4485: Fortunately each of the two problems you mentione

[jira] [Resolved] (BEAM-4485) Incredibly difficult to use KakfaIO + TLS + DataflowRunner

2018-06-05 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-4485. Resolution: Not A Problem Fix Version/s: Not applicable > Incredibly difficult to use KakfaIO

[jira] [Commented] (BEAM-2980) BagState.isEmpty needs a tighter spec

2018-05-23 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488425#comment-16488425 ] Raghu Angadi commented on BEAM-2980: Is there a pointer to rationale for existence of {

[jira] [Created] (BEAM-4372) Need an undeprecated Reshuffle transform

2018-05-21 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-4372: -- Summary: Need an undeprecated Reshuffle transform Key: BEAM-4372 URL: https://issues.apache.org/jira/browse/BEAM-4372 Project: Beam Issue Type: Bug Com

[jira] [Commented] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-04-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459481#comment-16459481 ] Raghu Angadi commented on BEAM-4038: In Kafka {{ConsumerRecord}} and {{ProducerRecord}}

[jira] [Created] (BEAM-4192) Pubsub benchmarks in Nexmark

2018-04-30 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-4192: -- Summary: Pubsub benchmarks in Nexmark Key: BEAM-4192 URL: https://issues.apache.org/jira/browse/BEAM-4192 Project: Beam Issue Type: Improvement Compone

[jira] [Updated] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-04-29 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-4038: --- Priority: Major (was: Minor) > Support Kafka Headers in KafkaIO > > >

[jira] [Commented] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-04-29 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458325#comment-16458325 ] Raghu Angadi commented on BEAM-4038: The interfaces with function call backs are proble

[jira] [Resolved] (BEAM-3851) Support element timestamps while publishing to Kafka.

2018-04-29 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-3851. Resolution: Fixed > Support element timestamps while publishing to Kafka. > -

[jira] [Resolved] (BEAM-591) Better handling of watermark in KafkaIO

2018-04-29 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-591. --- Resolution: Fixed Fix Version/s: 2.4.0 > Better handling of watermark in KafkaIO > --

[jira] [Commented] (BEAM-591) Better handling of watermark in KafkaIO

2018-04-29 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458307#comment-16458307 ] Raghu Angadi commented on BEAM-591: --- See following methods added to `KafkaIO.Read` in 3 pu

[jira] [Commented] (BEAM-4086) KafkaIOTest is flaky

2018-04-19 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1604#comment-1604 ] Raghu Angadi commented on BEAM-4086: {{testUnboundedSourceWithoutBoundedWrapper()}} is

[jira] [Commented] (BEAM-4086) KafkaIOTest is flaky

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443143#comment-16443143 ] Raghu Angadi commented on BEAM-4086: Nm. I was running gradle. It did finally run into

[jira] [Assigned] (BEAM-4086) KafkaIOTest is flaky

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4086: -- Assignee: Raghu Angadi > KafkaIOTest is flaky > > > Key: BE

[jira] [Commented] (BEAM-4086) KafkaIO is flaky with gradle build

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442915#comment-16442915 ] Raghu Angadi commented on BEAM-4086: [~iemejia], assigning it to you since it happened

[jira] [Assigned] (BEAM-4086) KafkaIO is flaky with gradle build

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4086: -- Assignee: Ismaël Mejía (was: Alexey Romanenko) > KafkaIO is flaky with gradle build > -

[jira] [Assigned] (BEAM-4086) KafkaIO is flaky with gradle build

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4086: -- Assignee: Alexey Romanenko (was: Raghu Angadi) > KafkaIO is flaky with gradle build > -

[jira] [Commented] (BEAM-4086) KafkaIO is flaky with gradle build

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442873#comment-16442873 ] Raghu Angadi commented on BEAM-4086: [~aromanenko], I am not able to reproduce it on my

[jira] [Assigned] (BEAM-4086) KafkaIO is flaky with gradle build

2018-04-18 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-4086: -- Assignee: Raghu Angadi > KafkaIO is flaky with gradle build > --

[jira] [Commented] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-04-11 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434033#comment-16434033 ] Raghu Angadi commented on BEAM-4038: Thanks. Will be happy to review. We need to make s

[jira] [Commented] (BEAM-4038) Support Kafka Headers in KafkaIO

2018-04-10 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432658#comment-16432658 ] Raghu Angadi commented on BEAM-4038: Reader support * I think this more critical and t

[jira] [Resolved] (BEAM-3703) java.io.IOException: KafkaWriter : failed to send 1 records (since last report)

2018-03-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-3703. Resolution: Not A Bug > java.io.IOException: KafkaWriter : failed to send 1 records (since last > re

[jira] [Commented] (BEAM-3703) java.io.IOException: KafkaWriter : failed to send 1 records (since last report)

2018-03-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400786#comment-16400786 ] Raghu Angadi commented on BEAM-3703: [~jsihota], most likely this is a network ACL issu

[jira] [Created] (BEAM-3851) Support element timestamps while publishing to Kafka.

2018-03-14 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-3851: -- Summary: Support element timestamps while publishing to Kafka. Key: BEAM-3851 URL: https://issues.apache.org/jira/browse/BEAM-3851 Project: Beam Issue Type: Impr

[jira] [Commented] (BEAM-3718) ClassNotFoundException on CloudResourceManager$Builder

2018-03-06 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388279#comment-16388279 ] Raghu Angadi commented on BEAM-3718: Assigned it to [~chamikara] for triaging. Looks li

[jira] [Assigned] (BEAM-3718) ClassNotFoundException on CloudResourceManager$Builder

2018-03-06 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-3718: -- Assignee: Chamikara Jayalath (was: Raghu Angadi) > ClassNotFoundException on CloudResourceManag

[jira] [Resolved] (BEAM-3705) ApproximateUnique discards accumulated data with multiple firings.

2018-03-05 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-3705. Resolution: Fixed Fix Version/s: 2.4.0 > ApproximateUnique discards accumulated data with mult

[jira] [Resolved] (BEAM-3754) KAFKA - Can't set commitOffsetsInFinalizeEnabled to false with KafkaIO.readBytes()

2018-03-05 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-3754. Resolution: Fixed Fix Version/s: 2.4.0 > KAFKA - Can't set commitOffsetsInFinalizeEnabled to f

[jira] [Commented] (BEAM-3754) KAFKA - Can't set commitOffsetsInFinalizeEnabled to false with KafkaIO.readBytes()

2018-03-02 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384330#comment-16384330 ] Raghu Angadi commented on BEAM-3754: Just saw this. You are correct. Thanks for reporti

[jira] [Resolved] (BEAM-3770) The problem of kafkaIO sdk for data latency

2018-03-02 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-3770. Resolution: Not A Bug > The problem of kafkaIO sdk for data latency > ---

[jira] [Commented] (BEAM-3770) The problem of kafkaIO sdk for data latency

2018-03-02 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384259#comment-16384259 ] Raghu Angadi commented on BEAM-3770: This is being discussed on user mailing list : [h

[jira] [Created] (BEAM-3705) ApproximateUnique discards accumulated data with multiple firings.

2018-02-14 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-3705: -- Summary: ApproximateUnique discards accumulated data with multiple firings. Key: BEAM-3705 URL: https://issues.apache.org/jira/browse/BEAM-3705 Project: Beam I

[jira] [Updated] (BEAM-3689) Direct runner leak a reader for every 10 input records

2018-02-11 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-3689: --- Fix Version/s: 2.4.0 > Direct runner leak a reader for every 10 input records > ---

[jira] [Created] (BEAM-3689) Direct runner leak a reader for every 10 input records

2018-02-11 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-3689: -- Summary: Direct runner leak a reader for every 10 input records Key: BEAM-3689 URL: https://issues.apache.org/jira/browse/BEAM-3689 Project: Beam Issue Type: Imp

[jira] [Assigned] (BEAM-3689) Direct runner leak a reader for every 10 input records

2018-02-11 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-3689: -- Assignee: Raghu Angadi (was: Thomas Groh) > Direct runner leak a reader for every 10 input reco

[jira] [Created] (BEAM-3620) Deprecate older Kafka clients

2018-02-05 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-3620: -- Summary: Deprecate older Kafka clients Key: BEAM-3620 URL: https://issues.apache.org/jira/browse/BEAM-3620 Project: Beam Issue Type: Improvement Compon

[jira] [Updated] (BEAM-3526) Support for checkpointMark finalize in KafkaIO

2018-02-05 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-3526: --- Fix Version/s: 2.4.0 > Support for checkpointMark finalize in KafkaIO > ---

[jira] [Closed] (BEAM-3526) Support for checkpointMark finalize in KafkaIO

2018-02-05 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi closed BEAM-3526. -- Resolution: Fixed > Support for checkpointMark finalize in KafkaIO >

[jira] [Created] (BEAM-3611) Split KafkaIO.java into smaller files

2018-02-02 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-3611: -- Summary: Split KafkaIO.java into smaller files Key: BEAM-3611 URL: https://issues.apache.org/jira/browse/BEAM-3611 Project: Beam Issue Type: Improvement

[jira] [Assigned] (BEAM-3611) Split KafkaIO.java into smaller files

2018-02-02 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-3611: -- Assignee: Raghu Angadi (was: Reuven Lax) > Split KafkaIO.java into smaller files >

[jira] [Updated] (BEAM-3526) Support for checkpointMark finalize in KafkaIO

2018-01-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-3526: --- Affects Version/s: 2.2.0 > Support for checkpointMark finalize in KafkaIO > ---

[jira] [Created] (BEAM-3526) Support for checkpointMark finalize in KafkaIO

2018-01-24 Thread Raghu Angadi (JIRA)
Raghu Angadi created BEAM-3526: -- Summary: Support for checkpointMark finalize in KafkaIO Key: BEAM-3526 URL: https://issues.apache.org/jira/browse/BEAM-3526 Project: Beam Issue Type: Bug

[jira] [Assigned] (BEAM-3526) Support for checkpointMark finalize in KafkaIO

2018-01-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-3526: -- Assignee: Raghu Angadi (was: Reuven Lax) > Support for checkpointMark finalize in KafkaIO > ---

[jira] [Closed] (BEAM-2257) KafkaIO write without key requires a producer fn

2018-01-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi closed BEAM-2257. -- Resolution: Fixed > KafkaIO write without key requires a producer fn > --

[jira] [Closed] (BEAM-2704) KafkaIO: NPE without key serializer set

2018-01-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi closed BEAM-2704. -- Resolution: Fixed Fix Version/s: 2.3.0 > KafkaIO: NPE without key serializer set > ---

[jira] [Commented] (BEAM-2704) KafkaIO: NPE without key serializer set

2018-01-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338195#comment-16338195 ] Raghu Angadi commented on BEAM-2704: Resolving this as  BEAM-2257 is fixed (it still ne

[jira] [Comment Edited] (BEAM-3093) add an option 'FirstPollOffsetStrategy' to KafkaIO

2017-11-27 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268168#comment-16268168 ] Raghu Angadi edited comment on BEAM-3093 at 11/28/17 5:52 AM: --

[jira] [Commented] (BEAM-3093) add an option 'FirstPollOffsetStrategy' to KafkaIO

2017-11-27 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268168#comment-16268168 ] Raghu Angadi commented on BEAM-3093: [~mingmxu], assigning this to you. Let me know `wi

[jira] [Assigned] (BEAM-3093) add an option 'FirstPollOffsetStrategy' to KafkaIO

2017-11-27 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-3093: -- Assignee: Xu Mingmin (was: Raghu Angadi) > add an option 'FirstPollOffsetStrategy' to KafkaIO >

[jira] [Commented] (BEAM-3218) Change PubsubBoundedWriter to check total byte size

2017-11-21 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261488#comment-16261488 ] Raghu Angadi commented on BEAM-3218: Thanks [~thang], the quota enhancements also make

[jira] [Commented] (BEAM-2257) KafkaIO write without key requires a producer fn

2017-11-08 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244375#comment-16244375 ] Raghu Angadi commented on BEAM-2257: Thanks [~nerdynick] for the fix. [~jbonofre] pleas

[jira] [Updated] (BEAM-2257) KafkaIO write without key requires a producer fn

2017-11-08 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2257: --- Fix Version/s: 2.3.0 > KafkaIO write without key requires a producer fn > -

[jira] [Commented] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226010#comment-16226010 ] Raghu Angadi commented on BEAM-2979: It was marked a blocker by mistake. > Race condi

[jira] [Updated] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2979: --- Fix Version/s: (was: 2.2.0) 2.3.0 > Race condition between KafkaIO.UnboundedKafk

[jira] [Updated] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2979: --- Priority: Major (was: Blocker) > Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() an

[jira] [Commented] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225831#comment-16225831 ] Raghu Angadi commented on BEAM-2979: I think it is good to have. There is no work aroun

[jira] [Updated] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2979: --- Fix Version/s: 2.2.0 > Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and > KafkaI

[jira] [Commented] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225817#comment-16225817 ] Raghu Angadi commented on BEAM-2979: No, Cham is waiting on Jenkings before merging. I

[jira] [Updated] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2979: --- Fix Version/s: (was: 2.2.0) > Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() an

[jira] [Assigned] (BEAM-307) Upgrade/Test to Kafka 0.10

2017-10-30 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-307: - Assignee: Xu Mingmin (was: Jean-Baptiste Onofré) > Upgrade/Test to Kafka 0.10 > --

[jira] [Commented] (BEAM-1664) Support Kafka0.8.x client in KafkaIO

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221039#comment-16221039 ] Raghu Angadi commented on BEAM-1664: [~r7raul], there are no plans to support 0.8x. It

[jira] [Assigned] (BEAM-1664) Support Kafka0.8.x client in KafkaIO

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reassigned BEAM-1664: -- Assignee: Reuven Lax (was: Raghu Angadi) > Support Kafka0.8.x client in KafkaIO >

[jira] [Updated] (BEAM-2704) KafkaIO: NPE without key serializer set

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2704: --- Priority: Major (was: Blocker) > KafkaIO: NPE without key serializer set > ---

[jira] [Updated] (BEAM-2704) KafkaIO: NPE without key serializer set

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2704: --- Fix Version/s: (was: 2.2.0) > KafkaIO: NPE without key serializer set > ---

[jira] [Updated] (BEAM-2257) KafkaIO write without key requires a producer fn

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2257: --- Priority: Major (was: Blocker) > KafkaIO write without key requires a producer fn > --

[jira] [Updated] (BEAM-2257) KafkaIO write without key requires a producer fn

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2257: --- Fix Version/s: (was: 2.2.0) > KafkaIO write without key requires a producer fn > --

[jira] [Updated] (BEAM-2257) KafkaIO write without key requires a producer fn

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2257: --- Fix Version/s: 2.2.0 > KafkaIO write without key requires a producer fn > -

[jira] [Updated] (BEAM-2257) KafkaIO write without key requires a producer fn

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2257: --- Priority: Blocker (was: Major) > KafkaIO write without key requires a producer fn > --

[jira] [Updated] (BEAM-2704) KafkaIO: NPE without key serializer set

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2704: --- Priority: Blocker (was: Major) > KafkaIO: NPE without key serializer set > ---

[jira] [Updated] (BEAM-2704) KafkaIO: NPE without key serializer set

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2704: --- Fix Version/s: 2.2.0 > KafkaIO: NPE without key serializer set > --

[jira] [Updated] (BEAM-2703) KafkaIO: watermark outside the bounds of BoundedWindow

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2703: --- Fix Version/s: (was: 2.2.0) > KafkaIO: watermark outside the bounds of BoundedWindow >

[jira] [Updated] (BEAM-2703) KafkaIO: watermark outside the bounds of BoundedWindow

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2703: --- Priority: Blocker (was: Major) > KafkaIO: watermark outside the bounds of BoundedWindow >

[jira] [Updated] (BEAM-2703) KafkaIO: watermark outside the bounds of BoundedWindow

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2703: --- Priority: Major (was: Blocker) > KafkaIO: watermark outside the bounds of BoundedWindow >

[jira] [Updated] (BEAM-2703) KafkaIO: watermark outside the bounds of BoundedWindow

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2703: --- Fix Version/s: 2.2.0 > KafkaIO: watermark outside the bounds of BoundedWindow > ---

[jira] [Updated] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2979: --- Priority: Blocker (was: Major) > Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() an

[jira] [Updated] (BEAM-2979) Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and KafkaIO.UnboundedKafkaReader.advance()

2017-10-26 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-2979: --- Fix Version/s: 2.2.0 > Race condition between KafkaIO.UnboundedKafkaReader.getWatermark() and > KafkaI

[jira] [Commented] (BEAM-3093) add an option 'FirstPollOffsetStrategy' to KafkaIO

2017-10-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217318#comment-16217318 ] Raghu Angadi commented on BEAM-3093: Yes. withStartReadTime(). > add an option 'First

[jira] [Comment Edited] (BEAM-3093) add an option 'FirstPollOffsetStrategy' to KafkaIO

2017-10-24 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217240#comment-16217240 ] Raghu Angadi edited comment on BEAM-3093 at 10/24/17 4:54 PM: --

  1   2   >