Github user glaksh100 commented on a diff in the pull request:
https://github.com/apache/flink/pull/5860#discussion_r186223099
--- Diff:
flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java
---
@@ -908,6
Github user glaksh100 commented on a diff in the pull request:
https://github.com/apache/flink/pull/5860#discussion_r186223237
--- Diff:
flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java
---
@@ -473,6
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/5860
Thanks for reviewing @fhueske @aljoscha and @kl0u ! I have addressed the
latest review comments. Can you PTAL (again) ?
---
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/5860
@fhueske I gave it some thought and your suggestion makes sense to me. I
have extended `checkForInactiveBuckets` to include the rollover check. I have
also updated Javadocs in a few places
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/5860
@fhueske Can you PTAL and merge this PR?
---
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/6300
@StephanEwen I have run this on the following set up:
```
Number of shards on Kinesis stream: 384
Number of task slots: 384 / 192 / 96
Throughput achieved per shard (with adaptive
GitHub user glaksh100 opened a pull request:
https://github.com/apache/flink/pull/6408
[FLINK-9897] Make adaptive reads depend on run loop time instead of
fetchintervalmillis
## What is the purpose of the change
[FLINK-9692](https://github.com/apache/flink/pull/6300) introduced
GitHub user glaksh100 opened a pull request:
https://github.com/apache/flink/pull/6409
Flink 9899.kinesis connector metrics
## What is the purpose of the change
The purpose of this change is to add metrics to the `ShardConsumer` to get
more observability
Github user glaksh100 commented on a diff in the pull request:
https://github.com/apache/flink/pull/6408#discussion_r204942574
--- Diff:
flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java
GitHub user glaksh100 opened a pull request:
https://github.com/apache/flink/pull/6300
[FLINK-9692] Adaptive reads from Kinesis
## What is the purpose of the change
The purpose of this change is to provide an option to the Kinesis connector
to optimize the amount of data
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/6300
@fhueske @tzulitai @tweise Can you please take a look when you have a
chance?
---
Github user glaksh100 commented on a diff in the pull request:
https://github.com/apache/flink/pull/6300#discussion_r202156901
--- Diff:
flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java
---
@@ -330,4
Github user glaksh100 commented on a diff in the pull request:
https://github.com/apache/flink/pull/6300#discussion_r202227845
--- Diff:
flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java
---
@@ -330,4
Github user glaksh100 commented on a diff in the pull request:
https://github.com/apache/flink/pull/6300#discussion_r202227834
--- Diff:
flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/config/ConsumerConfigConstants.java
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/6300
The idea here is that `maxNumberOfRecordsPerFetch` should never be a value
that gets records that exceeds the read limit (2 Mb/sec) from the math here.
```
2 Mb/sec
GitHub user glaksh100 opened a pull request:
https://github.com/apache/flink/pull/5860
[FLINK-9138][filesystem-connectors] Implement time based rollover in
BucketingSink
## What is the purpose of the change
This pull request enables a time-based rollover of the part file
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/5860
@aljoscha @fhueske Can you please take a look?
---
Github user glaksh100 commented on the issue:
https://github.com/apache/flink/pull/5860
@fhueske Thank you for reviewing. I have incorporated the changes that
include update to the
[documentation](https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors
18 matches
Mail list logo
