[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-06-29 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-869760121 cc @nsivabalan @codope do any of you have cycles to test this PR out on top of s3 and see if any perf improvements happen (My guess is no). -- This is an automated message

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-06-28 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-869760121 cc @nsivabalan @codope do any of you have cycles to test this PR out on top of s3 and see if any perf improvements happen (My guess is no). -- This is an automated message

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-05-25 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-848039122 I have not been able to test this on S3. let me pick it up later next week. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-05-11 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-838065811 @prashantwason ping! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-03-02 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-789459406 @prashantwason I rebased this against master. still have some test failures. could you please take a look, so we can land this --

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-02-17 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-780489181 @prashantwason can we get the PR to pass tests? I can take a final pass for landing. it'd be good to get this in ---

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-01-29 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-770018126 Once we fix CI and the minor stuff, we can land This is an automated message from the Apache Git Service. To

[GitHub] [hudi] vinothchandar commented on pull request #2496: [HUDI-1554] Introduced buffering for streams in HUDI.

2021-01-29 Thread GitBox
vinothchandar commented on pull request #2496: URL: https://github.com/apache/hudi/pull/2496#issuecomment-770011435 cc @umehrot2 would this additional buffering pose inefficiencies for S3 FileSystem? TL;DR HDFS's `DistributedFileSystem` does not buffer reads, neither does the parquet reade