[GitHub] [hudi] wangxianghu commented on pull request #1935: [HUDI-1121][DOC]Provide a document describing how to use callback

wangxianghu commented on pull request #1935: URL: https://github.com/apache/hudi/pull/1935#issuecomment-670828182 @yanghua @leesf please take a look when free This is an automated message from the Apache Git Service. To

[GitHub] [hudi] wangxianghu opened a new pull request #1935: [HUDI-1121][DOC]Provide a document describing how to use callback

wangxianghu opened a new pull request #1935: URL: https://github.com/apache/hudi/pull/1935 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[jira] [Updated] (HUDI-1121) Provide a document describing how to use callback

[ https://issues.apache.org/jira/browse/HUDI-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1121: - Labels: pull-request-available (was: ) > Provide a document describing how to use callback >

[GitHub] [hudi] bvaradar commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

bvaradar commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670826574 @umehrot2 : Can you confirm if all review comments are resolved and the PR is ready otherwise. This is an

[GitHub] [hudi] bvaradar commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

bvaradar commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670826438 @umehrot2 : Thanks for the update. Yeah, the integration test flakiness is a know issue and the logs shows the same pattern. Let me do one pass of it along with other bootstrap

[GitHub] [hudi] xushiyan opened a new pull request #1934: [MINOR] Move a test method to Transformations

xushiyan opened a new pull request #1934: URL: https://github.com/apache/hudi/pull/1934 - Move TestHoodieKeyLocationFetchHandle#getRecordsPerPartition to Transformations - Improve some var namings ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please

[GitHub] [hudi] leesf commented on pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

leesf commented on pull request #1931: URL: https://github.com/apache/hudi/pull/1931#issuecomment-670820501 rerun tests This is an automated message from the Apache Git Service. To respond to the message, please log on to

[hudi] branch master updated: [HUDI-1026] Removed slf4j dependency from HoodieClientTestHarness (#1928)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1072f27 [HUDI-1026] Removed slf4j dependency from

[GitHub] [hudi] leesf merged pull request #1928: [HUDI-1026]: removed slf4j dependency from HoodieClientTestHarness

leesf merged pull request #1928: URL: https://github.com/apache/hudi/pull/1928 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] leesf merged pull request #1932: [MINOR]Remove unused import

leesf merged pull request #1932: URL: https://github.com/apache/hudi/pull/1932 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[hudi] branch master updated: [MINOR] Remove unused import (#1932)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8b66524 [MINOR] Remove unused import (#1932)

[GitHub] [hudi] garyli1019 commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

garyli1019 commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670818469 The integration test fails sometimes for no reason. I have been seeing this for a few times. Maybe rerun will fix if lucky.

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #363

See Changes: -- [...truncated 2.59 KB...] cdi-api-1.0.jar cdi-api.license commons-cli-1.4.jar commons-cli.license commons-io-2.5.jar commons-io.license

[GitHub] [hudi] umehrot2 commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

umehrot2 commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670814850 @vinothchandar the unit tests issues are resolved now. But the integration tests are behaving crazy. They passed the last time, and failed now even though I didn't make any code

[GitHub] [hudi] umehrot2 commented on a change in pull request #1702: [HUDI-426] Bootstrap datasource integration

umehrot2 commented on a change in pull request #1702: URL: https://github.com/apache/hudi/pull/1702#discussion_r46735 ## File path: hudi-spark/src/test/scala/org/apache/hudi/functional/TestDataSourceForBootstrap.scala ## @@ -0,0 +1,616 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] garyli1019 commented on a change in pull request #1702: [HUDI-426] Bootstrap datasource integration

garyli1019 commented on a change in pull request #1702: URL: https://github.com/apache/hudi/pull/1702#discussion_r467341322 ## File path: hudi-spark/src/test/scala/org/apache/hudi/functional/TestDataSourceForBootstrap.scala ## @@ -0,0 +1,616 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] garyli1019 commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

garyli1019 commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670799557 > @vinothchandar I fixed the rebase issue, and resolved the `bootstrap` related test failures. I still see `MOR data source` related unit test failures because of `spark

[GitHub] [hudi] umehrot2 commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

umehrot2 commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670793211 @vinothchandar I fixed the rebase issue, and resolved the `bootstrap` related test failures. I still see `MOR data source` related unit test failures because of `spark context`.

[GitHub] [hudi] umehrot2 commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

umehrot2 commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670717238 > @umehrot2 some tests are failing . looking at them later today. > > Before we head into the weekend, is this PR ready from your perspective. if so, I will take care of

[GitHub] [hudi] zhedoubushishi commented on a change in pull request #1869: [HUDI-427] Implement CLI support for performing bootstrap

zhedoubushishi commented on a change in pull request #1869: URL: https://github.com/apache/hudi/pull/1869#discussion_r467272326 ## File path: hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java ## @@ -240,13 +240,21 @@ private

[jira] [Resolved] (HUDI-69) Support realtime view in Spark datasource #136

[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-69. Resolution: Fixed > Support realtime view in Spark datasource #136 >

[jira] [Resolved] (HUDI-1052) Support vectorized reader for MOR datasource reader

[ https://issues.apache.org/jira/browse/HUDI-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-1052. -- Resolution: Fixed > Support vectorized reader for MOR datasource reader >

[jira] [Resolved] (HUDI-1050) Support filter pushdown and column pruning for MOR table on Spark Datasource

[ https://issues.apache.org/jira/browse/HUDI-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li resolved HUDI-1050. -- Resolution: Fixed > Support filter pushdown and column pruning for MOR table on Spark

[jira] [Updated] (HUDI-1052) Support vectorized reader for MOR datasource reader

[ https://issues.apache.org/jira/browse/HUDI-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanjia Gary Li updated HUDI-1052: - Status: In Progress (was: Open) > Support vectorized reader for MOR datasource reader >

[GitHub] [hudi] rufferjr commented on issue #1923: [SUPPORT] Hive Sync fails to add decimal partition

rufferjr commented on issue #1923: URL: https://github.com/apache/hudi/issues/1923#issuecomment-670636479 @bvaradar would you like the S3 partition path? If so, the following examples may be of use: s3://data-beta/vault/cod_combinations/partition_val=1003

[jira] [Comment Edited] (HUDI-1146) DeltaStreamer fails to start when No updated records + schemaProvider not supplied

[ https://issues.apache.org/jira/browse/HUDI-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173348#comment-17173348 ] Balaji Varadarajan edited comment on HUDI-1146 at 8/7/20, 5:25 PM: ---

[jira] [Comment Edited] (HUDI-1146) DeltaStreamer fails to start when No updated records + schemaProvider not supplied

[ https://issues.apache.org/jira/browse/HUDI-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173348#comment-17173348 ] Balaji Varadarajan edited comment on HUDI-1146 at 8/7/20, 5:25 PM: ---

[jira] [Comment Edited] (HUDI-1146) DeltaStreamer fails to start when No updated records + schemaProvider not supplied

[ https://issues.apache.org/jira/browse/HUDI-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173348#comment-17173348 ] Balaji Varadarajan edited comment on HUDI-1146 at 8/7/20, 5:24 PM: ---

[jira] [Commented] (HUDI-1146) DeltaStreamer fails to start when No updated records + schemaProvider not supplied

[ https://issues.apache.org/jira/browse/HUDI-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173348#comment-17173348 ] Balaji Varadarajan commented on HUDI-1146: -- [~bdscheller]: I think InputBatch::getSchemaProvider

[jira] [Created] (HUDI-1171) Hudi 0.5.2 with ScalaTest and Spark 2.4.0 java.lang.NoClassDefFoundError: Could not initialize class org.apache.hudi.avro.model.HoodieCleanerPlan

Prashanth created HUDI-1171: --- Summary: Hudi 0.5.2 with ScalaTest and Spark 2.4.0 java.lang.NoClassDefFoundError: Could not initialize class org.apache.hudi.avro.model.HoodieCleanerPlan Key: HUDI-1171 URL:

[GitHub] [hudi] zhedoubushishi commented on pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

zhedoubushishi commented on pull request #1933: URL: https://github.com/apache/hudi/pull/1933#issuecomment-670594285 > @vinothchandar : It looks like @zhedoubushishi had addressed the same issue in his original PR. So, I am going to close this one. @zhedoubushishi : Can you us the changes

[GitHub] [hudi] UZi5136225 commented on pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 commented on pull request #1931: URL: https://github.com/apache/hudi/pull/1931#issuecomment-670594131 @leesf Please review This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Assigned] (HUDI-971) Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

[ https://issues.apache.org/jira/browse/HUDI-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-971: --- Assignee: Wenning Ding (was: Balaji Varadarajan) > Fix

[GitHub] [hudi] bvaradar commented on a change in pull request #1869: [HUDI-427] Implement CLI support for performing bootstrap

bvaradar commented on a change in pull request #1869: URL: https://github.com/apache/hudi/pull/1869#discussion_r467135559 ## File path: hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java ## @@ -240,13 +240,21 @@ private

[GitHub] [hudi] bvaradar edited a comment on pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

bvaradar edited a comment on pull request #1933: URL: https://github.com/apache/hudi/pull/1933#issuecomment-670592925 @vinothchandar : It looks like @zhedoubushishi had addressed the same issue in his original PR. So, I am going to close this one. @zhedoubushishi : Can you us the changes

[GitHub] [hudi] bvaradar commented on pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

bvaradar commented on pull request #1933: URL: https://github.com/apache/hudi/pull/1933#issuecomment-670592925 @vinothchandar : It looks like @zhedoubushishi had addressed it in his original PR. So, I am going to close this one. @zhedoubushishi : Can you us the changes in this PR

[GitHub] [hudi] bvaradar closed pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

bvaradar closed pull request #1933: URL: https://github.com/apache/hudi/pull/1933 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] bvaradar commented on a change in pull request #1869: [HUDI-427] Implement CLI support for performing bootstrap

bvaradar commented on a change in pull request #1869: URL: https://github.com/apache/hudi/pull/1869#discussion_r467133890 ## File path: hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java ## @@ -240,13 +240,21 @@ private

[GitHub] [hudi] bvaradar commented on a change in pull request #1869: [HUDI-427] Implement CLI support for performing bootstrap

bvaradar commented on a change in pull request #1869: URL: https://github.com/apache/hudi/pull/1869#discussion_r467133890 ## File path: hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java ## @@ -240,13 +240,21 @@ private

[jira] [Commented] (HUDI-1015) Audit all getAllPartitionPaths() calls and keep em out of fast path

[ https://issues.apache.org/jira/browse/HUDI-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173242#comment-17173242 ] Balaji Varadarajan commented on HUDI-1015: -- Subtasks added to track all location where we list

[jira] [Updated] (HUDI-1170) File Listing during log file rollback is affecting ingestion latency in S3

[ https://issues.apache.org/jira/browse/HUDI-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1170: - Fix Version/s: 0.6.1 > File Listing during log file rollback is affecting ingestion

[jira] [Updated] (HUDI-1170) File Listing during log file rollback is affecting ingestion latency in S3

[ https://issues.apache.org/jira/browse/HUDI-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1170: - Status: Open (was: New) > File Listing during log file rollback is affecting ingestion

[jira] [Updated] (HUDI-1170) File Listing during log file rollback is affecting ingestion latency in S3

[ https://issues.apache.org/jira/browse/HUDI-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1170: - Description: (Source : [https://github.com/apache/hudi/issues/1852])   : 

[jira] [Created] (HUDI-1170) File Listing during log file rollback is affecting ingestion latency in S3

Balaji Varadarajan created HUDI-1170: Summary: File Listing during log file rollback is affecting ingestion latency in S3 Key: HUDI-1170 URL: https://issues.apache.org/jira/browse/HUDI-1170

[jira] [Updated] (HUDI-1015) Audit all getAllPartitionPaths() calls and keep em out of fast path

[ https://issues.apache.org/jira/browse/HUDI-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1015: - Priority: Major (was: Blocker) > Audit all getAllPartitionPaths() calls and keep em out

[jira] [Updated] (HUDI-1015) Audit all getAllPartitionPaths() calls and keep em out of fast path

[ https://issues.apache.org/jira/browse/HUDI-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1015: - Fix Version/s: (was: 0.6.0) 0.6.1 > Audit all

[jira] [Created] (HUDI-1169) Audit Partition Listing : Snapshot Copier and Exporter Utilities

Balaji Varadarajan created HUDI-1169: Summary: Audit Partition Listing : Snapshot Copier and Exporter Utilities Key: HUDI-1169 URL: https://issues.apache.org/jira/browse/HUDI-1169 Project: Apache

[jira] [Created] (HUDI-1168) Audit Partition Listing : Savepoint Creation

Balaji Varadarajan created HUDI-1168: Summary: Audit Partition Listing : Savepoint Creation Key: HUDI-1168 URL: https://issues.apache.org/jira/browse/HUDI-1168 Project: Apache Hudi Issue

[GitHub] [hudi] UZi5136225 commented on pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 commented on pull request #1931: URL: https://github.com/apache/hudi/pull/1931#issuecomment-670584668 

[GitHub] [hudi] UZi5136225 commented on pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 commented on pull request #1931: URL: https://github.com/apache/hudi/pull/1931#issuecomment-670584994 

[GitHub] [hudi] vinothchandar commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

vinothchandar commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670584927 @umehrot2 some tests are failing . looking at them later today. Before we head into the weekend, is this PR ready from your perspective. if so, I will take care of

[GitHub] [hudi] UZi5136225 removed a comment on pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 removed a comment on pull request #1931: URL: https://github.com/apache/hudi/pull/1931#issuecomment-670584668 

[jira] [Created] (HUDI-1167) Audit Partition Listing : Hive Syncing

Balaji Varadarajan created HUDI-1167: Summary: Audit Partition Listing : Hive Syncing Key: HUDI-1167 URL: https://issues.apache.org/jira/browse/HUDI-1167 Project: Apache Hudi Issue Type:

[GitHub] [hudi] UZi5136225 commented on pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 commented on pull request #1931: URL: https://github.com/apache/hudi/pull/1931#issuecomment-670583885 

[jira] [Created] (HUDI-1166) Audit Partition Listing : Rollback By Listing

Balaji Varadarajan created HUDI-1166: Summary: Audit Partition Listing : Rollback By Listing Key: HUDI-1166 URL: https://issues.apache.org/jira/browse/HUDI-1166 Project: Apache Hudi

[jira] [Created] (HUDI-1165) Audit Partition Listing : Compaction Scheduling

Balaji Varadarajan created HUDI-1165: Summary: Audit Partition Listing : Compaction Scheduling Key: HUDI-1165 URL: https://issues.apache.org/jira/browse/HUDI-1165 Project: Apache Hudi

[GitHub] [hudi] jpugliesi commented on issue #1925: [SUPPORT] Support for Confluent Cloud SchemaRegistryProvider

jpugliesi commented on issue #1925: URL: https://github.com/apache/hudi/issues/1925#issuecomment-670581206 @bvaradar brilliant, didn't think of this - I'll give it a try and report back. This is an automated message from

[GitHub] [hudi] vinothchandar commented on pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

vinothchandar commented on pull request #1933: URL: https://github.com/apache/hudi/pull/1933#issuecomment-670580147 @bvaradar is this a release blocker? sounds like that? This is an automated message from the Apache Git

[jira] [Updated] (HUDI-1164) Audit Partition Listing Location : CleanPlanner.getPartitionPathsForFullCleaning

[ https://issues.apache.org/jira/browse/HUDI-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1164: - Description: This ticket is to track all known locations where we call

[jira] [Updated] (HUDI-1164) Audit Partition Listing Location : CleanPlanner.getPartitionPathsForFullCleaning

[ https://issues.apache.org/jira/browse/HUDI-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1164: - Status: Open (was: New) > Audit Partition Listing Location : >

[jira] [Created] (HUDI-1164) Audit Partition Listing Location : CleanPlanner.getPartitionPathsForFullCleaning

Balaji Varadarajan created HUDI-1164: Summary: Audit Partition Listing Location : CleanPlanner.getPartitionPathsForFullCleaning Key: HUDI-1164 URL: https://issues.apache.org/jira/browse/HUDI-1164

[jira] [Updated] (HUDI-1163) Audit Partition Listing Location : Global Simple Index lookup

[ https://issues.apache.org/jira/browse/HUDI-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1163: - Status: Open (was: New) > Audit Partition Listing Location : Global Simple Index lookup

[jira] [Created] (HUDI-1163) Audit Partition Listing Location : Global Simple Index lookup

Balaji Varadarajan created HUDI-1163: Summary: Audit Partition Listing Location : Global Simple Index lookup Key: HUDI-1163 URL: https://issues.apache.org/jira/browse/HUDI-1163 Project: Apache

[jira] [Updated] (HUDI-1162) Audit Partition Listing Location : Global Bloom Index lookup

[ https://issues.apache.org/jira/browse/HUDI-1162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1162: - Status: Open (was: New) > Audit Partition Listing Location : Global Bloom Index lookup >

[jira] [Created] (HUDI-1162) Audit Partition Listing Location : Global Bloom Index lookup

Balaji Varadarajan created HUDI-1162: Summary: Audit Partition Listing Location : Global Bloom Index lookup Key: HUDI-1162 URL: https://issues.apache.org/jira/browse/HUDI-1162 Project: Apache

[jira] [Updated] (HUDI-971) Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

[ https://issues.apache.org/jira/browse/HUDI-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-971: Status: Patch Available (was: In Progress) > Fix

[jira] [Updated] (HUDI-971) Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

[ https://issues.apache.org/jira/browse/HUDI-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-971: Status: In Progress (was: Open) > Fix HFileBootstrapIndexReader.getIndexedPartitions()

[jira] [Updated] (HUDI-971) Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

[ https://issues.apache.org/jira/browse/HUDI-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-971: Labels: pull-request-available (was: ) > Fix HFileBootstrapIndexReader.getIndexedPartitions()

[GitHub] [hudi] bvaradar commented on pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

bvaradar commented on pull request #1933: URL: https://github.com/apache/hudi/pull/1933#issuecomment-670573069 @zhedoubushishi : Can you review this. This would impact your bootstrap CLI. This is an automated message from

[GitHub] [hudi] bvaradar opened a new pull request #1933: [HUDI-971] Fix HFileBootstrapIndexReader.getIndexedPartitions() returns unclean partition name

bvaradar opened a new pull request #1933: URL: https://github.com/apache/hudi/pull/1933 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] Yungthuis opened a new pull request #1932: [MINOR]Remove unused import

Yungthuis opened a new pull request #1932: URL: https://github.com/apache/hudi/pull/1932 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] UZi5136225 opened a new pull request #1931: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 opened a new pull request #1931: URL: https://github.com/apache/hudi/pull/1931 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] UZi5136225 closed pull request #1930: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 closed pull request #1930: URL: https://github.com/apache/hudi/pull/1930 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] UZi5136225 opened a new pull request #1930: [HUDI-210] hudi-support-prometheus-pushgateway

UZi5136225 opened a new pull request #1930: URL: https://github.com/apache/hudi/pull/1930 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] UZi5136225 closed pull request #1726: [HUDI-210]Hudi support prometheus

UZi5136225 closed pull request #1726: URL: https://github.com/apache/hudi/pull/1726 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (HUDI-1159) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/HUDI-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated HUDI-1159: --- Description: Provide an interface for Parquet column encryption policy engine clients. >

[jira] [Updated] (HUDI-1159) Parquet encryption policy interface

[ https://issues.apache.org/jira/browse/HUDI-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated HUDI-1159: --- Summary: Parquet encryption policy interface (was: Encryption policy interface) > Parquet

[GitHub] [hudi] Mathieu1124 commented on pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes

Mathieu1124 commented on pull request #1901: URL: https://github.com/apache/hudi/pull/1901#issuecomment-670506233 @cheshta2904 @pratyakshsharma I have addressed all your concerns, thanks for your detailed review :) This is

[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes

Mathieu1124 commented on a change in pull request #1901: URL: https://github.com/apache/hudi/pull/1901#discussion_r467025816 ## File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/HiveSyncDagGenerator.java ## @@ -31,6 +31,9 @@ import

[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes

Mathieu1124 commented on a change in pull request #1901: URL: https://github.com/apache/hudi/pull/1901#discussion_r467025753 ## File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/ComplexDagGenerator.java ## @@ -33,6 +33,9 @@ import

[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes

Mathieu1124 commented on a change in pull request #1901: URL: https://github.com/apache/hudi/pull/1901#discussion_r467025989 ## File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java ## @@ -28,6 +28,9 @@ import

[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes

Mathieu1124 commented on a change in pull request #1901: URL: https://github.com/apache/hudi/pull/1901#discussion_r467025252 ## File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/ITTestBase.java ## @@ -48,6 +48,9 @@ import static

[GitHub] [hudi] leesf opened a new pull request #1929: [HUDI-1160] Support update partial fields for CoW table

leesf opened a new pull request #1929: URL: https://github.com/apache/hudi/pull/1929 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull

[jira] [Updated] (HUDI-1160) Support update partial fields for CoW table

[ https://issues.apache.org/jira/browse/HUDI-1160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1160: - Labels: pull-request-available (was: ) > Support update partial fields for CoW table >

[jira] [Created] (HUDI-1161) Support update partial fields for MoR table

leesf created HUDI-1161: --- Summary: Support update partial fields for MoR table Key: HUDI-1161 URL: https://issues.apache.org/jira/browse/HUDI-1161 Project: Apache Hudi Issue Type: Sub-task

[jira] [Created] (HUDI-1160) Support update partial fields for CoW table

leesf created HUDI-1160: --- Summary: Support update partial fields for CoW table Key: HUDI-1160 URL: https://issues.apache.org/jira/browse/HUDI-1160 Project: Apache Hudi Issue Type: Sub-task

[GitHub] [hudi] nsivabalan commented on pull request #1912: [HUDI-1098] Adding TimedWaitOnAppearConsistencyGuard

nsivabalan commented on pull request #1912: URL: https://github.com/apache/hudi/pull/1912#issuecomment-670480225 @umehrot2 : Would appreciate if you agree on the approach here. Before I go ahead and address feedback want to have consensus.

[GitHub] [hudi] nsivabalan commented on pull request #1912: [HUDI-1098] Adding TimedWaitOnAppearConsistencyGuard

nsivabalan commented on pull request #1912: URL: https://github.com/apache/hudi/pull/1912#issuecomment-670479909 @bvaradar : since you suggested to have the TimedWaitOnAppearCG as default opt in, I would suggest to introduce a new config for the sleep time. so that we can set it to 2 or 3

[GitHub] [hudi] nsivabalan commented on a change in pull request #1858: [HUDI-1014] Adding Upgrade and downgrade infra for smooth transitioning from list based rollback to marker based rollback

nsivabalan commented on a change in pull request #1858: URL: https://github.com/apache/hudi/pull/1858#discussion_r466991187 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -186,10 +188,14 @@ public HoodieMetrics getMetrics()

[GitHub] [hudi] nsivabalan commented on a change in pull request #1858: [HUDI-1014] Adding Upgrade and downgrade infra for smooth transitioning from list based rollback to marker based rollback

nsivabalan commented on a change in pull request #1858: URL: https://github.com/apache/hudi/pull/1858#discussion_r466989376 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java ## @@ -329,9 +341,34 @@ private static int

[jira] [Created] (HUDI-1159) Encryption policy interface

Gidon Gershinsky created HUDI-1159: -- Summary: Encryption policy interface Key: HUDI-1159 URL: https://issues.apache.org/jira/browse/HUDI-1159 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] Mathieu1124 commented on pull request #1886: [HUDI-1122]Introduce a kafka implementation of hoodie write commit ca…

Mathieu1124 commented on pull request #1886: URL: https://github.com/apache/hudi/pull/1886#issuecomment-670453889 > > > > I was wondering can we move this implement to hudi-client module just like the way all the implementations of metrics does. > > I think we can move this

[GitHub] [hudi] vinothchandar commented on pull request #1702: [HUDI-426] Bootstrap datasource integration

vinothchandar commented on pull request #1702: URL: https://github.com/apache/hudi/pull/1702#issuecomment-670414375 @umehrot2 I rebased this after landing @garyli1019 's PR. Please take a look at `DefaultSource` again to make sure things are ok

[GitHub] [hudi] vinothchandar closed pull request #1678: [HUDI-242] Metadata Bootstrap changes

vinothchandar closed pull request #1678: URL: https://github.com/apache/hudi/pull/1678 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] vinothchandar commented on pull request #1678: [HUDI-242] Metadata Bootstrap changes

vinothchandar commented on pull request #1678: URL: https://github.com/apache/hudi/pull/1678#issuecomment-670393921 closing this. There is a followup JIRA assigned to you @bvaradar with some of the unaddressed comments from here

[GitHub] [hudi] vinothchandar commented on a change in pull request #1760: [HUDI-1040] Update apis for spark3 compatibility

vinothchandar commented on a change in pull request #1760: URL: https://github.com/apache/hudi/pull/1760#discussion_r466890310 ## File path: hudi-spark/src/main/scala/org/apache/hudi/AvroConversionUtils.scala ## @@ -78,4 +79,21 @@ object AvroConversionUtils { def

[GitHub] [hudi] pratyakshsharma commented on pull request #1928: [HUDI-1026]: removed slf4j dependency from HoodieClientTestHarness

pratyakshsharma commented on pull request #1928: URL: https://github.com/apache/hudi/pull/1928#issuecomment-670390527 LGTM! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] cheshta2904 opened a new pull request #1928: [HUDI-1026]: removed slf4j dependency from HoodieClientTestHarness

cheshta2904 opened a new pull request #1928: URL: https://github.com/apache/hudi/pull/1928 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[jira] [Updated] (HUDI-1026) Remove slf4j dependency from HoodieClientTestHarness

[ https://issues.apache.org/jira/browse/HUDI-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1026: - Labels: pull-request-available (was: ) > Remove slf4j dependency from HoodieClientTestHarness >

[GitHub] [hudi] luffyd commented on issue #1913: [SUPPORT][MOR]Too many open files on IOException and Crash

luffyd commented on issue #1913: URL: https://github.com/apache/hudi/issues/1913#issuecomment-670385009 It is this, seems latest. This is whatever comes in AWS emr

  1   2   >