[GitHub] [hudi] hudi-bot removed a comment on pull request #3813: [HUDI-2563][hudi-client] Refactor CompactionTriggerStrategy.

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3813: URL: https://github.com/apache/hudi/pull/3813#issuecomment-961588588 ## CI report: * 4afb1a587e0809c4d6d3106aa33e6cf1eda47f0e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969670756 ## CI report: * d6ba30a69737a8842afa5fe41fa8b3e0453b5e47 UNKNOWN * e710a71570654b50d9dde1a1c5a3f4c683cb16bf Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969589850 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[jira] [Commented] (HUDI-2745) Record count does not match input after compaction is scheduled when running Hudi Kafka Connect sink

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444218#comment-17444218 ] Ethan Guo commented on HUDI-2745: - I check the {{MergeOnReadSnapshotRelation}}  and file index built for

[jira] [Updated] (HUDI-2745) Record count does not match input after compaction is scheduled when running Hudi Kafka Connect sink

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2745: Description: Spark Shell command to do snapshot query: {code:java} val basePath =

[GitHub] [hudi] hudi-bot removed a comment on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969511369 ## CI report: * f6e072b53f37544aef6194bfde7868162da1ac39 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969617935 ## CI report: * c3c27d6062111b539575c342b4ea1d5e3057bd7c Azure:

[jira] [Commented] (HUDI-2735) Fix archival of commits in Java client for Kafka Connect

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444216#comment-17444216 ] Ethan Guo commented on HUDI-2735: - The archival process is triggered post every commit.  Deltacommits are

[jira] [Updated] (HUDI-2765) Archival does not count the number of rollback instants

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2765: Description: This issue is found during testing Kafka Connect Sink with Java client:

[jira] [Updated] (HUDI-2765) Archival does not count the number of rollback instants

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-2765: Summary: Archival does not count the number of rollback instants (was: Archival does not count the number

[jira] [Created] (HUDI-2765) Archival does not count the number of rollback commits

2021-11-15 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-2765: --- Summary: Archival does not count the number of rollback commits Key: HUDI-2765 URL: https://issues.apache.org/jira/browse/HUDI-2765 Project: Apache Hudi Issue Type:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969584592 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969589850 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969579136 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969584592 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-969579136 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3992: [MINOR] Add more configuration to Kafka setup script

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3992: URL: https://github.com/apache/hudi/pull/3992#issuecomment-968222615 ## CI report: * 6db27eba4ff0a2ff9e1059b96982fd59fbc2d46d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734] Setting default metadata enable per engine

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969548904 ## CI report: * 2722db996ffccfc690bba36eaad6252ec738590c UNKNOWN * d693cfa69a5b303398fb11d7d3b56b1175c14777 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734] Setting default metadata enable per engine

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969463811 ## CI report: * f33211a768603d63ad7a834b085346cc584d0188 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969444267 ## CI report: * f6e072b53f37544aef6194bfde7868162da1ac39 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969511369 ## CI report: * f6e072b53f37544aef6194bfde7868162da1ac39 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734] Setting default metadata enable per engine

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969421874 ## CI report: * f33211a768603d63ad7a834b085346cc584d0188 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734] Setting default metadata enable per engine

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969463811 ## CI report: * f33211a768603d63ad7a834b085346cc584d0188 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969438707 ## CI report: * f6e072b53f37544aef6194bfde7868162da1ac39 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969444267 ## CI report: * f6e072b53f37544aef6194bfde7868162da1ac39 Azure:

[jira] [Created] (HUDI-2764) Address test failures after enabling virtual keys support for the metadata table

2021-11-15 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2764: Summary: Address test failures after enabling virtual keys support for the metadata table Key: HUDI-2764 URL: https://issues.apache.org/jira/browse/HUDI-2764

[GitHub] [hudi] hudi-bot removed a comment on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969437363 ## CI report: * ac0b1acace22b81fa2655f851adeacc4de5733f3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969438707 ## CI report: * f6e072b53f37544aef6194bfde7868162da1ac39 Azure:

[jira] [Updated] (HUDI-2763) Avoid persisting redundant key field in the Metadata table record payload

2021-11-15 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-2763: - Fix Version/s: 0.10.0 > Avoid persisting redundant key field in the Metadata table record

[jira] [Created] (HUDI-2763) Avoid persisting redundant key field in the Metadata table record payload

2021-11-15 Thread Manoj Govindassamy (Jira)
Manoj Govindassamy created HUDI-2763: Summary: Avoid persisting redundant key field in the Metadata table record payload Key: HUDI-2763 URL: https://issues.apache.org/jira/browse/HUDI-2763

[GitHub] [hudi] hudi-bot removed a comment on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969435854 ## CI report: * ac0b1acace22b81fa2655f851adeacc4de5733f3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969437363 ## CI report: * ac0b1acace22b81fa2655f851adeacc4de5733f3 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3968: URL: https://github.com/apache/hudi/pull/3968#issuecomment-969385181 ## CI report: * 7e8177ecb8e0446630fc2990ec79816183ba7625 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3968: URL: https://github.com/apache/hudi/pull/3968#issuecomment-969436232 ## CI report: * 57721a3df00051ff31145778e7f9f88c1d9b7ec7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969435854 ## CI report: * ac0b1acace22b81fa2655f851adeacc4de5733f3 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3416: [HUDI-2362] Add external config file support

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-969118756 ## CI report: * ac0b1acace22b81fa2655f851adeacc4de5733f3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969427971 ## CI report: * 0ce830a2c15b326a53acba37ec72015b6c2eea11 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969364737 ## CI report: * 61028ffcab8f870141eadfd1b97db93d32962fd7 Azure:

[GitHub] [hudi] umehrot2 commented on pull request #3956: [HUDI-2641] Avoid deleting all inflight commits heartbeats while rolling back failed writes

2021-11-15 Thread GitBox
umehrot2 commented on pull request #3956: URL: https://github.com/apache/hudi/pull/3956#issuecomment-969423349 > can we please add UT/functional tests for the fix. @nsivabalan It seems there are no functional tests which test the scenarios where multiple concurrent upserts/inserts

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734] Setting default metadata enable per engine

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969421874 ## CI report: * f33211a768603d63ad7a834b085346cc584d0188 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734] Setting default metadata enable per engine

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969418676 ## CI report: * f33211a768603d63ad7a834b085346cc584d0188 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969418676 ## CI report: * f33211a768603d63ad7a834b085346cc584d0188 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969415457 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[jira] [Commented] (HUDI-2602) Publish design doc/RFC for metadata based range index

2021-11-15 Thread Manoj Govindassamy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444171#comment-17444171 ] Manoj Govindassamy commented on HUDI-2602: -- https://issues.apache.org/jira/browse/HUDI-2589 takes

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969415457 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969405504 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3985: URL: https://github.com/apache/hudi/pull/3985#issuecomment-969412049 ## CI report: * 1976a49024ffcd80c97bc1e4dfffad332fb11a71 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3985: URL: https://github.com/apache/hudi/pull/3985#issuecomment-969352552 ## CI report: * a4dc81fdd62782c79a2b1f59196429a2e0abf8ff Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969405504 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969402385 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot commented on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969402385 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4003: [HUDI-2734][WIP] Setting default metadata enable as false for Java

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #4003: URL: https://github.com/apache/hudi/pull/4003#issuecomment-969158504 ## CI report: * 1892c70ed68c2ea6fb3a032eea7262116bd56622 Azure:

[GitHub] [hudi] umehrot2 commented on a change in pull request #3979: [HUDI-2636] - made release notes more discoverable

2021-11-15 Thread GitBox
umehrot2 commented on a change in pull request #3979: URL: https://github.com/apache/hudi/pull/3979#discussion_r749743954 ## File path: website/releases/all-releases.md ## @@ -0,0 +1,66 @@ +--- +title: Releases +sidebar_position: 1 +keywords: [hudi, download, release notes]

[GitHub] [hudi] umehrot2 commented on pull request #3979: [HUDI-2636] - made release notes more discoverable

2021-11-15 Thread GitBox
umehrot2 commented on pull request #3979: URL: https://github.com/apache/hudi/pull/3979#issuecomment-969400388 I am honestly not sure why we need this change. Even Spark follows the same page naming https://spark.apache.org/downloads.html. If at all, one thing we should do is move the

[GitHub] [hudi] yihua commented on a change in pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
yihua commented on a change in pull request #3857: URL: https://github.com/apache/hudi/pull/3857#discussion_r749735457 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/MultipleSparkJobExecutionStrategy.java ## @@ -206,11

[GitHub] [hudi] hudi-bot commented on pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3968: URL: https://github.com/apache/hudi/pull/3968#issuecomment-969385181 ## CI report: * 7e8177ecb8e0446630fc2990ec79816183ba7625 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3968: URL: https://github.com/apache/hudi/pull/3968#issuecomment-969383359 ## CI report: * 7e8177ecb8e0446630fc2990ec79816183ba7625 Azure:

[GitHub] [hudi] manojpec commented on a change in pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3968: URL: https://github.com/apache/hudi/pull/3968#discussion_r749730096 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java ## @@ -151,6 +155,14 @@ protected

[GitHub] [hudi] hudi-bot removed a comment on pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3968: URL: https://github.com/apache/hudi/pull/3968#issuecomment-968160640 ## CI report: * 7e8177ecb8e0446630fc2990ec79816183ba7625 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3968: URL: https://github.com/apache/hudi/pull/3968#issuecomment-969383359 ## CI report: * 7e8177ecb8e0446630fc2990ec79816183ba7625 Azure:

[GitHub] [hudi] manojpec commented on a change in pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3968: URL: https://github.com/apache/hudi/pull/3968#discussion_r749729855 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java ## @@ -120,28 +120,32 @@ private int

[GitHub] [hudi] manojpec commented on a change in pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3968: URL: https://github.com/apache/hudi/pull/3968#discussion_r749729587 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/SpillableMapUtils.java ## @@ -115,22 +115,28 @@ public static long

[GitHub] [hudi] manojpec commented on a change in pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3968: URL: https://github.com/apache/hudi/pull/3968#discussion_r749729504 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java ## @@ -343,11 +362,13 @@ private void

[GitHub] [hudi] manojpec commented on a change in pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3968: URL: https://github.com/apache/hudi/pull/3968#discussion_r749729108 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/SimpleKeyGenerator.java ## @@ -46,10 +49,10 @@ public

[GitHub] [hudi] manojpec commented on a change in pull request #3968: [WIP] [HUDI-2593] Virtual keys support for metadata table

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3968: URL: https://github.com/apache/hudi/pull/3968#discussion_r749728654 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java ## @@ -89,6 +89,10 @@

[GitHub] [hudi] hudi-bot removed a comment on pull request #3950: [HUDI-2151] Part3 Enabling marker based rollback as default rollback strategy

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3950: URL: https://github.com/apache/hudi/pull/3950#issuecomment-969338015 ## CI report: * e0fd589bf2bee928dec4baf993f99b95d28db750 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3950: [HUDI-2151] Part3 Enabling marker based rollback as default rollback strategy

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3950: URL: https://github.com/apache/hudi/pull/3950#issuecomment-969381342 ## CI report: * 5771d7e4d7dcada42d4a929bff877b2db8f78fed Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969322877 ## CI report: * 4fe7a886380689fdf1d9a038fb97c1c6bf5aaa83 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969377387 ## CI report: * 68cba72835e1e89f9c1b8096f5ae36724c48a9c1 Azure:

[jira] [Comment Edited] (HUDI-2734) Disable metadata by default for some engines and infra

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444126#comment-17444126 ] Ethan Guo edited comment on HUDI-2734 at 11/15/21, 10:11 PM: - I hit `Caused

[GitHub] [hudi] yihua commented on a change in pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
yihua commented on a change in pull request #3857: URL: https://github.com/apache/hudi/pull/3857#discussion_r749716962 ## File path: hudi-client/hudi-java-client/src/main/java/org/apache/hudi/client/clustering/run/strategy/JavaExecutionStrategy.java ## @@ -0,0 +1,245 @@ +/* +

[GitHub] [hudi] hudi-bot commented on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969364737 ## CI report: * 61028ffcab8f870141eadfd1b97db93d32962fd7 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969342179 ## CI report: * 61028ffcab8f870141eadfd1b97db93d32962fd7 Azure:

[jira] [Commented] (HUDI-2734) Disable metadata by default for some engines and infra

2021-11-15 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444126#comment-17444126 ] Ethan Guo commented on HUDI-2734: - I hit `Caused by: java.io.FileNotFoundException: File

[GitHub] [hudi] hudi-bot commented on pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3985: URL: https://github.com/apache/hudi/pull/3985#issuecomment-969352552 ## CI report: * a4dc81fdd62782c79a2b1f59196429a2e0abf8ff Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3985: URL: https://github.com/apache/hudi/pull/3985#issuecomment-969350547 ## CI report: * a4dc81fdd62782c79a2b1f59196429a2e0abf8ff Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3985: URL: https://github.com/apache/hudi/pull/3985#issuecomment-969350547 ## CI report: * a4dc81fdd62782c79a2b1f59196429a2e0abf8ff Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3985: URL: https://github.com/apache/hudi/pull/3985#issuecomment-967754259 ## CI report: * a4dc81fdd62782c79a2b1f59196429a2e0abf8ff Azure:

[GitHub] [hudi] jintaoguan commented on a change in pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
jintaoguan commented on a change in pull request #3985: URL: https://github.com/apache/hudi/pull/3985#discussion_r749574597 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/IncrementalRelation.scala ## @@ -98,7 +97,12 @@ class

[GitHub] [hudi] hudi-bot removed a comment on pull request #3986: [HUDI-2550][WIP] Expand File-Group candidates list for appending for MOR tables

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3986: URL: https://github.com/apache/hudi/pull/3986#issuecomment-969305412 ## CI report: * 786d593265ce66cff0167261dc34fcb9310f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3986: [HUDI-2550][WIP] Expand File-Group candidates list for appending for MOR tables

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3986: URL: https://github.com/apache/hudi/pull/3986#issuecomment-969344397 ## CI report: * d6873a12f5a3d16b4e3505243780ea56339b3d2e Azure:

[GitHub] [hudi] jintaoguan commented on a change in pull request #3985: [HUDI-2754] Performance improvement for IncrementalRelation

2021-11-15 Thread GitBox
jintaoguan commented on a change in pull request #3985: URL: https://github.com/apache/hudi/pull/3985#discussion_r749574597 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/IncrementalRelation.scala ## @@ -98,7 +97,12 @@ class

[GitHub] [hudi] hudi-bot commented on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969342179 ## CI report: * 61028ffcab8f870141eadfd1b97db93d32962fd7 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969340087 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969340087 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969317457 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3950: [HUDI-2151] Part3 Enabling marker based rollback as default rollback strategy

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3950: URL: https://github.com/apache/hudi/pull/3950#issuecomment-969338015 ## CI report: * e0fd589bf2bee928dec4baf993f99b95d28db750 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3950: [HUDI-2151] Part3 Enabling marker based rollback as default rollback strategy

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3950: URL: https://github.com/apache/hudi/pull/3950#issuecomment-969335818 ## CI report: * e0fd589bf2bee928dec4baf993f99b95d28db750 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3950: [HUDI-2151] Part3 Enabling marker based rollback as default rollback strategy

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3950: URL: https://github.com/apache/hudi/pull/3950#issuecomment-969335818 ## CI report: * e0fd589bf2bee928dec4baf993f99b95d28db750 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3950: [HUDI-2151] Part3 Enabling marker based rollback as default rollback strategy

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3950: URL: https://github.com/apache/hudi/pull/3950#issuecomment-967117857 ## CI report: * e0fd589bf2bee928dec4baf993f99b95d28db750 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969322877 ## CI report: * 4fe7a886380689fdf1d9a038fb97c1c6bf5aaa83 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969320414 ## CI report: * 4fe7a886380689fdf1d9a038fb97c1c6bf5aaa83 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969189228 ## CI report: * 4fe7a886380689fdf1d9a038fb97c1c6bf5aaa83 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3989: URL: https://github.com/apache/hudi/pull/3989#issuecomment-969320414 ## CI report: * 4fe7a886380689fdf1d9a038fb97c1c6bf5aaa83 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969317457 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] manojpec commented on a change in pull request #3989: [HUDI-2589] RFC-37: Metadata table based bloom index

2021-11-15 Thread GitBox
manojpec commented on a change in pull request #3989: URL: https://github.com/apache/hudi/pull/3989#discussion_r749678848 ## File path: rfc/rfc-37/rfc-37.md ## @@ -0,0 +1,264 @@ + +# RFC-37: Metadata based Bloom Index + +## Proposers +- @nsivabalan +- @manojpec + +## Approvers

[GitHub] [hudi] hudi-bot removed a comment on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969315245 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot removed a comment on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-968730377 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3857: [HUDI-2332] Add clustering and compaction in Kafka Connect Sink

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3857: URL: https://github.com/apache/hudi/pull/3857#issuecomment-969315245 ## CI report: * 4bf71657f3669177acddef529f5673e8fa758c13 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3986: [HUDI-2550][WIP] Expand File-Group candidates list for appending for MOR tables

2021-11-15 Thread GitBox
hudi-bot commented on pull request #3986: URL: https://github.com/apache/hudi/pull/3986#issuecomment-969305412 ## CI report: * 786d593265ce66cff0167261dc34fcb9310f Azure:

<    1   2   3   4   >