[GitHub] [hudi] umehrot2 commented on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

umehrot2 commented on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-864335470 @vinothchandar @pengzhiwei2018 addressed the latest comments. If it looks good to you guys, I can land it. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] jkdll removed a comment on issue #3113: [SUPPORT] Deltastreamer Error when reading AVRO Schema with UNION Types

jkdll removed a comment on issue #3113: URL: https://github.com/apache/hudi/issues/3113#issuecomment-864336980 I have also tested with the latest version on the `master` branch. Same error. Moreover, without `--transformer-class

[GitHub] [hudi] codecov-commenter commented on pull request #3114: [MINOR] Fix wrong package name

codecov-commenter commented on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338505 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3114?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[jira] [Assigned] (HUDI-1175) Investigate CI test flakiness (hangs)

[ https://issues.apache.org/jira/browse/HUDI-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-1175: Assignee: sivabalan narayanan > Investigate CI test flakiness (hangs) >

[jira] [Updated] (HUDI-1175) Investigate CI test flakiness (hangs)

[ https://issues.apache.org/jira/browse/HUDI-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-1175: - Status: Open (was: New) > Investigate CI test flakiness (hangs) > -

[jira] [Resolved] (HUDI-1175) Investigate CI test flakiness (hangs)

[ https://issues.apache.org/jira/browse/HUDI-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu resolved HUDI-1175. -- Fix Version/s: 0.9.0 Resolution: Fixed > Investigate CI test flakiness (hangs) >

[GitHub] [hudi] hudi-bot commented on pull request #3114: [MINOR] Fix wrong package name

hudi-bot commented on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338167 ## CI report: * b8f7f02c0bc4b9621f79397f45cd62cd2c1c6ec8 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3114: [MINOR] Fix wrong package name

codecov-commenter edited a comment on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338505 #

[GitHub] [hudi] hudi-bot edited a comment on pull request #3115: [MINOR] Fix Javadoc wrong references

hudi-bot edited a comment on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864342921 ## CI report: * f76a8482b21f7311a4360331aba4343d9d4352af Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3115: [MINOR] Fix Javadoc wrong references

codecov-commenter edited a comment on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864343564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] umehrot2 commented on a change in pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

umehrot2 commented on a change in pull request #2893: URL: https://github.com/apache/hudi/pull/2893#discussion_r654728183 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieFileIndex.scala ## @@ -151,13 +184,33 @@ case class HoodieFileIndex(

[GitHub] [hudi] umehrot2 commented on a change in pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

umehrot2 commented on a change in pull request #2893: URL: https://github.com/apache/hudi/pull/2893#discussion_r654728218 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieFileIndex.scala ## @@ -151,13 +184,33 @@ case class HoodieFileIndex(

[GitHub] [hudi] hudi-bot edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

hudi-bot edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-863021428 ## CI report: * 50e2f1bbcffbd2db4f16901ac577302ffeb335b4 Azure:

[GitHub] [hudi] umehrot2 commented on a change in pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

umehrot2 commented on a change in pull request #2893: URL: https://github.com/apache/hudi/pull/2893#discussion_r654728264 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieFileIndex.scala ## @@ -107,34 +113,61 @@ case class HoodieFileIndex(

[GitHub] [hudi] jkdll commented on issue #3113: [SUPPORT] Deltastreamer Error when reading AVRO Schema with UNION Types

jkdll commented on issue #3113: URL: https://github.com/apache/hudi/issues/3113#issuecomment-864336980 I have also tested with the latest version on the `master` branch. Same error. Moreover, without `--transformer-class org.apache.hudi.utilities.transform.FlatteningTransformer` I

[GitHub] [hudi] jkdll edited a comment on issue #3113: [SUPPORT] Deltastreamer Error when reading AVRO Schema with UNION Types

jkdll edited a comment on issue #3113: URL: https://github.com/apache/hudi/issues/3113#issuecomment-864336980 I have also tested with the latest version on the `master` branch. Same error. Moreover, without `--transformer-class

[GitHub] [hudi] vinothchandar merged pull request #3115: [MINOR] Fix Javadoc wrong references

vinothchandar merged pull request #3115: URL: https://github.com/apache/hudi/pull/3115 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[hudi] branch master updated: [MINOR] Fix Javadoc wrong references (#3115)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 7865da1 [MINOR] Fix Javadoc wrong references

[GitHub] [hudi] vinothchandar commented on pull request #3035: [HUDI-1936] Introduce a optional property for conditional upsert

vinothchandar commented on pull request #3035: URL: https://github.com/apache/hudi/pull/3035#issuecomment-864355261 cc @vingov do you mind taking a review at this, given its a python benefiting change -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 #

[GitHub] [hudi] hudi-bot edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

hudi-bot edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-863021428 ## CI report: * 70b70e9e18dd7717b7567c83f7c07e1c0c6c0086 Azure:

[GitHub] [hudi] chaplinthink opened a new pull request #3115: [MINOR] Fix Javadoc wrong references

chaplinthink opened a new pull request #3115: URL: https://github.com/apache/hudi/pull/3115 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[GitHub] [hudi] hudi-bot commented on pull request #3115: [MINOR] Fix Javadoc wrong references

hudi-bot commented on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864342921 ## CI report: * f76a8482b21f7311a4360331aba4343d9d4352af UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3114: [MINOR] Fix wrong package name

codecov-commenter edited a comment on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Commented] (HUDI-1248) [UMBRELLA] Tests cleanup and fixes

[ https://issues.apache.org/jira/browse/HUDI-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365844#comment-17365844 ] Raymond Xu commented on HUDI-1248: -- Starting from [this

[GitHub] [hudi] hudi-bot edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

hudi-bot edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-863021428 ## CI report: * 8dddccdd400bc19605abbbad78c2889764cff4c2 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3114: [MINOR] Fix wrong package name

codecov-commenter edited a comment on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338505 #

[GitHub] [hudi] hudi-bot edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

hudi-bot edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-863021428 ## CI report: * 8dddccdd400bc19605abbbad78c2889764cff4c2 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3115: [MINOR] Fix Javadoc wrong references

codecov-commenter edited a comment on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864343564 #

[GitHub] [hudi] vinothchandar commented on pull request #3110: [HUDI-1248] Increase timeout for deltaStreamerTestRunner in TestHoodi…

vinothchandar commented on pull request #3110: URL: https://github.com/apache/hudi/pull/3110#issuecomment-864354224 does seem like a bandaid :/, may be necessary for now. any idea why that specific value works? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] vinothchandar commented on pull request #3093: [HUDI-2031] JVM occasionally crashes during compaction when spark speculative execution is enabled

vinothchandar commented on pull request #3093: URL: https://github.com/apache/hudi/pull/3093#issuecomment-864355908 Change looks reasonable. let me think bit more around the scenarios before merging this. Anyway we can write a unit test around this scenario? -- This is an

[GitHub] [hudi] chaplinthink opened a new pull request #3114: [MINOR] Fix wrong package name

chaplinthink opened a new pull request #3114: URL: https://github.com/apache/hudi/pull/3114 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[GitHub] [hudi] hudi-bot edited a comment on pull request #3114: [MINOR] Fix wrong package name

hudi-bot edited a comment on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338167 ## CI report: * b8f7f02c0bc4b9621f79397f45cd62cd2c1c6ec8 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3115: [MINOR] Fix Javadoc wrong references

codecov-commenter edited a comment on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864343564 #

[hudi] branch master updated: [MINOR] Fix wrong package name (#3114)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 5339606 [MINOR] Fix wrong package name (#3114)

[GitHub] [hudi] leesf commented on a change in pull request #3114: [MINOR] Fix wrong package name

leesf commented on a change in pull request #3114: URL: https://github.com/apache/hudi/pull/3114#discussion_r654742890 ## File path: hudi-utilities/src/test/java/org/apache/hudi/utilities/sources/TestSqlSource.java ## @@ -26,8 +26,6 @@ import

[GitHub] [hudi] leesf merged pull request #3114: [MINOR] Fix wrong package name

leesf merged pull request #3114: URL: https://github.com/apache/hudi/pull/3114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] vinothchandar commented on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

vinothchandar commented on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-864356223 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] umehrot2 commented on a change in pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

umehrot2 commented on a change in pull request #2893: URL: https://github.com/apache/hudi/pull/2893#discussion_r654727713 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ## @@ -105,6 +107,24 @@ public

[GitHub] [hudi] hudi-bot edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

hudi-bot edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-863021428 ## CI report: * 50e2f1bbcffbd2db4f16901ac577302ffeb335b4 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

hudi-bot edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-863021428 ## CI report: * 50e2f1bbcffbd2db4f16901ac577302ffeb335b4 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2893: [HUDI-1371] [HUDI-1893] Support metadata based listing for Spark DataSource and Spark SQL

codecov-commenter edited a comment on pull request #2893: URL: https://github.com/apache/hudi/pull/2893#issuecomment-828848333 #

[GitHub] [hudi] codecov-commenter commented on pull request #3115: [MINOR] Fix Javadoc wrong references

codecov-commenter commented on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864343564 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3115?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] hudi-bot edited a comment on pull request #3114: [MINOR] Fix wrong package name

hudi-bot edited a comment on pull request #3114: URL: https://github.com/apache/hudi/pull/3114#issuecomment-864338167 ## CI report: * b8f7f02c0bc4b9621f79397f45cd62cd2c1c6ec8 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3115: [MINOR] Fix Javadoc wrong references

hudi-bot edited a comment on pull request #3115: URL: https://github.com/apache/hudi/pull/3115#issuecomment-864342921 ## CI report: * f76a8482b21f7311a4360331aba4343d9d4352af Azure:

[jira] [Closed] (HUDI-2041) Code reuse in schduling the compaction plan when using Flink

[ https://issues.apache.org/jira/browse/HUDI-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoyuGeng closed HUDI-2041. Resolution: Fixed > Code reuse in schduling the compaction plan when using Flink >

[jira] [Created] (HUDI-2041) Code reuse in schduling the compaction plan when using Flink

XiaoyuGeng created HUDI-2041: Summary: Code reuse in schduling the compaction plan when using Flink Key: HUDI-2041 URL: https://issues.apache.org/jira/browse/HUDI-2041 Project: Apache Hudi

[jira] [Commented] (HUDI-1248) [UMBRELLA] Tests cleanup and fixes

[ https://issues.apache.org/jira/browse/HUDI-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365501#comment-17365501 ] Sagar Sumit commented on HUDI-1248: --- Couple more failures:  {code:java}

[jira] [Commented] (HUDI-1818) Validate and check required option for HoodieTable

[ https://issues.apache.org/jira/browse/HUDI-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365503#comment-17365503 ] 谢波 commented on HUDI-1818: -- 7a5af806cfe54474014db5c70641c0c6269fff03 > Validate and check required option for

[jira] [Resolved] (HUDI-1818) Validate and check required option for HoodieTable

[ https://issues.apache.org/jira/browse/HUDI-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 谢波 resolved HUDI-1818. -- Resolution: Fixed 7a5af806cfe54474014db5c70641c0c6269fff03 > Validate and check required option for HoodieTable >

[jira] [Issue Comment Deleted] (HUDI-1818) Validate and check required option for HoodieTable

[ https://issues.apache.org/jira/browse/HUDI-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 谢波 updated HUDI-1818: - Comment: was deleted (was: 7a5af806cfe54474014db5c70641c0c6269fff03) > Validate and check required option for

[jira] [Comment Edited] (HUDI-1248) [UMBRELLA] Tests cleanup and fixes

[ https://issues.apache.org/jira/browse/HUDI-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365501#comment-17365501 ] Sagar Sumit edited comment on HUDI-1248 at 6/18/21, 1:28 PM: - Couple more

[jira] [Updated] (HUDI-2034) Support explicit partition compaction strategy for flink batch compaction

[ https://issues.apache.org/jira/browse/HUDI-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2034: - Labels: pull-request-available (was: ) > Support explicit partition compaction strategy for

[jira] [Updated] (HUDI-2043) HoodieDefaultTimeline$filterPendingCompactionTImeline() method have wrong filter condition

[ https://issues.apache.org/jira/browse/HUDI-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2043: - Labels: pull-request-available (was: ) > HoodieDefaultTimeline$filterPendingCompactionTImeline()

[GitHub] [hudi] hudi-bot commented on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

hudi-bot commented on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-864208731 ## CI report: * 20890458e5ec2cce973c33ad760447d00e9ae2db UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot edited a comment on pull request #3083: [HUDI-2016] Fixed bootstrap of Metadata Table when some actions are in progress.

hudi-bot edited a comment on pull request #3083: URL: https://github.com/apache/hudi/pull/3083#issuecomment-861654193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[jira] [Updated] (HUDI-2042) Compare the field object directly in OverwriteWithLatestAvroPayload

[ https://issues.apache.org/jira/browse/HUDI-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2042: - Labels: pull-request-available (was: ) > Compare the field object directly in

[jira] [Created] (HUDI-2044) Extend support for rocked for spoilable map to all consumers of ExternalSpillableMap

Rajesh Mahindra created HUDI-2044: - Summary: Extend support for rocked for spoilable map to all consumers of ExternalSpillableMap Key: HUDI-2044 URL: https://issues.apache.org/jira/browse/HUDI-2044

[jira] [Assigned] (HUDI-2044) Extend support for rocked for spoilable map to all consumers of ExternalSpillableMap

[ https://issues.apache.org/jira/browse/HUDI-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra reassigned HUDI-2044: - Assignee: Rajesh Mahindra > Extend support for rocked for spoilable map to all consumers

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3110: [HUDI-1248] Increase timeout for deltaStreamerTestRunner in TestHoodi…

codecov-commenter edited a comment on pull request #3110: URL: https://github.com/apache/hudi/pull/3110#issuecomment-864030127 #

[GitHub] [hudi] hudi-bot commented on pull request #3106: [HUDI-2040] Make flink writer as exactly-once by default

hudi-bot commented on pull request #3106: URL: https://github.com/apache/hudi/pull/3106#issuecomment-863726728 ## CI report: * 18000d3cdc8dfc387b0ae86f46e4a3134177d1db UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-2040) Make flink writer as exactly-once by default

[ https://issues.apache.org/jira/browse/HUDI-2040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2040: - Labels: pull-request-available (was: ) > Make flink writer as exactly-once by default >

[jira] [Updated] (HUDI-1248) [UMBRELLA] Tests cleanup and fixes

[ https://issues.apache.org/jira/browse/HUDI-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1248: - Labels: hudi-umbrellas pull-request-available (was: hudi-umbrellas) > [UMBRELLA] Tests cleanup

[GitHub] [hudi] liujinhui1994 opened a new pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

liujinhui1994 opened a new pull request #2438: URL: https://github.com/apache/hudi/pull/2438 DeltaStreamer kafka source supports consuming from specified timestamp ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review

[GitHub] [hudi] codecov-commenter commented on pull request #3109: [HUDI-2043] HoodieDefaultTimeline$filterPendingCompactionTImeline() m…

codecov-commenter commented on pull request #3109: URL: https://github.com/apache/hudi/pull/3109#issuecomment-864012960 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3109?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] hudi-bot edited a comment on pull request #3100: [HUDI-2034] Support explicit partition compaction strategy for flink …

hudi-bot edited a comment on pull request #3100: URL: https://github.com/apache/hudi/pull/3100#issuecomment-862905306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] codecov-commenter commented on pull request #3111: Fix KafkaAvroSchemaDeserializer to not rely on reflection

codecov-commenter commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-864241005 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3111?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] sbernauer opened a new pull request #3111: [WIP] Fix KafkaAvroSchemaDeserializer to not rely on reflection

sbernauer opened a new pull request #3111: URL: https://github.com/apache/hudi/pull/3111 Hi Hudi Team! ## What is the purpose of the pull request We are using Deltastreamer to ingest events from kafka into an S3 store. We had some incident this week after some of our producers

[jira] [Updated] (HUDI-2038) Rollback pending compaction when schedule new compaction

[ https://issues.apache.org/jira/browse/HUDI-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2038: - Labels: pull-request-available (was: ) > Rollback pending compaction when schedule new

[GitHub] [hudi] hudi-bot edited a comment on pull request #3112: [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in SpillableMap

hudi-bot edited a comment on pull request #3112: URL: https://github.com/apache/hudi/pull/3112#issuecomment-864259550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] hudi-bot commented on pull request #2049: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

hudi-bot commented on pull request #2049: URL: https://github.com/apache/hudi/pull/2049#issuecomment-864282756 ## CI report: * 7f806744cd2ce83e68298391608fedf5e2a067f3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot edited a comment on pull request #2049: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

hudi-bot edited a comment on pull request #2049: URL: https://github.com/apache/hudi/pull/2049#issuecomment-864282756 ## CI report: * 7f806744cd2ce83e68298391608fedf5e2a067f3 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2049: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

hudi-bot edited a comment on pull request #2049: URL: https://github.com/apache/hudi/pull/2049#issuecomment-864282756 ## CI report: * 7f806744cd2ce83e68298391608fedf5e2a067f3 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

codecov-commenter edited a comment on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-850284847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] swuferhong opened a new pull request #3109: [HUDI-2043] HoodieDefaultTimeline$filterPendingCompactionTImeline() m…

swuferhong opened a new pull request #3109: URL: https://github.com/apache/hudi/pull/3109 …ethod have wrong filter condition ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull

[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #3086: [HUDI-1776] Support AlterCommand For Hoodie

pengzhiwei2018 commented on a change in pull request #3086: URL: https://github.com/apache/hudi/pull/3086#discussion_r654097033 ## File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTable.scala ## @@ -0,0 +1,117 @@ +/* + * Licensed

[GitHub] [hudi] hudi-bot edited a comment on pull request #3111: [WIP] Fix KafkaAvroSchemaDeserializer to not rely on reflection

hudi-bot edited a comment on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-864054252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3100: [HUDI-2034] Support explicit partition compaction strategy for flink …

codecov-commenter edited a comment on pull request #3100: URL: https://github.com/apache/hudi/pull/3100#issuecomment-862906759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

hudi-bot edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-864208731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[jira] [Updated] (HUDI-2028) Implement RockDbBasedMap as an alternate to DiskBasedMap in SpillableMap

[ https://issues.apache.org/jira/browse/HUDI-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2028: - Labels: pull-request-available (was: ) > Implement RockDbBasedMap as an alternate to

[GitHub] [hudi] yanghua merged pull request #3106: [HUDI-2040] Make flink writer as exactly-once by default

yanghua merged pull request #3106: URL: https://github.com/apache/hudi/pull/3106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] nsivabalan commented on a change in pull request #2915: [HUDI-251] Adds JDBC source support for DeltaStreamer

nsivabalan commented on a change in pull request #2915: URL: https://github.com/apache/hudi/pull/2915#discussion_r654115003 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/JdbcSource.java ## @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2915: [HUDI-251] Adds JDBC source support for DeltaStreamer

codecov-commenter edited a comment on pull request #2915: URL: https://github.com/apache/hudi/pull/2915#issuecomment-832697076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #3111: [WIP] Fix KafkaAvroSchemaDeserializer to not rely on reflection

hudi-bot commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-864054252 ## CI report: * 8985d48eb28ad448a6ec6607fd20c3efcc92b3d2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] codecov-commenter commented on pull request #3110: [HUDI-1248] Increase timeout for deltaStreamerTestRunner in TestHoodi…

codecov-commenter commented on pull request #3110: URL: https://github.com/apache/hudi/pull/3110#issuecomment-864030127 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3110?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] minihippo opened a new pull request #3108: [HUDI-2042] Compare the field object directly in OverwriteWithLatestA…

minihippo opened a new pull request #3108: URL: https://github.com/apache/hudi/pull/3108 …vroPayload ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is

[GitHub] [hudi] rmahindra321 opened a new pull request #3112: Implement RockDbBasedMap as an alternate to DiskBasedMap in SpillableMap

rmahindra321 opened a new pull request #3112: URL: https://github.com/apache/hudi/pull/3112 ## What is the purpose of the pull request This pull request adds a new alternative based on RockDb for the Disk Based Map that is used within the ExternalSpillableMap. Our benchmark results

[GitHub] [hudi] dwshmilyss opened a new issue #3107: [SUPPORT]

dwshmilyss opened a new issue #3107: URL: https://github.com/apache/hudi/issues/3107 Use Spark to Hudi and JVM Metaspace OOM - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? No - Join the mailing list to engage in conversations and get

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-863968623 deltaSync should reset this(...kafka.checkpoint.type) configuration (similar to how we reset checkpoints) In this way, we may need to store this in the metadata file. If

[GitHub] [hudi] hudi-bot edited a comment on pull request #3109: [HUDI-2043] HoodieDefaultTimeline$filterPendingCompactionTImeline() m…

hudi-bot edited a comment on pull request #3109: URL: https://github.com/apache/hudi/pull/3109#issuecomment-863900788 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] hudi-bot edited a comment on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

hudi-bot edited a comment on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-863310563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hudi] hudi-bot edited a comment on pull request #3110: [HUDI-1248] Increase timeout for deltaStreamerTestRunner in TestHoodi…

hudi-bot edited a comment on pull request #3110: URL: https://github.com/apache/hudi/pull/3110#issuecomment-863909633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[jira] [Updated] (HUDI-2036) Move the compaction plan scheduling out of flink writer coordinator

[ https://issues.apache.org/jira/browse/HUDI-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2036: - Labels: pull-request-available (was: ) > Move the compaction plan scheduling out of flink writer

[GitHub] [hudi] hudi-bot commented on pull request #3110: [HUDI-1248] Increase timeout for deltaStreamerTestRunner in TestHoodi…

hudi-bot commented on pull request #3110: URL: https://github.com/apache/hudi/pull/3110#issuecomment-863909633 ## CI report: * fcf4d5c2fe72ad8dfde06236344770426a53ee97 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

  1   2   >