[GitHub] [hudi] vinothchandar commented on pull request #2487: [WIP HUDI-53] Adding Record Level Index based on hoodie backed table

2021-06-23 Thread GitBox
vinothchandar commented on pull request #2487: URL: https://github.com/apache/hudi/pull/2487#issuecomment-867210438 yes for random uuids, bloomfilters/min/max is less helpful. @lw309637554 At a high level, we need some foundational work around metadata table to add record level

[GitHub] [hudi] vinothchandar commented on issue #2992: [SUPPORT] Insert_Override Api not working as expected in Hudi 0.7.0

2021-06-23 Thread GitBox
vinothchandar commented on issue #2992: URL: https://github.com/apache/hudi/issues/2992#issuecomment-867206424 folks, whats the next step here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (HUDI-1628) Improve data locality during ingestion

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1628: - Fix Version/s: 0.10.0 > Improve data locality during ingestion >

[jira] [Commented] (HUDI-1628) Improve data locality during ingestion

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368485#comment-17368485 ] Vinoth Chandar commented on HUDI-1628: -- [~thirumalai.raj] apologies for the delay. This is up for

[jira] [Updated] (HUDI-1912) Presto defaults to GenericHiveRecordCursor for all Hudi tables

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1912: - Affects Version/s: 0.7.0 > Presto defaults to GenericHiveRecordCursor for all Hudi tables >

[jira] [Updated] (HUDI-1912) Presto defaults to GenericHiveRecordCursor for all Hudi tables

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1912: - Fix Version/s: 0.9.0 > Presto defaults to GenericHiveRecordCursor for all Hudi tables >

[jira] [Commented] (HUDI-1827) Add ORC support in Bootstrap Op

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368480#comment-17368480 ] Vinoth Chandar commented on HUDI-1827: -- [~manasaks] please let us know if you need more help to take

[jira] [Commented] (HUDI-818) Optimize the default value of hoodie.memory.merge.max.size option

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368479#comment-17368479 ] Vinoth Chandar commented on HUDI-818: - [~rmahindra] seems like we are good here, already. Close this

[jira] [Commented] (HUDI-1537) Move validation of file listings to something that happens before each write

2021-06-23 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368478#comment-17368478 ] Vinoth Chandar commented on HUDI-1537: -- So there is no validation machinery anymore? > Move

[GitHub] [hudi] vinothchandar commented on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-23 Thread GitBox
vinothchandar commented on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-867185527 Thanks. Thats where I got stuck as well. Thanks for rebasing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3117: [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in ExternalSpillableMap

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3117: URL: https://github.com/apache/hudi/pull/3117#issuecomment-864499795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3117: [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in ExternalSpillableMap

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3117: URL: https://github.com/apache/hudi/pull/3117#issuecomment-864499549 ## CI report: * dd248a3ab08bd58980bf731e24d8e403b6f64c17 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3117: [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in ExternalSpillableMap

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3117: URL: https://github.com/apache/hudi/pull/3117#issuecomment-864499795 #

[GitHub] [hudi] hudi-bot edited a comment on pull request #3117: [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in ExternalSpillableMap

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3117: URL: https://github.com/apache/hudi/pull/3117#issuecomment-864499549 ## CI report: * 93dad0545f41eab1d09086cacf9852c30f4d02b2 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3117: [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in ExternalSpillableMap

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3117: URL: https://github.com/apache/hudi/pull/3117#issuecomment-864499549 ## CI report: * 93dad0545f41eab1d09086cacf9852c30f4d02b2 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-867078369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] rubenssoto closed issue #3143: [SUPPORT] Skew partition on simple count in a Hudi Table

2021-06-23 Thread GitBox
rubenssoto closed issue #3143: URL: https://github.com/apache/hudi/issues/3143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] rubenssoto commented on issue #3143: [SUPPORT] Skew partition on simple count in a Hudi Table

2021-06-23 Thread GitBox
rubenssoto commented on issue #3143: URL: https://github.com/apache/hudi/issues/3143#issuecomment-867100912 I think the problem was my data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-867071378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] rubenssoto opened a new issue #3143: [SUPPORT] Skew partition on simple count in a Hudi Table

2021-06-23 Thread GitBox
rubenssoto opened a new issue #3143: URL: https://github.com/apache/hudi/issues/3143 Hello Guys, I'm trying to run a simples spark query on a Hudi dataset but took a long to finished and I realized that exist very skew partitions, but I didnt understand why. The table has

[GitHub] [hudi] codecov-commenter commented on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-23 Thread GitBox
codecov-commenter commented on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-867078369 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3142?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] vburenin commented on pull request #3111: Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-23 Thread GitBox
vburenin commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-867072324 Test are definitely not ideal, I had a lot of failures around different places when the change was just to reduce info to debug level. -- This is an automated message from the

[GitHub] [hudi] codecov-commenter commented on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
codecov-commenter commented on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-867071378 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3141?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] sbernauer commented on pull request #3111: Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-23 Thread GitBox
sbernauer commented on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-867069605 The previous test run failed because of the hudi-utilities. The current testrun failed two other tests, the hudi-utilities were fine. How do we progress here? I think the tests

[GitHub] [hudi] hudi-bot edited a comment on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-866975175 ## CI report: * 556076d5963d883fe0982b7d72ba3af9094e4939 UNKNOWN * 0409b46762a7eff3c1029095baaf769e8c4caf21 Azure:

[GitHub] [hudi] zhedoubushishi commented on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-23 Thread GitBox
zhedoubushishi commented on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-867034257 @vinothchandar I resolved the merge conflicts. Can you take a final review? There's one test failed but it seems even using master branch that test sometimes also failed.

[GitHub] [hudi] hudi-bot edited a comment on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-866996072 ## CI report: * 4a651f8d6b32f63838d8317c9f083508c17e4458 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-866975175 ## CI report: * f255a19f3d7f99666d3501e4942dd704563f1bd2 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-866996072 ## CI report: * 4a651f8d6b32f63838d8317c9f083508c17e4458 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-23 Thread GitBox
hudi-bot commented on pull request #3142: URL: https://github.com/apache/hudi/pull/3142#issuecomment-866996072 ## CI report: * 4a651f8d6b32f63838d8317c9f083508c17e4458 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-1483) async clustering for deltastreamer

2021-06-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1483: - Labels: pull-request-available (was: ) > async clustering for deltastreamer >

[GitHub] [hudi] codope opened a new pull request #3142: [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

2021-06-23 Thread GitBox
codope opened a new pull request #3142: URL: https://github.com/apache/hudi/pull/3142 ## What is the purpose of the pull request This pull request adds async clustering support for HoodieDeltaStreamer and Spark streaming writes to Hudi table. ## Brief change log -

[GitHub] [hudi] hudi-bot edited a comment on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-866975175 ## CI report: * f255a19f3d7f99666d3501e4942dd704563f1bd2 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-866975175 ## CI report: * f255a19f3d7f99666d3501e4942dd704563f1bd2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
hudi-bot commented on pull request #3141: URL: https://github.com/apache/hudi/pull/3141#issuecomment-866975175 ## CI report: * f255a19f3d7f99666d3501e4942dd704563f1bd2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-2064) Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2064: - Labels: pull-request-available (was: ) > Fix

[GitHub] [hudi] leesf opened a new pull request #3141: [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread GitBox
leesf opened a new pull request #3141: URL: https://github.com/apache/hudi/pull/3141 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull

[jira] [Created] (HUDI-2064) Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded

2021-06-23 Thread leesf (Jira)
leesf created HUDI-2064: --- Summary: Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded Key: HUDI-2064 URL: https://issues.apache.org/jira/browse/HUDI-2064 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] karan867 edited a comment on issue #3077: [SUPPORT] Large latencies in hudi writes using upsert mode.

2021-06-23 Thread GitBox
karan867 edited a comment on issue #3077: URL: https://github.com/apache/hudi/issues/3077#issuecomment-866913205 @n3nash Thank you for the suggestions. I tried the parameters you suggested. * Setting hoodie.parquet.small.file.limit to zero did not make much difference and the

[GitHub] [hudi] karan867 commented on issue #3077: [SUPPORT] Large latencies in hudi writes using upsert mode.

2021-06-23 Thread GitBox
karan867 commented on issue #3077: URL: https://github.com/apache/hudi/issues/3077#issuecomment-866913205 @n3nash Thank you for the suggestions. I tried the parameters you suggested. * Setting hoodie.parquet.small.file.limit to zero did not make much difference and the subsequent

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 #

[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * 0de8f0708cee42f6c5272762c4749b52493e76e3 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * b705768fe421e0dcfb71834ce2045a8104d1dbf6 Azure:

[GitHub] [hudi] yuzhaojing commented on a change in pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
yuzhaojing commented on a change in pull request #3138: URL: https://github.com/apache/hudi/pull/3138#discussion_r657146346 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/profile/WriteProfiles.java ## @@ -150,8 +150,8 @@ public static void

[GitHub] [hudi] pengzhiwei2018 edited a comment on pull request #3105: [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator

2021-06-23 Thread GitBox
pengzhiwei2018 edited a comment on pull request #3105: URL: https://github.com/apache/hudi/pull/3105#issuecomment-866859726 Hi @Danny, I am fixing the issue 2 about schema evolution.  The issue 1 is  about TestHoodieBackedMetadata#testOnlyValidPartitionsAdded which may need @prashantwason

[GitHub] [hudi] pengzhiwei2018 commented on pull request #3105: [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator

2021-06-23 Thread GitBox
pengzhiwei2018 commented on pull request #3105: URL: https://github.com/apache/hudi/pull/3105#issuecomment-866859726 Hi @Danny, I am fixing the issue 2 about schema evolution.  The issue 1 is  about TestHoodieBackedMetadata#testOnlyValidPartitionsAdded which may need @prashantwason to

[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * 4d9ebad202e9ef439f2bd3958ec746003188baa0 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866598468 ## CI report: * 4d9ebad202e9ef439f2bd3958ec746003188baa0 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 94b89ac962a190f1a10f534755f0bee0e54c5c2e Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 94b89ac962a190f1a10f534755f0bee0e54c5c2e Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 #

[hudi] branch master updated: [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator (#3105)

2021-06-23 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 380518e [HUDI-2038] Support rollback inflight

[GitHub] [hudi] leesf merged pull request #3105: [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator

2021-06-23 Thread GitBox
leesf merged pull request #3105: URL: https://github.com/apache/hudi/pull/3105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865868967 #

[GitHub] [hudi] veenaypatil commented on pull request #3136: [HUDI-2060] Added tests for KafkaOffsetGen

2021-06-23 Thread GitBox
veenaypatil commented on pull request #3136: URL: https://github.com/apache/hudi/pull/3136#issuecomment-866795227 @vinothchandar @leesf can you pls review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] danny0405 commented on pull request #3105: [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator

2021-06-23 Thread GitBox
danny0405 commented on pull request #3105: URL: https://github.com/apache/hudi/pull/3105#issuecomment-866791909 @leesf Can we merge this first, the tests failure are not caused by this patch and @pengzhiwei2018 was fixing it. -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 94b89ac962a190f1a10f534755f0bee0e54c5c2e Azure:

[GitHub] [hudi] yanghua commented on a change in pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
yanghua commented on a change in pull request #3138: URL: https://github.com/apache/hudi/pull/3138#discussion_r657032825 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/profile/WriteProfiles.java ## @@ -150,8 +150,8 @@ public static void clean(String

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866741631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 8afc24ac9d323debf0d861f73b2b8e0c4df487e3 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3133: [HUDI-2053] Insert Static Partition With DateType Return Incorrect P…

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3133: URL: https://github.com/apache/hudi/pull/3133#issuecomment-865758353 ## CI report: * 8afc24ac9d323debf0d861f73b2b8e0c4df487e3 Azure:

[jira] [Resolved] (HUDI-1776) Support AlterCommand For Hoodie

2021-06-23 Thread pengzhiwei (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengzhiwei resolved HUDI-1776. -- Resolution: Done > Support AlterCommand For Hoodie > --- > >

[jira] [Resolved] (HUDI-1883) Support Truncate Table For Hoodie

2021-06-23 Thread pengzhiwei (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pengzhiwei resolved HUDI-1883. -- Resolution: Done > Support Truncate Table For Hoodie > - > >

[jira] [Updated] (HUDI-2063) Add Doc For Spark Sql Integrates With Hudi

2021-06-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2063: - Labels: pull-request-available (was: ) > Add Doc For Spark Sql Integrates With Hudi >

[GitHub] [hudi] pengzhiwei2018 opened a new pull request #3140: [HUDI-2063] Add Doc For Spark Sql Integrates With Hudi

2021-06-23 Thread GitBox
pengzhiwei2018 opened a new pull request #3140: URL: https://github.com/apache/hudi/pull/3140 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[jira] [Created] (HUDI-2063) Add Doc For Spark Sql Integrates With Hudi

2021-06-23 Thread pengzhiwei (Jira)
pengzhiwei created HUDI-2063: Summary: Add Doc For Spark Sql Integrates With Hudi Key: HUDI-2063 URL: https://issues.apache.org/jira/browse/HUDI-2063 Project: Apache Hudi Issue Type: Sub-task

[GitHub] [hudi] codecov-commenter commented on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
codecov-commenter commented on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866741631 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3139?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-850284847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866671675 ## CI report: * 29536de9ee87bbc207daefef84b80b23c52fca9d Azure:

[GitHub] [hudi] codecov-commenter commented on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
codecov-commenter commented on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-866723821 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3138?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 0b3283067503c07bd1168cd88cae6d4bf8f215ba Azure:

[jira] [Updated] (HUDI-2062) Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread yuzhaojing (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuzhaojing updated HUDI-2062: - Description: The function WriteProfiles #getCommitMetadataSafely expect get instant safely, if instant 

[jira] [Updated] (HUDI-2062) Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread yuzhaojing (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuzhaojing updated HUDI-2062: - Description:   > Catch IOException in WriteProfiles #getCommitMetadataSafely >

[jira] [Updated] (HUDI-2062) Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread yuzhaojing (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuzhaojing updated HUDI-2062: - Description: (was: Catch IOException in WriteProfiles #getCommitMetadataSafely) > Catch IOException

[GitHub] [hudi] hudi-bot edited a comment on pull request #2833: [HUDI-89] Add configOption & refactor Hudi configuration framework

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #2833: URL: https://github.com/apache/hudi/pull/2833#issuecomment-864208731 ## CI report: * 78dd4297c7503795c05647afad048ed3d809e958 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-863310563 ## CI report: * ea5ed9da433064022a69e06c98f58fc10c09e8b6 Azure:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3111: Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-864241005 #

[GitHub] [hudi] liujinhui1994 closed pull request #2211: [WIP]Add configuration to make HoodieBootstrap support ignoring file suffix

2021-06-23 Thread GitBox
liujinhui1994 closed pull request #2211: URL: https://github.com/apache/hudi/pull/2211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [hudi] hudi-bot edited a comment on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866671675 ## CI report: * 29536de9ee87bbc207daefef84b80b23c52fca9d Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3111: Fix KafkaAvroSchemaDeserializer to not rely on reflection

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3111: URL: https://github.com/apache/hudi/pull/3111#issuecomment-864054252 ## CI report: * 8b47c4d4f0bdbfbd7ed0f8038a9e145192632674 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
hudi-bot commented on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866671675 ## CI report: * 29536de9ee87bbc207daefef84b80b23c52fca9d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot edited a comment on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
hudi-bot edited a comment on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 0b3283067503c07bd1168cd88cae6d4bf8f215ba Azure:

[GitHub] [hudi] xiarixiaoyao commented on pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
xiarixiaoyao commented on pull request #3139: URL: https://github.com/apache/hudi/pull/3139#issuecomment-866670997 we add a new UT file, since TestCowDataSource.scala is too large, add this ut to it will lead checkstyle failed! -- This is an automated message from the Apache Git

[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
xiarixiaoyao commented on a change in pull request #3139: URL: https://github.com/apache/hudi/pull/3139#discussion_r656909752 ## File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestCOWDataSourceExtend.scala ## @@ -0,0 +1,80 @@ +/* Review

[jira] [Updated] (HUDI-2058) support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2058: - Labels: pull-request-available (was: ) > support incremental query for

[GitHub] [hudi] xiarixiaoyao opened a new pull request #3139: [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-23 Thread GitBox
xiarixiaoyao opened a new pull request #3139: URL: https://github.com/apache/hudi/pull/3139 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the

[jira] [Commented] (HUDI-2062) Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367982#comment-17367982 ] vinoyang commented on HUDI-2062: Hi [~yuzhaojing] What's the exception and stack trace did you meet? Can

[GitHub] [hudi] hudi-bot commented on pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
hudi-bot commented on pull request #3138: URL: https://github.com/apache/hudi/pull/3138#issuecomment-87254 ## CI report: * 0b3283067503c07bd1168cd88cae6d4bf8f215ba UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] yanghua commented on pull request #3130: [HUDI-1826] Add ORC support in HoodieSnapshotExporter

2021-06-23 Thread GitBox
yanghua commented on pull request #3130: URL: https://github.com/apache/hudi/pull/3130#issuecomment-86237 @vaibhav-sinha What's your Jira ID? Can you share it with me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Closed] (HUDI-1826) Add ORC support in HoodieSnapshotExporter

2021-06-23 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-1826. -- Fix Version/s: 0.9.0 Resolution: Fixed 43b9c1fa1caf97f6fb2baf68e350615541ea0a0c > Add ORC support in

[GitHub] [hudi] yanghua merged pull request #3130: [HUDI-1826] Add ORC support in HoodieSnapshotExporter

2021-06-23 Thread GitBox
yanghua merged pull request #3130: URL: https://github.com/apache/hudi/pull/3130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[hudi] branch master updated: [HUDI-1826] Add ORC support in HoodieSnapshotExporter (#3130)

2021-06-23 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 43b9c1f [HUDI-1826] Add ORC support in

[jira] [Updated] (HUDI-2062) Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2062: - Labels: pull-request-available (was: ) > Catch IOException in WriteProfiles

[GitHub] [hudi] yuzhaojing opened a new pull request #3138: [HUDI-2062] Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread GitBox
yuzhaojing opened a new pull request #3138: URL: https://github.com/apache/hudi/pull/3138 Catch IOException in WriteProfiles #getCommitMetadataSafely ## What is the purpose of the pull request *(For example: This pull request adds quick-start document.)* ## Brief change

[jira] [Created] (HUDI-2062) Catch IOException in WriteProfiles #getCommitMetadataSafely

2021-06-23 Thread yuzhaojing (Jira)
yuzhaojing created HUDI-2062: Summary: Catch IOException in WriteProfiles #getCommitMetadataSafely Key: HUDI-2062 URL: https://issues.apache.org/jira/browse/HUDI-2062 Project: Apache Hudi Issue

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3137: [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table

2021-06-23 Thread GitBox
codecov-commenter edited a comment on pull request #3137: URL: https://github.com/apache/hudi/pull/3137#issuecomment-866645203 #

<    1   2   3   >