[jira] [Updated] (HUDI-1413) Need binary release of Hudi to distribute tools like hudi-cli.sh and hudi-sync

2021-06-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1413: -- Labels: sev:critical (was: ) > Need binary release of Hudi to distribute tools like

[jira] [Assigned] (HUDI-1413) Need binary release of Hudi to distribute tools like hudi-cli.sh and hudi-sync

2021-06-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-1413: - Assignee: sivabalan narayanan > Need binary release of Hudi to distribute tools

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861892572 #

[GitHub] [hudi] n3nash closed issue #3008: [SUPPORT] Hive Sync issues on deletes and non partitioned table

2021-06-15 Thread GitBox
n3nash closed issue #3008: URL: https://github.com/apache/hudi/issues/3008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] n3nash commented on issue #3008: [SUPPORT] Hive Sync issues on deletes and non partitioned table

2021-06-15 Thread GitBox
n3nash commented on issue #3008: URL: https://github.com/apache/hudi/issues/3008#issuecomment-862012567 @pranotishanbhag We will add the right documentation for the GlobalDeleteKeyGenerator. Can you please expand on what code changes are needed for the second issue ? -- This is an

[GitHub] [hudi] n3nash commented on issue #2265: Arrays with nulls in them result in broken parquet files

2021-06-15 Thread GitBox
n3nash commented on issue #2265: URL: https://github.com/apache/hudi/issues/2265#issuecomment-862016892 The fix has been landed and a FAQ has been added here -> https://cwiki.apache.org/confluence/display/HUDI/FAQ?focusedCommentId=181310323#comment-181310323 -- This is an automated

[GitHub] [hudi] n3nash closed issue #2265: Arrays with nulls in them result in broken parquet files

2021-06-15 Thread GitBox
n3nash closed issue #2265: URL: https://github.com/apache/hudi/issues/2265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] hudi-bot edited a comment on pull request #3035: [HUDI-1936] Introduce a optional property for conditional upsert

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3035: URL: https://github.com/apache/hudi/pull/3035#issuecomment-862017744 ## CI report: * 26dadb6627c90c9f06e66fba0b8bd24e5579665f Azure:

[jira] [Updated] (HUDI-1492) Handle DeltaWriteStat correctly for storage schemes that support appends

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1492: - Priority: Blocker (was: Major) > Handle DeltaWriteStat correctly for storage schemes that

[jira] [Commented] (HUDI-1042) [Umbrella] Support clustering on filegroups

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364073#comment-17364073 ] Vinoth Chandar commented on HUDI-1042: -- [~uditme] Can you please add any suggestions around config

[GitHub] [hudi] atharshah-ea edited a comment on issue #2522: [SUPPORT] Avoid UPSERT unchanged records from source

2021-06-15 Thread GitBox
atharshah-ea edited a comment on issue #2522: URL: https://github.com/apache/hudi/issues/2522#issuecomment-862002125 Hi, also looking for an example of how to specify the DefaultHoodieRecordPayload. Setting the following option did not work for us:

[jira] [Created] (HUDI-2026) Add documentation for GlobalDeleteKeyGenerator

2021-06-15 Thread Nishith Agarwal (Jira)
Nishith Agarwal created HUDI-2026: - Summary: Add documentation for GlobalDeleteKeyGenerator Key: HUDI-2026 URL: https://issues.apache.org/jira/browse/HUDI-2026 Project: Apache Hudi Issue

[GitHub] [hudi] hudi-bot commented on pull request #2210: [HUDI-1348] Provide option to clean up DFS sources

2021-06-15 Thread GitBox
hudi-bot commented on pull request #2210: URL: https://github.com/apache/hudi/pull/2210#issuecomment-862028641 ## CI report: * b845e34d11e4e44e2b41e2089349baddc3a10b80 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2210: [HUDI-1348] Provide option to clean up DFS sources

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #2210: URL: https://github.com/apache/hudi/pull/2210#issuecomment-862054362 #

[jira] [Updated] (HUDI-1047) Support asynchronize clustering in CoW mode

2021-06-15 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1047: -- Fix Version/s: 0.9.0 > Support asynchronize clustering in CoW mode >

[GitHub] [hudi] codecov-commenter commented on pull request #2210: [HUDI-1348] Provide option to clean up DFS sources

2021-06-15 Thread GitBox
codecov-commenter commented on pull request #2210: URL: https://github.com/apache/hudi/pull/2210#issuecomment-862054362 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2210?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[jira] [Updated] (HUDI-1048) Support Asynchronize clustering in MoR mode

2021-06-15 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1048: -- Priority: Blocker (was: Major) > Support Asynchronize clustering in MoR mode >

[jira] [Updated] (HUDI-1048) Support Asynchronize clustering in MoR mode

2021-06-15 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1048: -- Fix Version/s: 0.9.0 > Support Asynchronize clustering in MoR mode >

[jira] [Created] (HUDI-2027) Certify bulk_insert row writing for COW and MOR w/ test suite infra

2021-06-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2027: - Summary: Certify bulk_insert row writing for COW and MOR w/ test suite infra Key: HUDI-2027 URL: https://issues.apache.org/jira/browse/HUDI-2027 Project:

[GitHub] [hudi] n3nash commented on issue #3065: Not in Marker Dir occurs when I write to HDFS using Spark

2021-06-15 Thread GitBox
n3nash commented on issue #3065: URL: https://github.com/apache/hudi/issues/3065#issuecomment-862009477 @wangfeigithub Are you trying to upgrade from a previous older version of Hudi or are you directly writing new files using 0.8 ? -- This is an automated message from the Apache Git

[jira] [Updated] (HUDI-1309) Listing Metadata unreadable in S3 as the log block is deemed corrupted

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1309: - Fix Version/s: 0.9.0 > Listing Metadata unreadable in S3 as the log block is deemed corrupted >

[GitHub] [hudi] n3nash commented on issue #1845: [SUPPORT] Support for Schema evolution. Facing an error

2021-06-15 Thread GitBox
n3nash commented on issue #1845: URL: https://github.com/apache/hudi/issues/1845#issuecomment-862016359 @nsivabalan Can you please reply above ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] nsivabalan commented on a change in pull request #3035: [HUDI-1936] Introduce a optional property for conditional upsert

2021-06-15 Thread GitBox
nsivabalan commented on a change in pull request #3035: URL: https://github.com/apache/hudi/pull/3035#discussion_r652332012 ## File path: hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteWithCustomAvroPayload.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to the

[jira] [Assigned] (HUDI-1309) Listing Metadata unreadable in S3 as the log block is deemed corrupted

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar reassigned HUDI-1309: Assignee: Nishith Agarwal > Listing Metadata unreadable in S3 as the log block is deemed

[jira] [Updated] (HUDI-1500) Support incrementally reading clustering commit via Spark Datasource/DeltaStreamer

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1500: - Summary: Support incrementally reading clustering commit via Spark Datasource/DeltaStreamer

[GitHub] [hudi] atharshah-ea commented on issue #2522: [SUPPORT] Avoid UPSERT unchanged records from source

2021-06-15 Thread GitBox
atharshah-ea commented on issue #2522: URL: https://github.com/apache/hudi/issues/2522#issuecomment-862002125 Hi, also looking for an example of how to specify the DefaultHoodieRecordPayload. Setting the following option did not work for us: **'hoodie.datasource.write.payload.class':

[GitHub] [hudi] hudi-bot edited a comment on pull request #3086: [HUDI-1776] Support AlterCommand For Hoodie

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3086: URL: https://github.com/apache/hudi/pull/3086#issuecomment-861654333 ## CI report: * c041853f41119f23760388d1ab5c7173fe22936b Azure:

[GitHub] [hudi] n3nash closed issue #1829: [SUPPORT] S3 slow file listing causes Hudi read performance.

2021-06-15 Thread GitBox
n3nash closed issue #1829: URL: https://github.com/apache/hudi/issues/1829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] n3nash commented on issue #1829: [SUPPORT] S3 slow file listing causes Hudi read performance.

2021-06-15 Thread GitBox
n3nash commented on issue #1829: URL: https://github.com/apache/hudi/issues/1829#issuecomment-862016095 With 0.7.0, one can set `hoodie.metadata.enable` to true to eliminate issues due to file listings. Closing this ticket now. -- This is an automated message from the Apache Git

[GitHub] [hudi] hudi-bot edited a comment on pull request #3035: [HUDI-1936] Introduce a optional property for conditional upsert

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3035: URL: https://github.com/apache/hudi/pull/3035#issuecomment-862017744 ## CI report: * 26dadb6627c90c9f06e66fba0b8bd24e5579665f Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2210: [HUDI-1348] Provide option to clean up DFS sources

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #2210: URL: https://github.com/apache/hudi/pull/2210#issuecomment-862028641 ## CI report: * b845e34d11e4e44e2b41e2089349baddc3a10b80 Azure:

[jira] [Updated] (HUDI-1706) Test flakiness w/ multiwriter test

2021-06-15 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1706: -- Priority: Blocker (was: Major) > Test flakiness w/ multiwriter test >

[jira] [Updated] (HUDI-1706) Test flakiness w/ multiwriter test

2021-06-15 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishith Agarwal updated HUDI-1706: -- Fix Version/s: 0.9.0 > Test flakiness w/ multiwriter test > --

[jira] [Updated] (HUDI-1048) Support Asynchronize clustering in MoR mode

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1048: - Status: Open (was: New) > Support Asynchronize clustering in MoR mode >

[jira] [Updated] (HUDI-2025) Ensure parity between row writer bulk_insert and rdd based bulk_insert

2021-06-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2025: -- Description: Ensure parity between row writer bulk_insert and rdd based bulk_insert

[jira] [Updated] (HUDI-2025) Ensure parity between row writer bulk_insert and rdd based bulk_insert

2021-06-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2025: -- Summary: Ensure parity between row writer bulk_insert and rdd based bulk_insert (was:

[GitHub] [hudi] n3nash commented on issue #3063: Hive database not auto created when syncing

2021-06-15 Thread GitBox
n3nash commented on issue #3063: URL: https://github.com/apache/hudi/issues/3063#issuecomment-862010383 Closing this ticket since the issue is resolved. Thanks @veenaypatil ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] n3nash closed issue #3063: Hive database not auto created when syncing

2021-06-15 Thread GitBox
n3nash closed issue #3063: URL: https://github.com/apache/hudi/issues/3063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] hudi-bot commented on pull request #3035: [HUDI-1936] Introduce a optional property for conditional upsert

2021-06-15 Thread GitBox
hudi-bot commented on pull request #3035: URL: https://github.com/apache/hudi/pull/3035#issuecomment-862017744 ## CI report: * 26dadb6627c90c9f06e66fba0b8bd24e5579665f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] nsivabalan commented on issue #1845: [SUPPORT] Support for Schema evolution. Facing an error

2021-06-15 Thread GitBox
nsivabalan commented on issue #1845: URL: https://github.com/apache/hudi/issues/1845#issuecomment-862039137 yes, I am in sync w/ @sbernauer via slack. He confirmed that the PR we have put up works for him (older records able to be upserted to hudi after schema evolved w/ hudi table). He

[GitHub] [hudi] wangfeigithub commented on issue #3065: Not in Marker Dir occurs when I write to HDFS using Spark

2021-06-15 Thread GitBox
wangfeigithub commented on issue #3065: URL: https://github.com/apache/hudi/issues/3065#issuecomment-862055951 https://github.com/apache/hudi/issues/3065#issuecomment-862009477 directly writing new files using 0.8 -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] n3nash commented on issue #3078: [SUPPORT] combineAndGetUpdateValue is not getting called when Schema evolution happens

2021-06-15 Thread GitBox
n3nash commented on issue #3078: URL: https://github.com/apache/hudi/issues/3078#issuecomment-862008843 @tandonraghav To expect the payload semantics for compaction, you need to override the `preCombine` implementation and have the same implementation across `combineAndGetUpdateValue` and

[GitHub] [hudi] n3nash commented on issue #3059: when java client api support MERGE_ON_READ ?

2021-06-15 Thread GitBox
n3nash commented on issue #3059: URL: https://github.com/apache/hudi/issues/3059#issuecomment-862010731 @lppsuixn Gentle ping to respond to @leesf comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] n3nash closed issue #1679: [HUDI-1609] How to disable Hive JDBC and enable metastore

2021-06-15 Thread GitBox
n3nash closed issue #1679: URL: https://github.com/apache/hudi/issues/1679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact

[GitHub] [hudi] n3nash commented on issue #1679: [HUDI-1609] How to disable Hive JDBC and enable metastore

2021-06-15 Thread GitBox
n3nash commented on issue #1679: URL: https://github.com/apache/hudi/issues/1679#issuecomment-862014466 Closing this ticket due to inactivity. There is a [PR](https://github.com/apache/hudi/pull/2879) open that will provide ways to disable JDBC. -- This is an automated message from

[jira] [Updated] (HUDI-1500) support incremental read clustering commit in deltastreamer

2021-06-15 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1500: - Component/s: Spark Integration > support incremental read clustering commit in deltastreamer >

[jira] [Commented] (HUDI-1975) Upgrade java-prometheus-client from 3.1.2 to 4.x

2021-06-15 Thread Nishith Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363911#comment-17363911 ] Nishith Agarwal commented on HUDI-1975: --- [~vinaypatil18] I think there are 2 options :  # Shade the

[hudi] branch master updated (b8fe5b9 -> 910fe48)

2021-06-15 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from b8fe5b9 [HUDI-764] [HUDI-765] ORC reader writer Implementation (#2999) add 910fe48 [MINOR] Rename broken

[GitHub] [hudi] hudi-bot edited a comment on pull request #3086: [HUDI-1776] Support AlterCommand For Hoodie

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3086: URL: https://github.com/apache/hudi/pull/3086#issuecomment-861654333 ## CI report: * c041853f41119f23760388d1ab5c7173fe22936b Azure:

[GitHub] [hudi] prashantwason merged pull request #2999: [HUDI-764] [HUDI-765] ORC reader writer Implementation

2021-06-15 Thread GitBox
prashantwason merged pull request #2999: URL: https://github.com/apache/hudi/pull/2999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [hudi] vinothchandar merged pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
vinothchandar merged pull request #3088: URL: https://github.com/apache/hudi/pull/3088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[hudi] branch master updated: [HUDI-2022] Release writer for append handle #close (#3087)

2021-06-15 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 61efc6a [HUDI-2022] Release writer for append

[GitHub] [hudi] yanghua merged pull request #3087: [HUDI-2022] Release writer for append handle #close

2021-06-15 Thread GitBox
yanghua merged pull request #3087: URL: https://github.com/apache/hudi/pull/3087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] xushiyan commented on pull request #3086: [HUDI-1776] Support AlterCommand For Hoodie

2021-06-15 Thread GitBox
xushiyan commented on pull request #3086: URL: https://github.com/apache/hudi/pull/3086#issuecomment-861991019 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[hudi] branch master updated: [HUDI-764] [HUDI-765] ORC reader writer Implementation (#2999)

2021-06-15 Thread pwason
This is an automated email from the ASF dual-hosted git repository. pwason pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b8fe5b9 [HUDI-764] [HUDI-765] ORC reader writer

[GitHub] [hudi] hudi-bot commented on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
hudi-bot commented on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861890266 ## CI report: * e168476083aaea02220d5c3502d2ca20840cd236 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] vinothchandar commented on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
vinothchandar commented on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861890684 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] danny0405 commented on a change in pull request #3025: [HUDI-1955]Fix the filter condition is missing in the judgment condition of comp…

2021-06-15 Thread GitBox
danny0405 commented on a change in pull request #3025: URL: https://github.com/apache/hudi/pull/3025#discussion_r652258797 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/BaseScheduleCompactionActionExecutor.java ## @@ -63,7

[GitHub] [hudi] danny0405 commented on pull request #3085: [HUDI-2019] Update writeConfig in every task

2021-06-15 Thread GitBox
danny0405 commented on pull request #3085: URL: https://github.com/apache/hudi/pull/3085#issuecomment-861919796 Nice catch, can we make the commit message more clear: Set up the file system view storage config for singleton embedded server write config every time -- This is an automated

[jira] [Created] (HUDI-2025) Bring parity between row writer bulk_insert and rdd based bulk_insert

2021-06-15 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2025: - Summary: Bring parity between row writer bulk_insert and rdd based bulk_insert Key: HUDI-2025 URL: https://issues.apache.org/jira/browse/HUDI-2025 Project:

[jira] [Comment Edited] (HUDI-2025) Bring parity between row writer bulk_insert and rdd based bulk_insert

2021-06-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364021#comment-17364021 ] sivabalan narayanan edited comment on HUDI-2025 at 6/16/21, 2:48 AM: -

[GitHub] [hudi] hudi-bot edited a comment on pull request #2906: [HUDI-393] remove travis

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #2906: URL: https://github.com/apache/hudi/pull/2906#issuecomment-830929050 ## CI report: * e96f2c03ab0f4fd6deb6803479fa6624eb21ed73 UNKNOWN * 0985b9b4a64b8015257eae8d85dfd899acf7a910 UNKNOWN *

[jira] [Commented] (HUDI-764) Implement HoodieOrcWriter

2021-06-15 Thread Jintao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363931#comment-17363931 ] Jintao commented on HUDI-764: - The PR#2999 has been landed. We can close this ticket. > Implement

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861892572 #

[GitHub] [hudi] hudi-bot edited a comment on pull request #3086: [HUDI-1776] Support AlterCommand For Hoodie

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3086: URL: https://github.com/apache/hudi/pull/3086#issuecomment-861654333 ## CI report: * c041853f41119f23760388d1ab5c7173fe22936b Azure:

[GitHub] [hudi] prashantwason commented on a change in pull request #3079: [HUDI-2013] Removed option to fallback to file listing when Metadata Table is enabled.

2021-06-15 Thread GitBox
prashantwason commented on a change in pull request #3079: URL: https://github.com/apache/hudi/pull/3079#discussion_r652145709 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java ## @@ -101,11 +101,7 @@ protected

[GitHub] [hudi] vinothchandar opened a new pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
vinothchandar opened a new pull request #3088: URL: https://github.com/apache/hudi/pull/3088 - Stop polluting PRs with wrong coverage info - Retaining the file, so someone can try digging in ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please

[GitHub] [hudi] hudi-bot edited a comment on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861890266 ## CI report: * e168476083aaea02220d5c3502d2ca20840cd236 Azure:

[jira] [Closed] (HUDI-2022) Release writer for append handle #close

2021-06-15 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-2022. -- Fix Version/s: 0.9.0 Resolution: Done 61efc6af79c389ef0a77cda75e4f562ed59ef86b > Release writer for

[GitHub] [hudi] codecov-commenter commented on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
codecov-commenter commented on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861892572 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3088?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] hudi-bot edited a comment on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861890266 ## CI report: * e168476083aaea02220d5c3502d2ca20840cd236 Azure:

[GitHub] [hudi] danny0405 commented on pull request #3085: [HUDI-2019] Update writeConfig in every task

2021-06-15 Thread GitBox
danny0405 commented on pull request #3085: URL: https://github.com/apache/hudi/pull/3085#issuecomment-861980618 Also add a test case to indicate that the config is overridden when embedded server is reused. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] yuzhaojing commented on pull request #3085: [HUDI-2019] Update writeConfig in every task

2021-06-15 Thread GitBox
yuzhaojing commented on pull request #3085: URL: https://github.com/apache/hudi/pull/3085#issuecomment-861981444 > Also add a test case to indicate that the config is overridden when embedded server is reused. OK,I will add a test case -- This is an automated message from the

[jira] [Comment Edited] (HUDI-764) Implement HoodieOrcWriter

2021-06-15 Thread Jintao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363931#comment-17363931 ] Jintao edited comment on HUDI-764 at 6/15/21, 10:28 PM: The PR#2999 has been

[jira] [Commented] (HUDI-765) Implement OrcReaderIterator

2021-06-15 Thread Jintao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363932#comment-17363932 ] Jintao commented on HUDI-765: - This PR#2999 has been merged. We can close this ticket. > Implement

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3088: [MINOR] Rename broken codecov file

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3088: URL: https://github.com/apache/hudi/pull/3088#issuecomment-861892572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Commented] (HUDI-2025) Bring parity between row writer bulk_insert and rdd based bulk_insert

2021-06-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364021#comment-17364021 ] sivabalan narayanan commented on HUDI-2025: --- Trying to check differences between both flows. 

[jira] [Updated] (HUDI-2013) Fallback to file listing may lead to data loss

2021-06-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2013: - Labels: pull-request-available (was: ) > Fallback to file listing may lead to data loss >

[GitHub] [hudi] prashantwason opened a new pull request #3079: [HUDI-2013] Removed option to fallback to file listing when Metadata Table is enabled.

2021-06-15 Thread GitBox
prashantwason opened a new pull request #3079: URL: https://github.com/apache/hudi/pull/3079 ## What is the purpose of the pull request Fixed potential issues when metadata table is deployed in production. ## Brief change log Removed the option

[GitHub] [hudi] codecov-commenter commented on pull request #3079: [HUDI-2013] Removed option to fallback to file listing when Metadata Table is enabled.

2021-06-15 Thread GitBox
codecov-commenter commented on pull request #3079: URL: https://github.com/apache/hudi/pull/3079#issuecomment-861209776 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3079?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] codecov-commenter edited a comment on pull request #2819: [HUDI-1794] Moved static COMMIT_FORMATTER to thread local variable as SimpleDateFormat is not thread safe.

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #2819: URL: https://github.com/apache/hudi/pull/2819#issuecomment-860933035 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3067: [HUDI-1999] Refresh the base file view cache for WriteProfile

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3067: URL: https://github.com/apache/hudi/pull/3067#issuecomment-859459755 #

[jira] [Updated] (HUDI-2014) Support flink hive sync in batch mode

2021-06-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2014: - Labels: pull-request-available (was: ) > Support flink hive sync in batch mode >

[GitHub] [hudi] codecov-commenter commented on pull request #3082: [HUDI-1717] Metadata Reader should merge all the un-synced but complete instants from the dataset timeline.

2021-06-15 Thread GitBox
codecov-commenter commented on pull request #3082: URL: https://github.com/apache/hudi/pull/3082#issuecomment-861243412 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3082?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3067: [HUDI-1999] Refresh the base file view cache for WriteProfile

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3067: URL: https://github.com/apache/hudi/pull/3067#issuecomment-859459755 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3080: [MINOR] Fixed the log which should only be printed when the Metadata Table is disabled.

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3080: URL: https://github.com/apache/hudi/pull/3080#issuecomment-861251060 #

[GitHub] [hudi] prashantwason opened a new pull request #3083: [HUDI-2016] Fixed bootstrap of Metadata Table when some actions are in progress.

2021-06-15 Thread GitBox
prashantwason opened a new pull request #3083: URL: https://github.com/apache/hudi/pull/3083 ## What is the purpose of the pull request Metadata Table cannot be bootstrapped when any action is in progress. This is detected by the presence of inflight or requested instants. The

[jira] [Created] (HUDI-2014) Support flink hive sync in batch mode

2021-06-15 Thread Zheng yunhong (Jira)
Zheng yunhong created HUDI-2014: --- Summary: Support flink hive sync in batch mode Key: HUDI-2014 URL: https://issues.apache.org/jira/browse/HUDI-2014 Project: Apache Hudi Issue Type:

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3079: [HUDI-2013] Removed option to fallback to file listing when Metadata Table is enabled.

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3079: URL: https://github.com/apache/hudi/pull/3079#issuecomment-861209776 #

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3079: [HUDI-2013] Removed option to fallback to file listing when Metadata Table is enabled.

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3079: URL: https://github.com/apache/hudi/pull/3079#issuecomment-861209776 #

[GitHub] [hudi] codecov-commenter commented on pull request #3081: [HUDI-2014] Support flink hive sync in batch mode

2021-06-15 Thread GitBox
codecov-commenter commented on pull request #3081: URL: https://github.com/apache/hudi/pull/3081#issuecomment-861254263 # [Codecov](https://codecov.io/gh/apache/hudi/pull/3081?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)

[jira] [Created] (HUDI-2016) Metadata table bootstrap does not work when there are inflight instances

2021-06-15 Thread Prashant Wason (Jira)
Prashant Wason created HUDI-2016: Summary: Metadata table bootstrap does not work when there are inflight instances Key: HUDI-2016 URL: https://issues.apache.org/jira/browse/HUDI-2016 Project: Apache

[jira] [Updated] (HUDI-2017) Some of the Metadata table metrics are incorrect

2021-06-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2017: - Labels: pull-request-available (was: ) > Some of the Metadata table metrics are incorrect >

[GitHub] [hudi] danny0405 closed pull request #3067: [HUDI-1999] Refresh the base file view cache for WriteProfile

2021-06-15 Thread GitBox
danny0405 closed pull request #3067: URL: https://github.com/apache/hudi/pull/3067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hudi] prashantwason opened a new pull request #3080: [MINOR] Fixed the log which should only be printed when the Metadata Table is disabled.

2021-06-15 Thread GitBox
prashantwason opened a new pull request #3080: URL: https://github.com/apache/hudi/pull/3080 ## What is the purpose of the pull request The function initIfNeeded() is called through multiple code paths and should only log "Metadata table is disabled" when the table is disabled.

[jira] [Updated] (HUDI-1717) Metadata Table reader does not show correct view of the metadata

2021-06-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1717: - Labels: pull-request-available sev:critical user-support-issues (was: sev:critical

[GitHub] [hudi] codecov-commenter edited a comment on pull request #3067: [HUDI-1999] Refresh the base file view cache for WriteProfile

2021-06-15 Thread GitBox
codecov-commenter edited a comment on pull request #3067: URL: https://github.com/apache/hudi/pull/3067#issuecomment-859459755 #

[GitHub] [hudi] swuferhong opened a new pull request #3081: [HUDI-2014] Support flink hive sync in batch mode

2021-06-15 Thread GitBox
swuferhong opened a new pull request #3081: URL: https://github.com/apache/hudi/pull/3081 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[jira] [Created] (HUDI-2017) Some of the Metadata table metrics are incorrect

2021-06-15 Thread Prashant Wason (Jira)
Prashant Wason created HUDI-2017: Summary: Some of the Metadata table metrics are incorrect Key: HUDI-2017 URL: https://issues.apache.org/jira/browse/HUDI-2017 Project: Apache Hudi Issue

[GitHub] [hudi] hudi-bot edited a comment on pull request #2984: (Azure CI) test PR

2021-06-15 Thread GitBox
hudi-bot edited a comment on pull request #2984: URL: https://github.com/apache/hudi/pull/2984#issuecomment-846794102 ## CI report: * 480c169776dcf2260cbfebc7dc90bd2f1807e411 UNKNOWN * 913f8886136cfb46fbb4df3f408b0c1c73fcc2cb Azure:

  1   2   3   >