[GitHub] [hudi] hudi-bot edited a comment on pull request #2927: [HUDI-1129] Improving schema evolution support in hudi

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #2927: URL: https://github.com/apache/hudi/pull/2927#issuecomment-864700767 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393835#comment-17393835 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong closed pull request #3285: URL:

[GitHub] [hudi] pengzhiwei2018 commented on a change in pull request #3387: [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql

2021-08-05 Thread GitBox
pengzhiwei2018 commented on a change in pull request #3387: URL: https://github.com/apache/hudi/pull/3387#discussion_r682621582 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ## @@ -399,6 +400,11 @@ object

[GitHub] [hudi] pengzhiwei2018 commented on pull request #3360: [HUDI-2243] Support Time Travel Query For Hoodie Table

2021-08-05 Thread GitBox
pengzhiwei2018 commented on pull request #3360: URL: https://github.com/apache/hudi/pull/3360#issuecomment-892568554 > > > @pengzhiwei2018 Thanks for this. Maybe we could have couple of follow on tasks. > > > > > > 1. Allow user to specify `as.of.instant` in `-MM-DD` to

[jira] [Commented] (HUDI-2273) Bring down the total test run time with CI

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393839#comment-17393839 ] ASF GitHub Bot commented on HUDI-2273: -- nsivabalan commented on pull request #3398: URL:

[jira] [Commented] (HUDI-1129) AvroConversionUtils unable to handle avro to row transformation when passing evolved schema

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393838#comment-17393838 ] ASF GitHub Bot commented on HUDI-1129: -- hudi-bot edited a comment on pull request #2927: URL:

[jira] [Commented] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393840#comment-17393840 ] ASF GitHub Bot commented on HUDI-2233: -- pengzhiwei2018 commented on a change in pull request #3387:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3387: [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3387: URL: https://github.com/apache/hudi/pull/3387#issuecomment-891570386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] pengzhiwei2018 merged pull request #3379: [HUDI-2232] [SQL] MERGE INTO fails with table having nested struct type

2021-08-05 Thread GitBox
pengzhiwei2018 merged pull request #3379: URL: https://github.com/apache/hudi/pull/3379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393842#comment-17393842 ] ASF GitHub Bot commented on HUDI-2233: -- hudi-bot edited a comment on pull request #3387: URL:

[jira] [Commented] (HUDI-2243) Support Time Travel Query For Hoodie Table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393841#comment-17393841 ] ASF GitHub Bot commented on HUDI-2243: -- pengzhiwei2018 commented on pull request #3360: URL:

[GitHub] [hudi] nsivabalan commented on a change in pull request #3387: [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql

2021-08-05 Thread GitBox
nsivabalan commented on a change in pull request #3387: URL: https://github.com/apache/hudi/pull/3387#discussion_r682577407 ## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala ## @@ -399,6 +400,11 @@ object

[jira] [Commented] (HUDI-2232) [SQL] MERGE INTO fails with table having nested struct

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393844#comment-17393844 ] ASF GitHub Bot commented on HUDI-2232: -- pengzhiwei2018 merged pull request #3379: URL:

[jira] [Commented] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393846#comment-17393846 ] ASF GitHub Bot commented on HUDI-2233: -- nsivabalan commented on a change in pull request #3387: URL:

[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393849#comment-17393849 ] ASF GitHub Bot commented on HUDI-2170: -- danny0405 commented on pull request #3267: URL:

[GitHub] [hudi] danny0405 commented on pull request #3267: [HUDI-2170] Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread GitBox
danny0405 commented on pull request #3267: URL: https://github.com/apache/hudi/pull/3267#issuecomment-893111329 Have moved to https://github.com/apache/hudi/pull/3401. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (HUDI-1842) [SQL] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393833#comment-17393833 ] ASF GitHub Bot commented on HUDI-1842: -- pengzhiwei2018 edited a comment on pull request #3393: URL:

[jira] [Commented] (HUDI-1842) [SQL] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393852#comment-17393852 ] ASF GitHub Bot commented on HUDI-1842: -- pengzhiwei2018 commented on pull request #3393: URL:

[GitHub] [hudi] pengzhiwei2018 commented on pull request #3393: [HUDI-1842] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread GitBox
pengzhiwei2018 commented on pull request #3393: URL: https://github.com/apache/hudi/pull/3393#issuecomment-892526491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] vinothchandar commented on pull request #3405: [HUDI-1985] Added algolia search to the site + fixed broken links on the blog

2021-08-05 Thread GitBox
vinothchandar commented on pull request #3405: URL: https://github.com/apache/hudi/pull/3405#issuecomment-892845002 Woot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] zhedoubushishi edited a comment on pull request #1975: [HUDI-1194] Refactor HoodieHiveClient based on the way to call Hive API

2021-08-05 Thread GitBox
zhedoubushishi edited a comment on pull request #1975: URL: https://github.com/apache/hudi/pull/1975#issuecomment-893290649 > Sorry for the delay in responding. Similar PR was in progress and has been merged #2879. > We can close this PR. Yes since it's duplicated work, closed

[GitHub] [hudi] veenaypatil commented on pull request #3375: [HUDI-2257] Adding note to set Keygen class while deleting data

2021-08-05 Thread GitBox
veenaypatil commented on pull request #3375: URL: https://github.com/apache/hudi/pull/3375#issuecomment-892758248 Closing and creating a new PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HUDI-1194) Reorganize HoodieHiveClient and make it fully support Hive Metastore API

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393856#comment-17393856 ] ASF GitHub Bot commented on HUDI-1194: -- zhedoubushishi edited a comment on pull request #1975: URL:

[jira] [Commented] (HUDI-2257) Add a note to set keygenerator class while deleting data

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393857#comment-17393857 ] ASF GitHub Bot commented on HUDI-2257: -- veenaypatil commented on pull request #3375: URL:

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393747#comment-17393747 ] ASF GitHub Bot commented on HUDI-2277: -- zhangyue19921010 opened a new pull request #3413: URL:

[jira] [Updated] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2277: - Labels: pull-request-available (was: ) > Let HoodieDeltaStreamer reading ORC files using

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393750#comment-17393750 ] ASF GitHub Bot commented on HUDI-2277: -- hudi-bot commented on pull request #3413: URL:

[jira] [Commented] (HUDI-1842) [SQL] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393758#comment-17393758 ] ASF GitHub Bot commented on HUDI-1842: -- hudi-bot edited a comment on pull request #3393: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3393: [HUDI-1842] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3393: URL: https://github.com/apache/hudi/pull/3393#issuecomment-891812944 ## CI report: * 921d21fc6732fe51296e21fd3a26fa11c16bfca3 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3328: [HUDI-2208] Support Bulk Insert For Spark Sql

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3328: URL: https://github.com/apache/hudi/pull/3328#issuecomment-884869427 ## CI report: * 9c9f804618dd0275abdae10673c21bf1f5737caf UNKNOWN * 50539ec543951e7a4442798ac7c66e5dc3d3705a UNKNOWN *

[jira] [Created] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread Yue Zhang (Jira)
Yue Zhang created HUDI-2277: --- Summary: Let HoodieDeltaStreamer reading ORC files using ORCDFSSource Key: HUDI-2277 URL: https://issues.apache.org/jira/browse/HUDI-2277 Project: Apache Hudi Issue

[GitHub] [hudi] zhangyue19921010 opened a new pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
zhangyue19921010 opened a new pull request #3413: URL: https://github.com/apache/hudi/pull/3413 https://issues.apache.org/jira/projects/HUDI/issues/HUDI-2277?filter=reportedbyme ## What is the purpose of the pull request Develop a new Source named ORCDFSSource extended from

[jira] [Commented] (HUDI-2274) Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393749#comment-17393749 ] ASF GitHub Bot commented on HUDI-2274: -- danny0405 opened a new pull request #3403: URL:

[jira] [Commented] (HUDI-2274) Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393748#comment-17393748 ] ASF GitHub Bot commented on HUDI-2274: -- danny0405 closed pull request #3403: URL:

[GitHub] [hudi] hudi-bot commented on pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
hudi-bot commented on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-893311636 ## CI report: * 45fbd4f73a6ccd0918e545702900351a2ed1070b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] danny0405 closed pull request #3403: [HUDI-2274] Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread GitBox
danny0405 closed pull request #3403: URL: https://github.com/apache/hudi/pull/3403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-893311636 ## CI report: * 45fbd4f73a6ccd0918e545702900351a2ed1070b Azure:

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393751#comment-17393751 ] ASF GitHub Bot commented on HUDI-2277: -- hudi-bot edited a comment on pull request #3413: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3387: [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3387: URL: https://github.com/apache/hudi/pull/3387#issuecomment-891570386 ## CI report: * dae0d69eade3ba95d39e37c1851a56534f80e007 UNKNOWN * 6043d6a54b7e2d70a071f556b4eb3da8e3992e2c UNKNOWN *

[jira] [Commented] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393768#comment-17393768 ] ASF GitHub Bot commented on HUDI-2233: -- hudi-bot edited a comment on pull request #3387: URL:

[jira] [Commented] (HUDI-2208) [SQL] Support Bulk Insert For Spark Sql

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393781#comment-17393781 ] ASF GitHub Bot commented on HUDI-2208: -- hudi-bot edited a comment on pull request #3328: URL:

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393783#comment-17393783 ] ASF GitHub Bot commented on HUDI-2277: -- hudi-bot edited a comment on pull request #3413: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-893311636 ## CI report: * 45fbd4f73a6ccd0918e545702900351a2ed1070b Azure:

[GitHub] [hudi] satishkotha commented on pull request #3211: [HUDI-1468] Support more flexible clustering strategies and preserve commit …

2021-08-05 Thread GitBox
satishkotha commented on pull request #3211: URL: https://github.com/apache/hudi/pull/3211#issuecomment-892786714 > @codope @satishkotha what's the next step here? > Could I help somehow to get this moving along @codope is working on adding additional tests for this PR. he

[jira] [Commented] (HUDI-1468) incremental read support with clustering

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393858#comment-17393858 ] ASF GitHub Bot commented on HUDI-1468: -- satishkotha commented on pull request #3211: URL:

[GitHub] [hudi] pengzhiwei2018 commented on pull request #3379: [HUDI-2232] [SQL] MERGE INTO fails with table having nested struct type

2021-08-05 Thread GitBox
pengzhiwei2018 commented on pull request #3379: URL: https://github.com/apache/hudi/pull/3379#issuecomment-892527646 > @nsivabalan LGTM, can you please land this? Thanks for the review @codope . If not mind, I will land it @nsivabalan -- This is an automated message from the

[GitHub] [hudi] vinothchandar commented on pull request #3408: [MINOR] Move to ubuntu-latest for Azure CI

2021-08-05 Thread GitBox
vinothchandar commented on pull request #3408: URL: https://github.com/apache/hudi/pull/3408#issuecomment-892927539 ``` Downloaded from Maven Central: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.11.6/scala-library-2.11.6.jar (5.6 MB at 61 MB/s) [ERROR] error:

[jira] [Commented] (HUDI-2232) [SQL] MERGE INTO fails with table having nested struct

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393861#comment-17393861 ] ASF GitHub Bot commented on HUDI-2232: -- pengzhiwei2018 commented on pull request #3379: URL:

[GitHub] [hudi] hudi-bot commented on pull request #3412: [MINOR] Moving some scala tests in hudi-spark to functional

2021-08-05 Thread GitBox
hudi-bot commented on pull request #3412: URL: https://github.com/apache/hudi/pull/3412#issuecomment-893109672 ## CI report: * ceb79cfbcd97cd57f1815abbdcc4cd5294e5468a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot commented on pull request #3409: [MINOR] Move to ubuntu-latest for Azure CI

2021-08-05 Thread GitBox
hudi-bot commented on pull request #3409: URL: https://github.com/apache/hudi/pull/3409#issuecomment-892942265 ## CI report: * 51cfe55902a50902a0bfbd13b8374248bb0873a7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Commented] (HUDI-1194) Reorganize HoodieHiveClient and make it fully support Hive Metastore API

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393862#comment-17393862 ] ASF GitHub Bot commented on HUDI-1194: -- zhedoubushishi closed pull request #1975: URL:

[GitHub] [hudi] zhedoubushishi closed pull request #1975: [HUDI-1194] Refactor HoodieHiveClient based on the way to call Hive API

2021-08-05 Thread GitBox
zhedoubushishi closed pull request #1975: URL: https://github.com/apache/hudi/pull/1975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-1985) Website re-design implementation

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393863#comment-17393863 ] ASF GitHub Bot commented on HUDI-1985: -- vingov opened a new pull request #3405: URL:

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393864#comment-17393864 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong opened a new pull request #3285: URL:

[GitHub] [hudi] nsivabalan commented on issue #3399: [SUPPORT] How to purge older partitions in hudi partitioned tables ?

2021-08-05 Thread GitBox
nsivabalan commented on issue #3399: URL: https://github.com/apache/hudi/issues/3399#issuecomment-893138598 WriteClient has support for

[GitHub] [hudi] swuferhong commented on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie

2021-08-05 Thread GitBox
swuferhong commented on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-892603519 @hudi-bot run Azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] nsivabalan commented on pull request #3387: [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql

2021-08-05 Thread GitBox
nsivabalan commented on pull request #3387: URL: https://github.com/apache/hudi/pull/3387#issuecomment-892775157 my bad. I checked out the master. I did not realize we had sync mode even before this patch. Ignore my previous question. I see this code block. ``` if

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393865#comment-17393865 ] ASF GitHub Bot commented on HUDI-1771: -- swuferhong commented on pull request #3285: URL:

[jira] [Commented] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393866#comment-17393866 ] ASF GitHub Bot commented on HUDI-2233: -- nsivabalan commented on pull request #3387: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3398: [HUDI-2273] Moving some long running tests to functional

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3398: URL: https://github.com/apache/hudi/pull/3398#issuecomment-892305785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HUDI-2273) Bring down the total test run time with CI

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393868#comment-17393868 ] ASF GitHub Bot commented on HUDI-2273: -- hudi-bot edited a comment on pull request #3398: URL:

[GitHub] [hudi] zhangminglei closed pull request #3289: [HUDI-2187] Add a shim layer to support multiple hive version

2021-08-05 Thread GitBox
zhangminglei closed pull request #3289: URL: https://github.com/apache/hudi/pull/3289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] codope commented on pull request #3393: [HUDI-1842] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread GitBox
codope commented on pull request #3393: URL: https://github.com/apache/hudi/pull/3393#issuecomment-892777427 > > > Hey peng. I did a round of testing on this patch. Here are my findings. > > > Insert into is till prefixing col name to meta fields. (3rd col and 4th col) > > > ``` >

[jira] [Commented] (HUDI-2187) Hive integration Improvment

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393870#comment-17393870 ] ASF GitHub Bot commented on HUDI-2187: -- zhangminglei closed pull request #3289: URL:

[jira] [Commented] (HUDI-2167) HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393874#comment-17393874 ] ASF GitHub Bot commented on HUDI-2167: -- danny0405 commented on a change in pull request #3402: URL:

[GitHub] [hudi] danny0405 commented on a change in pull request #3402: [HUDI-2167] HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException

2021-08-05 Thread GitBox
danny0405 commented on a change in pull request #3402: URL: https://github.com/apache/hudi/pull/3402#discussion_r682456792 ## File path: hudi-flink/src/main/java/org/apache/hudi/util/StreamerUtil.java ## @@ -152,6 +153,7 @@ public static HoodieWriteConfig

[GitHub] [hudi] Ambarish-Giri edited a comment on issue #3395: [SUPPORT] Issues with read optimized query MOR table

2021-08-05 Thread GitBox
Ambarish-Giri edited a comment on issue #3395: URL: https://github.com/apache/hudi/issues/3395#issuecomment-893140263 Sure @nsivabalan eventually our test and prod environment will be EMR only. But before doing actual testing and derive the benchmarking metrics as I said earlier just

[GitHub] [hudi] Ambarish-Giri commented on issue #3395: [SUPPORT] Issues with read optimized query MOR table

2021-08-05 Thread GitBox
Ambarish-Giri commented on issue #3395: URL: https://github.com/apache/hudi/issues/3395#issuecomment-893140263 Sure @nsivabalan eventually our test and prod environment will be EMR only. But before doing actual testing and derive the benchmarking metrics as I said earlier just evaluating

[jira] [Commented] (HUDI-1842) [SQL] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393872#comment-17393872 ] ASF GitHub Bot commented on HUDI-1842: -- codope commented on pull request #3393: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3403: [HUDI-2274] Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3403: URL: https://github.com/apache/hudi/pull/3403#issuecomment-892564001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] vinothchandar edited a comment on pull request #3409: [MINOR] Move to ubuntu-18.04 for Azure CI

2021-08-05 Thread GitBox
vinothchandar edited a comment on pull request #3409: URL: https://github.com/apache/hudi/pull/3409#issuecomment-893017385 @xushiyan @nsivabalan I am landing this for now once CI passes, this gets rid of warnings. ubuntu-16.04 is soon to be deprecated. But ideally, we should be

[jira] [Commented] (HUDI-2274) Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393875#comment-17393875 ] ASF GitHub Bot commented on HUDI-2274: -- hudi-bot edited a comment on pull request #3403: URL:

[GitHub] [hudi] liujinhui1994 commented on issue #3280: [SUPPORT] Use structedstreaming to consume kafka to write to hudi error

2021-08-05 Thread GitBox
liujinhui1994 commented on issue #3280: URL: https://github.com/apache/hudi/issues/3280#issuecomment-893193914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] vinothchandar closed pull request #3408: [MINOR] Move to ubuntu-latest for Azure CI

2021-08-05 Thread GitBox
vinothchandar closed pull request #3408: URL: https://github.com/apache/hudi/pull/3408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-2268) Upgrade hoodie table to 0.9.0

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393882#comment-17393882 ] ASF GitHub Bot commented on HUDI-2268: -- hudi-bot edited a comment on pull request #3407: URL:

[GitHub] [hudi] zhedoubushishi commented on pull request #1975: [HUDI-1194] Refactor HoodieHiveClient based on the way to call Hive API

2021-08-05 Thread GitBox
zhedoubushishi commented on pull request #1975: URL: https://github.com/apache/hudi/pull/1975#issuecomment-893290649 > Sorry for the delay in responding. Similar PR was in progress and has been merged #2879. > We can close this PR. Yes feel free to close this PR. -- This is an

[GitHub] [hudi] nsivabalan commented on issue #3395: [SUPPORT] Issues with read optimized query MOR table

2021-08-05 Thread GitBox
nsivabalan commented on issue #3395: URL: https://github.com/apache/hudi/issues/3395#issuecomment-893136900 yes, For eg, something like this ``` spark-shell \ --packages org.apache.hudi:hudi-spark3-bundle_2.12:0.8.0,org.apache.spark:spark-avro_2.12:3.0.1 \ --conf

[GitHub] [hudi] hudi-bot edited a comment on pull request #3407: [HUDI-2268] Add upgrade/downgrade to and from 0.9.0

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3407: URL: https://github.com/apache/hudi/pull/3407#issuecomment-892882427 ## CI report: * 4f094a9786e520d69ceb51437af5a224cca63579 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3412: [MINOR] Moving some scala tests in hudi-spark to functional

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3412: URL: https://github.com/apache/hudi/pull/3412#issuecomment-893109672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HUDI-1194) Reorganize HoodieHiveClient and make it fully support Hive Metastore API

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393881#comment-17393881 ] ASF GitHub Bot commented on HUDI-1194: -- zhedoubushishi commented on pull request #1975: URL:

[GitHub] [hudi] nsivabalan commented on pull request #3404: [HUDI-2257] Adding note to set Keygen class while deleting data

2021-08-05 Thread GitBox
nsivabalan commented on pull request #3404: URL: https://github.com/apache/hudi/pull/3404#issuecomment-892991015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HUDI-2276) Enable Metadata Table by default for both writers and readers

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393886#comment-17393886 ] ASF GitHub Bot commented on HUDI-2276: -- umehrot2 opened a new pull request #3411: URL:

[jira] [Commented] (HUDI-2257) Add a note to set keygenerator class while deleting data

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393883#comment-17393883 ] ASF GitHub Bot commented on HUDI-2257: -- nsivabalan commented on pull request #3404: URL:

[jira] [Commented] (HUDI-2257) Add a note to set keygenerator class while deleting data

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393884#comment-17393884 ] ASF GitHub Bot commented on HUDI-2257: -- veenaypatil opened a new pull request #3404: URL:

[GitHub] [hudi] vinothchandar commented on pull request #3409: [MINOR] Move to ubuntu-latest for Azure CI

2021-08-05 Thread GitBox
vinothchandar commented on pull request #3409: URL: https://github.com/apache/hudi/pull/3409#issuecomment-892947745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] codope commented on issue #3280: [SUPPORT] Use structedstreaming to consume kafka to write to hudi error

2021-08-05 Thread GitBox
codope commented on issue #3280: URL: https://github.com/apache/hudi/issues/3280#issuecomment-892762965 @liujinhui1994 I tried to reproduce this locally but for me the streaming ran fine. My setup is Spark 2.4.7, Hadoop 2.7.3 and Hive 2.3.7. I tried for both COW and MOR table ```

[GitHub] [hudi] vingov commented on pull request #3404: [HUDI-2257] Adding note to set Keygen class while deleting data

2021-08-05 Thread GitBox
vingov commented on pull request #3404: URL: https://github.com/apache/hudi/pull/3404#issuecomment-893051174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HUDI-2257) Add a note to set keygenerator class while deleting data

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393893#comment-17393893 ] ASF GitHub Bot commented on HUDI-2257: -- vingov commented on pull request #3404: URL:

[GitHub] [hudi] nsivabalan commented on issue #3280: [SUPPORT] Use structedstreaming to consume kafka to write to hudi error

2021-08-05 Thread GitBox
nsivabalan commented on issue #3280: URL: https://github.com/apache/hudi/issues/3280#issuecomment-893194763 thanks @codope -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot edited a comment on pull request #3411: [HUDI-2276] Enable metadata table by default for readers and writers

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3411: URL: https://github.com/apache/hudi/pull/3411#issuecomment-893073002 ## CI report: * e441c95e938929d79f78fb9561869bd726dd69b8 Azure:

[jira] [Commented] (HUDI-2187) Hive integration Improvment

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393894#comment-17393894 ] ASF GitHub Bot commented on HUDI-2187: -- zhangminglei opened a new pull request #3289: URL:

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393896#comment-17393896 ] ASF GitHub Bot commented on HUDI-2277: -- zhangyue19921010 opened a new pull request #3413: URL:

[jira] [Commented] (HUDI-2276) Enable Metadata Table by default for both writers and readers

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393897#comment-17393897 ] ASF GitHub Bot commented on HUDI-2276: -- hudi-bot edited a comment on pull request #3411: URL:

[jira] [Created] (HUDI-2278) Use INT64 timestamp with precision 3 for flink parquet writer

2021-08-05 Thread Danny Chen (Jira)
Danny Chen created HUDI-2278: Summary: Use INT64 timestamp with precision 3 for flink parquet writer Key: HUDI-2278 URL: https://issues.apache.org/jira/browse/HUDI-2278 Project: Apache Hudi

[jira] [Commented] (HUDI-2167) HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393901#comment-17393901 ] ASF GitHub Bot commented on HUDI-2167: -- tsianglei commented on a change in pull request #3402: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3328: [HUDI-2208] Support Bulk Insert For Spark Sql

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3328: URL: https://github.com/apache/hudi/pull/3328#issuecomment-884869427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] tsianglei commented on a change in pull request #3402: [HUDI-2167] HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException

2021-08-05 Thread GitBox
tsianglei commented on a change in pull request #3402: URL: https://github.com/apache/hudi/pull/3402#discussion_r682461965 ## File path: hudi-flink/src/main/java/org/apache/hudi/util/StreamerUtil.java ## @@ -152,6 +153,7 @@ public static HoodieWriteConfig

<    1   2   3   4   5