[GitHub] [hudi] hudi-bot edited a comment on pull request #3277: [HUDI-2182] Support Compaction Command For Spark Sql

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3277: URL: https://github.com/apache/hudi/pull/3277#issuecomment-880506300 ## CI report: * 0eeef02b58baa7ade8fc0196c2c16c165daafcdf Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3387: [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3387: URL: https://github.com/apache/hudi/pull/3387#issuecomment-891570386 ## CI report: * dae0d69eade3ba95d39e37c1851a56534f80e007 UNKNOWN * 6043d6a54b7e2d70a071f556b4eb3da8e3992e2c UNKNOWN *

[jira] [Commented] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394003#comment-17394003 ] ASF GitHub Bot commented on HUDI-2233: -- hudi-bot edited a comment on pull request #3387: URL:

[jira] [Updated] (HUDI-2279) Support column name matching for insert * and update set * when sourceTable's columns contains all targetTable's columns

2021-08-05 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-2279: -- Description: Example: {code:java} val tableName = generateTableName // Create table spark.sql( s""" |create table

[GitHub] [hudi] danny0405 closed pull request #3403: [HUDI-2274] Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread GitBox
danny0405 closed pull request #3403: URL: https://github.com/apache/hudi/pull/3403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-2279) Support column name matching for insert * and update set * in merge into when sourceTable's columns contains all targetTable's columns

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394147#comment-17394147 ] ASF GitHub Bot commented on HUDI-2279: -- hudi-bot edited a comment on pull request #3415: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3415: [HUDI-2279]Support column name matching for insert * and update set *

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3415: URL: https://github.com/apache/hudi/pull/3415#issuecomment-893539789 ## CI report: * 10b2dc9c80f373a90f0140507fb20b85dfcf30d5 Azure:

[jira] [Commented] (HUDI-2275) HoodieDeltaStreamerException when using OCC and a second concurrent writer

2021-08-05 Thread Dave Hagman (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394190#comment-17394190 ] Dave Hagman commented on HUDI-2275: --- Comment in Slack from Shiv Narayan:   {code:java} here is my

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394195#comment-17394195 ] ASF GitHub Bot commented on HUDI-2277: -- hudi-bot edited a comment on pull request #3413: URL:

[GitHub] [hudi] vinothchandar commented on pull request #3409: [HUDI-2080] Move to ubuntu-18.04 for Azure CI

2021-08-05 Thread GitBox
vinothchandar commented on pull request #3409: URL: https://github.com/apache/hudi/pull/3409#issuecomment-893617720 CI did not pass. there seems to be a problem with one of the tests. :( -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (HUDI-2080) Migrate to ubuntu 20.04 for Azure pipeline builds

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394205#comment-17394205 ] ASF GitHub Bot commented on HUDI-2080: -- vinothchandar commented on pull request #3409: URL:

[jira] [Updated] (HUDI-2080) Migrate to ubuntu 20.04 for Azure pipeline builds

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2080: - Labels: pull-request-available (was: ) > Migrate to ubuntu 20.04 for Azure pipeline builds >

[jira] [Commented] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394282#comment-17394282 ] ASF GitHub Bot commented on HUDI-2281: -- satishkotha opened a new pull request #3417: URL:

[GitHub] [hudi] satishkotha commented on pull request #3417: [HUDI-2281] Add metadata client APIs to fetch list of data files and …

2021-08-05 Thread GitBox
satishkotha commented on pull request #3417: URL: https://github.com/apache/hudi/pull/3417#issuecomment-893779539 @n3nash @prashantwason Please take a look. I'm just upstreaming libraries being used internally and added tests. -- This is an automated message from the Apache Git

[jira] [Updated] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2281: - Labels: pull-request-available (was: ) > add metadata client to read snapshot and incremental

[GitHub] [hudi] satishkotha opened a new pull request #3417: [HUDI-2281] Add metadata client APIs to fetch list of data files and …

2021-08-05 Thread GitBox
satishkotha opened a new pull request #3417: URL: https://github.com/apache/hudi/pull/3417 ## What is the purpose of the pull request Provide generic APIs to * get all modified partitions since a specified commit time * get all data files written as part of latest commit

[jira] [Commented] (HUDI-2274) Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394324#comment-17394324 ] ASF GitHub Bot commented on HUDI-2274: -- danny0405 closed pull request #3403: URL:

[jira] [Commented] (HUDI-2274) Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394325#comment-17394325 ] ASF GitHub Bot commented on HUDI-2274: -- danny0405 opened a new pull request #3403: URL:

[GitHub] [hudi] danny0405 closed pull request #3403: [HUDI-2274] Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread GitBox
danny0405 closed pull request #3403: URL: https://github.com/apache/hudi/pull/3403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] nsivabalan commented on a change in pull request #3328: [HUDI-2208] Support Bulk Insert For Spark Sql

2021-08-05 Thread GitBox
nsivabalan commented on a change in pull request #3328: URL: https://github.com/apache/hudi/pull/3328#discussion_r683843426 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/InsertIntoHoodieTableCommand.scala ## @@ -209,19 +209,32

[jira] [Commented] (HUDI-2208) [SQL] Support Bulk Insert For Spark Sql

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394384#comment-17394384 ] ASF GitHub Bot commented on HUDI-2208: -- nsivabalan commented on a change in pull request #3328: URL:

[GitHub] [hudi] FeiZou opened a new issue #3418: [SUPPORT] Hudi Upsert Very Slow/ Failed With No Space Left on Device

2021-08-05 Thread GitBox
FeiZou opened a new issue #3418: URL: https://github.com/apache/hudi/issues/3418 **Describe the problem you faced** Hi there, I'm migrating a table from S3 data lake to Hudi data lake using Spark. The source table data size is around `600 GB` and `8 B rows`, each partition contains

[GitHub] [hudi] vinothchandar commented on a change in pull request #3401: [HUDI-2170] Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread GitBox
vinothchandar commented on a change in pull request #3401: URL: https://github.com/apache/hudi/pull/3401#discussion_r683820637 ## File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/io/TestHoodieMergeHandle.java ## @@ -231,6 +229,83 @@ public void

[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394382#comment-17394382 ] ASF GitHub Bot commented on HUDI-2170: -- vinothchandar commented on pull request #3401: URL:

[jira] [Commented] (HUDI-1129) AvroConversionUtils unable to handle avro to row transformation when passing evolved schema

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394385#comment-17394385 ] ASF GitHub Bot commented on HUDI-1129: -- hudi-bot edited a comment on pull request #2927: URL:

[jira] [Comment Edited] (HUDI-2151) Make performant out-of-box configs

2021-08-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377615#comment-17377615 ] Vinoth Chandar edited comment on HUDI-2151 at 8/5/21, 5:11 PM: --- [High

[jira] [Commented] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394283#comment-17394283 ] ASF GitHub Bot commented on HUDI-2281: -- satishkotha commented on pull request #3417: URL:

[GitHub] [hudi] hudi-bot commented on pull request #3417: [HUDI-2281] Add metadata client APIs to fetch list of data files and …

2021-08-05 Thread GitBox
hudi-bot commented on pull request #3417: URL: https://github.com/apache/hudi/pull/3417#issuecomment-893780233 ## CI report: * 0d7a174df07ec255ca29915086389b1889769b8b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Updated] (HUDI-2282) Upsert for an already existing record throws DuplicateKeyException with primary key spark sql table

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2282: -- Parent: HUDI-1658 Issue Type: Sub-task (was: Improvement) > Upsert for an

[jira] [Updated] (HUDI-2282) Upsert for an already existing record throws DuplicateKeyException with primary key spark sql table

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2282: -- Description: [https://gist.github.com/nsivabalan/9837a90b1481c479a9c600bf16bafa57]  

[jira] [Created] (HUDI-2282) Upsert for an already existing record throws DuplicateKeyException with primary key spark sql table

2021-08-05 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-2282: - Summary: Upsert for an already existing record throws DuplicateKeyException with primary key spark sql table Key: HUDI-2282 URL:

[jira] [Commented] (HUDI-1129) AvroConversionUtils unable to handle avro to row transformation when passing evolved schema

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394317#comment-17394317 ] ASF GitHub Bot commented on HUDI-1129: -- vinothchandar commented on a change in pull request #2927:

[jira] [Commented] (HUDI-2276) Enable Metadata Table by default for both writers and readers

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394357#comment-17394357 ] ASF GitHub Bot commented on HUDI-2276: -- hudi-bot edited a comment on pull request #3411: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3411: [HUDI-2276] Enable metadata table by default for readers and writers

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3411: URL: https://github.com/apache/hudi/pull/3411#issuecomment-893073002 ## CI report: * e441c95e938929d79f78fb9561869bd726dd69b8 Azure:

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394387#comment-17394387 ] ASF GitHub Bot commented on HUDI-1771: -- vinothchandar commented on a change in pull request #3285:

[GitHub] [hudi] vinothchandar commented on a change in pull request #3285: [HUDI-1771] Propagate CDC format for hoodie

2021-08-05 Thread GitBox
vinothchandar commented on a change in pull request #3285: URL: https://github.com/apache/hudi/pull/3285#discussion_r683844723 ## File path: hudi-flink/src/main/java/org/apache/hudi/table/format/mor/MergeOnReadInputFormat.java ## @@ -620,23 +633,31 @@ public boolean

[jira] [Created] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread satish (Jira)
satish created HUDI-2281: Summary: add metadata client to read snapshot and incremental information Key: HUDI-2281 URL: https://issues.apache.org/jira/browse/HUDI-2281 Project: Apache Hudi Issue

[jira] [Commented] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394312#comment-17394312 ] ASF GitHub Bot commented on HUDI-2281: -- hudi-bot edited a comment on pull request #3417: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3417: [HUDI-2281] Add metadata client APIs to fetch list of data files and …

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3417: URL: https://github.com/apache/hudi/pull/3417#issuecomment-893780233 ## CI report: * 0d7a174df07ec255ca29915086389b1889769b8b Azure:

[GitHub] [hudi] vinothchandar commented on a change in pull request #2927: [HUDI-1129] Improving schema evolution support in hudi

2021-08-05 Thread GitBox
vinothchandar commented on a change in pull request #2927: URL: https://github.com/apache/hudi/pull/2927#discussion_r683796125 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala ## @@ -92,22 +92,45 @@ object HoodieSparkUtils

[jira] [Updated] (HUDI-2282) Insert for an already existing record throws DuplicateKeyException with primary keyed spark sql table

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2282: -- Summary: Insert for an already existing record throws DuplicateKeyException with

[jira] [Commented] (HUDI-2278) Use INT64 timestamp with precision 3 for flink parquet writer

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394322#comment-17394322 ] ASF GitHub Bot commented on HUDI-2278: -- danny0405 opened a new pull request #3414: URL:

[jira] [Commented] (HUDI-2278) Use INT64 timestamp with precision 3 for flink parquet writer

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394321#comment-17394321 ] ASF GitHub Bot commented on HUDI-2278: -- danny0405 closed pull request #3414: URL:

[GitHub] [hudi] danny0405 closed pull request #3414: [HUDI-2278] Use INT64 timestamp with precision 3 for flink parquet wr…

2021-08-05 Thread GitBox
danny0405 closed pull request #3414: URL: https://github.com/apache/hudi/pull/3414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394330#comment-17394330 ] ASF GitHub Bot commented on HUDI-2170: -- vinothchandar commented on a change in pull request #3401:

[GitHub] [hudi] vinothchandar commented on pull request #3325: [WIP] Fixing payload instantiation to include preCombine field in LogRecordScanner

2021-08-05 Thread GitBox
vinothchandar commented on pull request #3325: URL: https://github.com/apache/hudi/pull/3325#issuecomment-893859774 Closing this in favor of #3401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] vinothchandar closed pull request #3325: [WIP] Fixing payload instantiation to include preCombine field in LogRecordScanner

2021-08-05 Thread GitBox
vinothchandar closed pull request #3325: URL: https://github.com/apache/hudi/pull/3325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394194#comment-17394194 ] ASF GitHub Bot commented on HUDI-2277: -- zhangyue19921010 commented on pull request #3413: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-893311636 ## CI report: * 45fbd4f73a6ccd0918e545702900351a2ed1070b Azure:

[GitHub] [hudi] vingov commented on pull request #3406: [DOCS] Continuous refinement of how we explain Hudi

2021-08-05 Thread GitBox
vingov commented on pull request #3406: URL: https://github.com/apache/hudi/pull/3406#issuecomment-893683964 @vinothchandar - I fixed the build issues, can you please rebase your branch with the latest asf-site branch and push again to fix this build? -- This is an automated message

[GitHub] [hudi] hudi-bot edited a comment on pull request #3416: Add external config files support

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-893712830 ## CI report: * f0d24f8b17a793c0fc925b75336d17b653102d61 Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2927: [HUDI-1129] Improving schema evolution support in hudi

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #2927: URL: https://github.com/apache/hudi/pull/2927#issuecomment-864700767 ## CI report: * f660b5a5ad27f5bcbfa915d2f3da3db0a37bd7a6 Azure:

[jira] [Commented] (HUDI-1129) AvroConversionUtils unable to handle avro to row transformation when passing evolved schema

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394334#comment-17394334 ] ASF GitHub Bot commented on HUDI-1129: -- hudi-bot edited a comment on pull request #2927: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2927: [HUDI-1129] Improving schema evolution support in hudi

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #2927: URL: https://github.com/apache/hudi/pull/2927#issuecomment-864700767 ## CI report: * fad0bc7d53c2b525fa80cfc4d23588df4ba2274e Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-893311636 ## CI report: * 45fbd4f73a6ccd0918e545702900351a2ed1070b Azure:

[jira] [Updated] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2233: -- Status: In Progress (was: Open) > [SQL] Hive sync is not working >

[jira] [Resolved] (HUDI-2233) [SQL] Hive sync is not working

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-2233. --- Resolution: Fixed > [SQL] Hive sync is not working > -- >

[jira] [Updated] (HUDI-2232) [SQL] MERGE INTO fails with table having nested struct

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-2232: -- Status: In Progress (was: Open) > [SQL] MERGE INTO fails with table having nested

[jira] [Resolved] (HUDI-2232) [SQL] MERGE INTO fails with table having nested struct

2021-08-05 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-2232. --- Resolution: Fixed > [SQL] MERGE INTO fails with table having nested struct >

[GitHub] [hudi] zhedoubushishi opened a new pull request #3416: Add external config files support

2021-08-05 Thread GitBox
zhedoubushishi opened a new pull request #3416: URL: https://github.com/apache/hudi/pull/3416 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[GitHub] [hudi] mithalee commented on issue #3336: [SUPPORT] Delete not functioning with deltastreamer

2021-08-05 Thread GitBox
mithalee commented on issue #3336: URL: https://github.com/apache/hudi/issues/3336#issuecomment-893711383 @nsivabalan @codope I came across this issue: https://issues.apache.org/jira/browse/HADOOP-17338 This may be the root cause of the error I am running into. I am running into

[GitHub] [hudi] vinothchandar merged pull request #3406: [DOCS] Continuous refinement of how we explain Hudi

2021-08-05 Thread GitBox
vinothchandar merged pull request #3406: URL: https://github.com/apache/hudi/pull/3406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch asf-site updated: [DOCS] Continuous refinement of how we explain Hudi (#3406)

2021-08-05 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new c50cc97 [DOCS] Continuous refinement of how

[jira] [Commented] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394286#comment-17394286 ] ASF GitHub Bot commented on HUDI-2281: -- hudi-bot commented on pull request #3417: URL:

[jira] [Commented] (HUDI-2277) Let HoodieDeltaStreamer reading ORC files using ORCDFSSource

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394198#comment-17394198 ] ASF GitHub Bot commented on HUDI-2277: -- hudi-bot edited a comment on pull request #3413: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #2927: [HUDI-1129] Improving schema evolution support in hudi

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #2927: URL: https://github.com/apache/hudi/pull/2927#issuecomment-864700767 ## CI report: * f660b5a5ad27f5bcbfa915d2f3da3db0a37bd7a6 Azure:

[jira] [Commented] (HUDI-1129) AvroConversionUtils unable to handle avro to row transformation when passing evolved schema

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394335#comment-17394335 ] ASF GitHub Bot commented on HUDI-1129: -- hudi-bot edited a comment on pull request #2927: URL:

[GitHub] [hudi] zhangyue19921010 commented on pull request #3413: [HUDI-2277] Let HoodieDeltaStreamer reading ORC files directly using ORCDFSSource

2021-08-05 Thread GitBox
zhangyue19921010 commented on pull request #3413: URL: https://github.com/apache/hudi/pull/3413#issuecomment-893610117 @hudi-bot run travis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (HUDI-2257) Add a note to set keygenerator class while deleting data

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394254#comment-17394254 ] ASF GitHub Bot commented on HUDI-2257: -- vingov commented on pull request #3404: URL:

[GitHub] [hudi] vingov commented on pull request #3404: [HUDI-2257] Adding note to set Keygen class while deleting data

2021-08-05 Thread GitBox
vingov commented on pull request #3404: URL: https://github.com/apache/hudi/pull/3404#issuecomment-893683457 @veenaypatil - I fixed the build issues, can you please rebase your branch with the latest asf-site branch and push again to fix this build. -- This is an automated message from

[GitHub] [hudi] hudi-bot commented on pull request #3416: Add external config files support

2021-08-05 Thread GitBox
hudi-bot commented on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-893712830 ## CI report: * f0d24f8b17a793c0fc925b75336d17b653102d61 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[GitHub] [hudi] hudi-bot edited a comment on pull request #3416: Add external config files support

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3416: URL: https://github.com/apache/hudi/pull/3416#issuecomment-893712830 ## CI report: * f0d24f8b17a793c0fc925b75336d17b653102d61 Azure:

[jira] [Commented] (HUDI-2281) add metadata client to read snapshot and incremental information

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394288#comment-17394288 ] ASF GitHub Bot commented on HUDI-2281: -- hudi-bot edited a comment on pull request #3417: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3417: [HUDI-2281] Add metadata client APIs to fetch list of data files and …

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3417: URL: https://github.com/apache/hudi/pull/3417#issuecomment-893780233 ## CI report: * 0d7a174df07ec255ca29915086389b1889769b8b Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3411: [HUDI-2276] Enable metadata table by default for readers and writers

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3411: URL: https://github.com/apache/hudi/pull/3411#issuecomment-893073002 ## CI report: * e441c95e938929d79f78fb9561869bd726dd69b8 Azure:

[jira] [Commented] (HUDI-2276) Enable Metadata Table by default for both writers and readers

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394359#comment-17394359 ] ASF GitHub Bot commented on HUDI-2276: -- hudi-bot edited a comment on pull request #3411: URL:

[jira] [Commented] (HUDI-1763) DefaultHoodieRecordPayload does not honor ordering value when records within multiple log files are merged

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394381#comment-17394381 ] ASF GitHub Bot commented on HUDI-1763: -- vinothchandar closed pull request #2977: URL:

[GitHub] [hudi] vinothchandar commented on pull request #2977: [HUDI-1763] Fixing honoring of Ordering val in DefaultHoodieRecordPayload.preCombine

2021-08-05 Thread GitBox
vinothchandar commented on pull request #2977: URL: https://github.com/apache/hudi/pull/2977#issuecomment-893875622 Closing in favor of #3401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HUDI-1763) DefaultHoodieRecordPayload does not honor ordering value when records within multiple log files are merged

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394380#comment-17394380 ] ASF GitHub Bot commented on HUDI-1763: -- vinothchandar commented on pull request #2977: URL:

[GitHub] [hudi] vinothchandar commented on pull request #3401: [HUDI-2170] [HUDI-1763] Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread GitBox
vinothchandar commented on pull request #3401: URL: https://github.com/apache/hudi/pull/3401#issuecomment-893875879 @swuferhong can you please address these and also verify once that #2977 is also fully handled by this PR -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] vinothchandar closed pull request #2977: [HUDI-1763] Fixing honoring of Ordering val in DefaultHoodieRecordPayload.preCombine

2021-08-05 Thread GitBox
vinothchandar closed pull request #2977: URL: https://github.com/apache/hudi/pull/2977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Comment Edited] (HUDI-2151) Make performant out-of-box configs

2021-08-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377663#comment-17377663 ] Vinoth Chandar edited comment on HUDI-2151 at 8/5/21, 11:27 PM: [High

[jira] [Comment Edited] (HUDI-2151) Make performant out-of-box configs

2021-08-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377618#comment-17377618 ] Vinoth Chandar edited comment on HUDI-2151 at 8/5/21, 11:27 PM:  [High

[jira] [Comment Edited] (HUDI-2151) Make performant out-of-box configs

2021-08-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377665#comment-17377665 ] Vinoth Chandar edited comment on HUDI-2151 at 8/5/21, 11:27 PM:  [High

[GitHub] [hudi] nsivabalan commented on a change in pull request #3393: [HUDI-1842] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread GitBox
nsivabalan commented on a change in pull request #3393: URL: https://github.com/apache/hudi/pull/3393#discussion_r683847551 ## File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestCreateTable.scala ## @@ -272,4 +277,154 @@ class

[jira] [Commented] (HUDI-1842) [SQL] Spark Sql Support For The Exists Hoodie Table

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394390#comment-17394390 ] ASF GitHub Bot commented on HUDI-1842: -- nsivabalan commented on a change in pull request #3393: URL:

[GitHub] [hudi] vinothchandar commented on a change in pull request #3277: [HUDI-2182] Support Compaction Command For Spark Sql

2021-08-05 Thread GitBox
vinothchandar commented on a change in pull request #3277: URL: https://github.com/apache/hudi/pull/3277#discussion_r683846483 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/CompactionShowHoodieTableCommand.scala ## @@ -0,0

[jira] [Comment Edited] (HUDI-2151) Make performant out-of-box configs

2021-08-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379161#comment-17379161 ] Vinoth Chandar edited comment on HUDI-2151 at 8/5/21, 11:31 PM: multi

[jira] [Comment Edited] (HUDI-2151) Make performant out-of-box configs

2021-08-05 Thread Vinoth Chandar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380174#comment-17380174 ] Vinoth Chandar edited comment on HUDI-2151 at 8/5/21, 11:32 PM:  [High

[jira] [Commented] (HUDI-2080) Migrate to ubuntu 20.04 for Azure pipeline builds

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394424#comment-17394424 ] ASF GitHub Bot commented on HUDI-2080: -- hudi-bot edited a comment on pull request #3409: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3411: [HUDI-2276] Enable metadata table by default for readers and writers

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3411: URL: https://github.com/apache/hudi/pull/3411#issuecomment-893073002 ## CI report: * dd289aac1d64bd9aaffb0157dee3432c5f509464 Azure:

[GitHub] [hudi] danny0405 merged pull request #3403: [HUDI-2274] Allows INSERT duplicates for Flink MOR table

2021-08-05 Thread GitBox
danny0405 merged pull request #3403: URL: https://github.com/apache/hudi/pull/3403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated (0dcd6a8 -> b7586a5)

2021-08-05 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 0dcd6a8 [HUDI-2233] Use HMS To Sync Hive Meta For Spark Sql (#3387) add b7586a5 [HUDI-2274] Allows INSERT

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394456#comment-17394456 ] ASF GitHub Bot commented on HUDI-1771: -- danny0405 commented on pull request #3285: URL:

[GitHub] [hudi] danny0405 commented on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie

2021-08-05 Thread GitBox
danny0405 commented on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-893964930 > Looks pretty un-intrusive to me. So even for Flink, we turn this on by default? For existing tables, this will cause issues with schema evol? or we only do. it for new tables?

[jira] [Commented] (HUDI-2276) Enable Metadata Table by default for both writers and readers

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394458#comment-17394458 ] ASF GitHub Bot commented on HUDI-2276: -- hudi-bot edited a comment on pull request #3411: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3411: [HUDI-2276] Enable metadata table by default for readers and writers

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3411: URL: https://github.com/apache/hudi/pull/3411#issuecomment-893073002 ## CI report: * 4b986e30d080fda684968f560a451367eac32519 Azure:

[jira] [Commented] (HUDI-2170) Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394487#comment-17394487 ] ASF GitHub Bot commented on HUDI-2170: -- hudi-bot edited a comment on pull request #3401: URL:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3401: [HUDI-2170] [HUDI-1763] Always choose the latest record for HoodieRecordPayload

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3401: URL: https://github.com/apache/hudi/pull/3401#issuecomment-892472052 ## CI report: * 7fe0db6bdda8a2f543d068efa1cbb60682b2ef95 UNKNOWN * 0b34d55f238b889fb2fcc2526e4657ea981c431c Azure:

[GitHub] [hudi] hudi-bot edited a comment on pull request #3233: [HUDI-1138] Add timeline-server-based marker file strategy for improving marker-related latency

2021-08-05 Thread GitBox
hudi-bot edited a comment on pull request #3233: URL: https://github.com/apache/hudi/pull/3233#issuecomment-875280958 ## CI report: * 2d22335c215ed620ce20018b1c83be189b7c70c6 UNKNOWN * 230205edfab190cfaf687d0323ae8d704f425e1d UNKNOWN *

<    1   2   3   4   5   >