[GitHub] [hudi] mkk1490 commented on issue #3313: [SUPPORT] CoW: Hudi Upsert not working when there is a timestamp field in the composite key

2021-07-23 Thread GitBox
mkk1490 commented on issue #3313: URL: https://github.com/apache/hudi/issues/3313#issuecomment-886010620 @nsivabalan I'm so sorry. That's my mistake. I'm trying to update the next field to src_pri_psbr_id which is pri_az_cust_id. Please find the dfs below: Insert: df_ins = spark.c

[jira] [Updated] (HUDI-2216) the words 'fiels' in the comments is incorrect

2021-07-23 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 董可伦 updated HUDI-2216: -- Attachment: HUDI-2216.png > the words 'fiels' in the comments is incorrect > ---

[jira] [Created] (HUDI-2216) the words 'fiels' in the comments is incorrect

2021-07-23 Thread Jira
董可伦 created HUDI-2216: - Summary: the words 'fiels' in the comments is incorrect Key: HUDI-2216 URL: https://issues.apache.org/jira/browse/HUDI-2216 Project: Apache Hudi Issue Type: Bug Componen

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386656#comment-17386656 ] ASF GitHub Bot commented on HUDI-2209: -- danny0405 closed pull request #3334: URL: htt

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386657#comment-17386657 ] ASF GitHub Bot commented on HUDI-2209: -- danny0405 opened a new pull request #3334: UR

[GitHub] [hudi] danny0405 closed pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
danny0405 closed pull request #3334: URL: https://github.com/apache/hudi/pull/3334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [hudi] nsivabalan commented on issue #3313: [SUPPORT] CoW: Hudi Upsert not working when there is a timestamp field in the composite key

2021-07-23 Thread GitBox
nsivabalan commented on issue #3313: URL: https://github.com/apache/hudi/issues/3313#issuecomment-886006267 Can you confirm something. I see that you have "src_pri_psbr_id" as part of your record key fields. So, I was expecting to update a record, you will not touch any of the fields th

[jira] [Commented] (HUDI-2215) Add rateLimiter when Flink writes to hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386652#comment-17386652 ] ASF GitHub Bot commented on HUDI-2215: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3338: [HUDI-2215] Add rateLimiter when Flink writes to hudi.

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3338: URL: https://github.com/apache/hudi/pull/3338#issuecomment-88662 ## CI report: * 44335b743bc600ca82282b2afeb208017e151c98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] nsivabalan commented on issue #3280: [SUPPORT] Use structedstreaming to consume kafka to write to hudi error

2021-07-23 Thread GitBox
nsivabalan commented on issue #3280: URL: https://github.com/apache/hudi/issues/3280#issuecomment-886004864 I could see two exceptions from your stacktrace: 1. Caused by: java.lang.ClassNotFoundException: org.apache.hudi.DefaultSource 2. User class threw exception: org.apache.spark.sq

[jira] [Commented] (HUDI-2189) Delete partition support in HoodieDeltaStreamer

2021-07-23 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386648#comment-17386648 ] Samrat Deb commented on HUDI-2189: -- I was trying the follow-up tasks suggested in [https

[GitHub] [hudi] nsivabalan edited a comment on issue #3336: [SUPPORT]

2021-07-23 Thread GitBox
nsivabalan edited a comment on issue #3336: URL: https://github.com/apache/hudi/issues/3336#issuecomment-886003132 Also, have you tried doing an update ("upsert") operation? Once that succeeds, then we know delete has some issue. but if update is failing, then could be some config issue or

[GitHub] [hudi] nsivabalan edited a comment on issue #3336: [SUPPORT]

2021-07-23 Thread GitBox
nsivabalan edited a comment on issue #3336: URL: https://github.com/apache/hudi/issues/3336#issuecomment-886003132 Also, have you tried doing an "update" operation? Once that succeeds, then we know delete has some issue. but if update is failing, then could be some config issue or we might

[GitHub] [hudi] nsivabalan commented on issue #3336: [SUPPORT]

2021-07-23 Thread GitBox
nsivabalan commented on issue #3336: URL: https://github.com/apache/hudi/issues/3336#issuecomment-886003132 Also, have you tried doing an update in general. Once that succeeds, then we know delete has some issue. but if update is failing, then could be some config issue or we might have to

[GitHub] [hudi] nsivabalan commented on issue #3336: [SUPPORT]

2021-07-23 Thread GitBox
nsivabalan commented on issue #3336: URL: https://github.com/apache/hudi/issues/3336#issuecomment-886003055 have you configured your record keys and partition path correctly ? I see from your configs, you have given "two" for all 3 configs(record keys, partition path and preCombine).

[GitHub] [hudi] nsivabalan commented on issue #3337: [SUPPORT] Trouble getting yyyy/mm partitioning to work with Hive sync

2021-07-23 Thread GitBox
nsivabalan commented on issue #3337: URL: https://github.com/apache/hudi/issues/3337#issuecomment-886002077 If vinoth's suggestion, does not work, do let us know. Might have to investigate further. -- This is an automated message from the Apache Git Service. To respond to the message, p

[jira] [Commented] (HUDI-2215) Add rateLimiter when Flink writes to hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386644#comment-17386644 ] ASF GitHub Bot commented on HUDI-2215: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3338: [HUDI-2215] Add rateLimiter when Flink writes to hudi.

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3338: URL: https://github.com/apache/hudi/pull/3338#issuecomment-88662 ## CI report: * 44335b743bc600ca82282b2afeb208017e151c98 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[jira] [Commented] (HUDI-2215) Add rateLimiter when Flink writes to hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386643#comment-17386643 ] ASF GitHub Bot commented on HUDI-2215: -- hudi-bot commented on pull request #3338: URL

[GitHub] [hudi] hudi-bot commented on pull request #3338: [HUDI-2215] Add rateLimiter when Flink writes to hudi.

2021-07-23 Thread GitBox
hudi-bot commented on pull request #3338: URL: https://github.com/apache/hudi/pull/3338#issuecomment-88662 ## CI report: * 44335b743bc600ca82282b2afeb208017e151c98 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run travis`

[jira] [Commented] (HUDI-2215) Add rateLimiter when Flink writes to hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386642#comment-17386642 ] ASF GitHub Bot commented on HUDI-2215: -- mincwang opened a new pull request #3338: URL

[jira] [Updated] (HUDI-2215) Add rateLimiter when Flink writes to hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2215: - Labels: flink pull-request-available (was: flink) > Add rateLimiter when Flink writes to hudi > -

[GitHub] [hudi] mincwang opened a new pull request #3338: [HUDI-2215] Add rateLimiter when Flink writes to hudi.

2021-07-23 Thread GitBox
mincwang opened a new pull request #3338: URL: https://github.com/apache/hudi/pull/3338 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pu

[hudi] branch master updated (b2f7fcb -> a14b19f)

2021-07-23 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from b2f7fcb [MINOR] Replace deprecated method isDir with isDirectory (#3319) add a14b19f [HUDI-1241] Automate the g

[jira] [Commented] (HUDI-1241) Generate config docs automatically

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386639#comment-17386639 ] ASF GitHub Bot commented on HUDI-1241: -- vinothchandar merged pull request #3302: URL:

[GitHub] [hudi] vinothchandar merged pull request #3302: [HUDI-1241] Automate the generation of configs webpage as configs are added to Hudi repo

2021-07-23 Thread GitBox
vinothchandar merged pull request #3302: URL: https://github.com/apache/hudi/pull/3302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

[jira] [Updated] (HUDI-2215) Add rateLimiter when Flink writes to hudi

2021-07-23 Thread WangMinChao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangMinChao updated HUDI-2215: -- Summary: Add rateLimiter when Flink writes to hudi (was: Add rate limiter when Flink writes to hudi) >

[jira] [Updated] (HUDI-2215) Add rate limiter when Flink writes to hudi

2021-07-23 Thread WangMinChao (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangMinChao updated HUDI-2215: -- Summary: Add rate limiter when Flink writes to hudi (was: Add rate limiter to Flink writes to hudi) >

[jira] [Created] (HUDI-2215) Add rate limiter to Flink writes to hudi

2021-07-23 Thread WangMinChao (Jira)
WangMinChao created HUDI-2215: - Summary: Add rate limiter to Flink writes to hudi Key: HUDI-2215 URL: https://issues.apache.org/jira/browse/HUDI-2215 Project: Apache Hudi Issue Type: Improvement

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386638#comment-17386638 ] ASF GitHub Bot commented on HUDI-2209: -- danny0405 opened a new pull request #3334: UR

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386637#comment-17386637 ] ASF GitHub Bot commented on HUDI-2209: -- danny0405 closed pull request #3334: URL: htt

[GitHub] [hudi] danny0405 closed pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
danny0405 closed pull request #3334: URL: https://github.com/apache/hudi/pull/3334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386632#comment-17386632 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386630#comment-17386630 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675936746 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/Zoptimize.scala ## @@ -0,0 +1,750 @@ +/* + * Licensed to the Apache Software F

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675936657 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/Zoptimize.scala ## @@ -0,0 +1,750 @@ +/* + * Licensed to the Apache Software F

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386628#comment-17386628 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386629#comment-17386629 ] ASF GitHub Bot commented on HUDI-1771: -- codope commented on pull request #3285: URL:

[GitHub] [hudi] codope commented on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie

2021-07-23 Thread GitBox
codope commented on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-885989683 > @codope please help me close out the schema evolution story here. > > my point was: when mixing old files where _hoodie_operation is NOT present with new files where it is p

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675936467 ## File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/OptimizeTableByCurve.scala ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386627#comment-17386627 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675936393 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/spark/sql/hudi/ZOrderingUtil.java ## @@ -0,0 +1,228 @@ +/* + * Licensed to the Apache Sof

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386626#comment-17386626 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675936338 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/cluster/SparkExecuteClusteringCommitActionExecutor.java ## @@ -105,6 +1

[jira] [Updated] (HUDI-1483) async clustering for deltastreamer

2021-07-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-1483: -- Status: Open (was: New) > async clustering for deltastreamer > -- > >

[jira] [Assigned] (HUDI-1483) async clustering for deltastreamer

2021-07-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-1483: - Assignee: Sagar Sumit (was: liwei) > async clustering for deltastreamer > --

[jira] [Resolved] (HUDI-1483) async clustering for deltastreamer

2021-07-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit resolved HUDI-1483. --- Resolution: Fixed > async clustering for deltastreamer > -- > >

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386625#comment-17386625 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[jira] [Assigned] (HUDI-1042) [Umbrella] Support clustering on filegroups

2021-07-23 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-1042: - Assignee: Sagar Sumit (was: liwei) > [Umbrella] Support clustering on filegroups > -

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675935594 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -182,6 +183,13 @@ public boolean commit

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386624#comment-17386624 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675935583 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -182,6 +183,13 @@ public boolean commit

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386623#comment-17386623 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675935750 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java ## @@ -238,6 +238,17 @@ private synchronized FileSystemViewM

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386622#comment-17386622 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675935583 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -182,6 +183,13 @@ public boolean commit

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386621#comment-17386621 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675935419 ## File path: hudi-client/hudi-spark-client/src/main/java/org/apache/spark/sql/hudi/UnsafeAccess.java ## @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache Softw

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386617#comment-17386617 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[jira] [Commented] (HUDI-2101) support z-order for hudi

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386618#comment-17386618 ] ASF GitHub Bot commented on HUDI-2101: -- leesf commented on a change in pull request #

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675934932 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/clustering/SparkZSortAndSizeExecutionStrategy.java ## @@ -0,0 +1,142 @@ +/* + *

[GitHub] [hudi] leesf commented on a change in pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

2021-07-23 Thread GitBox
leesf commented on a change in pull request #3330: URL: https://github.com/apache/hudi/pull/3330#discussion_r675934932 ## File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/clustering/SparkZSortAndSizeExecutionStrategy.java ## @@ -0,0 +1,142 @@ +/* + *

[jira] [Updated] (HUDI-2210) Replace deprecated method isDir with isDirectory

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2210: - Labels: pull-request-available (was: ) > Replace deprecated method isDir with isDirectory > -

[jira] [Commented] (HUDI-2210) Replace deprecated method isDir with isDirectory

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386613#comment-17386613 ] ASF GitHub Bot commented on HUDI-2210: -- danny0405 merged pull request #3319: URL: htt

[hudi] branch master updated: [MINOR] Replace deprecated method isDir with isDirectory (#3319)

2021-07-23 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b2f7fcb [MINOR] Replace deprecated method isDi

[GitHub] [hudi] danny0405 merged pull request #3319: [HUDI-2210] Replace deprecated method isDir with isDirectory

2021-07-23 Thread GitBox
danny0405 merged pull request #3319: URL: https://github.com/apache/hudi/pull/3319 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386609#comment-17386609 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * 6f7f38716d9a2c0ef10ebf2c349cdcf0e5f053de UNKNOWN * 6fecd2b043e75f3327d4a8f7348c7675682fcd70 Azure: [FAILURE](https://dev.azure.com/apache-hudi-

[GitHub] [hudi] vinothchandar commented on issue #3337: [SUPPORT] Trouble getting yyyy/mm partitioning to work with Hive sync

2021-07-23 Thread GitBox
vinothchandar commented on issue #3337: URL: https://github.com/apache/hudi/issues/3337#issuecomment-885979308 Have you tried ``` --partition-value-extractor org.apache.hudi.hive.MultiPartKeysValueExtractor --partitioned-by year, month ``` if you look at this code below, it e

[jira] [Commented] (HUDI-1241) Generate config docs automatically

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386605#comment-17386605 ] ASF GitHub Bot commented on HUDI-1241: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3302: [HUDI-1241] Automate the generation of configs webpage as configs are added to Hudi repo

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3302: URL: https://github.com/apache/hudi/pull/3302#issuecomment-882806783 ## CI report: * 5b8bba6d9860a787ac7413a1ce0bedf142d6b4c7 UNKNOWN * 27c7c05ca1705faa76fde2e3e888311bb62b8d54 UNKNOWN * 073738b3c4fe5c7fd44381b337a760daaa7ea50

[GitHub] [hudi] vinothchandar opened a new issue #3337: [SUPPORT] Trouble getting yyyy/mm partitioning to work with Hive sync

2021-07-23 Thread GitBox
vinothchandar opened a new issue #3337: URL: https://github.com/apache/hudi/issues/3337 **Describe the problem you faced** Hi, everyone! We ingest data with options: ``` hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.CustomKeyGenerator hoodie.deltastreamer

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386604#comment-17386604 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * 5bc8824bcb6983e12596d79a4b9df0b9c42a4502 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386602#comment-17386602 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * 5bc8824bcb6983e12596d79a4b9df0b9c42a4502 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386599#comment-17386599 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * 5bc8824bcb6983e12596d79a4b9df0b9c42a4502 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/re

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386598#comment-17386598 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[jira] [Commented] (HUDI-887) Add ability to read Snapshot and Incremental Queries from Presto

2021-07-23 Thread zhoukang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386597#comment-17386597 ] zhoukang commented on HUDI-887: --- any progress for this ? thanks > Add ability to read Snapsh

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * bec06d2304c67b544befff79bc6559520024f7b3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3302: [HUDI-1241] Automate the generation of configs webpage as configs are added to Hudi repo

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3302: URL: https://github.com/apache/hudi/pull/3302#issuecomment-882806783 ## CI report: * 5b8bba6d9860a787ac7413a1ce0bedf142d6b4c7 UNKNOWN * 27c7c05ca1705faa76fde2e3e888311bb62b8d54 UNKNOWN * 948ba7d5af217ef3a5046034cbcd958c6d18b34

[jira] [Commented] (HUDI-1241) Generate config docs automatically

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386593#comment-17386593 ] ASF GitHub Bot commented on HUDI-1241: -- hudi-bot edited a comment on pull request #33

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386587#comment-17386587 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[jira] [Commented] (HUDI-1241) Generate config docs automatically

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386585#comment-17386585 ] ASF GitHub Bot commented on HUDI-1241: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * bec06d2304c67b544befff79bc6559520024f7b3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[GitHub] [hudi] hudi-bot edited a comment on pull request #3302: [HUDI-1241] Automate the generation of configs webpage as configs are added to Hudi repo

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3302: URL: https://github.com/apache/hudi/pull/3302#issuecomment-882806783 ## CI report: * 5b8bba6d9860a787ac7413a1ce0bedf142d6b4c7 UNKNOWN * 27c7c05ca1705faa76fde2e3e888311bb62b8d54 UNKNOWN * 948ba7d5af217ef3a5046034cbcd958c6d18b34

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386584#comment-17386584 ] ASF GitHub Bot commented on HUDI-2209: -- hudi-bot edited a comment on pull request #33

[GitHub] [hudi] hudi-bot edited a comment on pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3334: URL: https://github.com/apache/hudi/pull/3334#issuecomment-885497812 ## CI report: * bec06d2304c67b544befff79bc6559520024f7b3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/res

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386580#comment-17386580 ] ASF GitHub Bot commented on HUDI-2209: -- danny0405 commented on a change in pull reque

[GitHub] [hudi] danny0405 commented on a change in pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
danny0405 commented on a change in pull request #3334: URL: https://github.com/apache/hudi/pull/3334#discussion_r675907954 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/bulk/BulkInsertWriterHelper.java ## @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] mithalee opened a new issue #3336: [SUPPORT]

2021-07-23 Thread GitBox
mithalee opened a new issue #3336: URL: https://github.com/apache/hudi/issues/3336 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? - Join the mailing list to engage in conversations and get faster supp

[jira] [Commented] (HUDI-1771) Propagate CDC format for hoodie

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386369#comment-17386369 ] ASF GitHub Bot commented on HUDI-1771: -- vinothchandar commented on pull request #3285

[GitHub] [hudi] vinothchandar commented on pull request #3285: [HUDI-1771] Propagate CDC format for hoodie

2021-07-23 Thread GitBox
vinothchandar commented on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-885764276 @codope please help me close out the schema evolution story here. my point was: when mixing old files where _hoodie_operation is NOT present with new files where it i

[GitHub] [hudi] yuzhaojing commented on a change in pull request #3334: [HUDI-2209] Bulk insert for flink writer

2021-07-23 Thread GitBox
yuzhaojing commented on a change in pull request #3334: URL: https://github.com/apache/hudi/pull/3334#discussion_r675673937 ## File path: hudi-flink/src/main/java/org/apache/hudi/sink/bulk/BulkInsertWriterHelper.java ## @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (HUDI-2209) Bulk insert for flink writer

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386366#comment-17386366 ] ASF GitHub Bot commented on HUDI-2209: -- yuzhaojing commented on a change in pull requ

[GitHub] [hudi] FelixKJose edited a comment on issue #3323: [SUPPORT] Trouble with Point in time, Incremental queries

2021-07-23 Thread GitBox
FelixKJose edited a comment on issue #3323: URL: https://github.com/apache/hudi/issues/3323#issuecomment-885762685 @nsivabalan is this only a problem in 0.8.0 version or older version as well? Could you please provide some insights? -- This is an automated message from the Apache Git Ser

[GitHub] [hudi] FelixKJose commented on issue #3323: [SUPPORT] Trouble with Point in time, Incremental queries

2021-07-23 Thread GitBox
FelixKJose commented on issue #3323: URL: https://github.com/apache/hudi/issues/3323#issuecomment-885762685 @nsivabalan is this only a problem in 0.8.0 version or older version as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (HUDI-1138) Re-implement marker files via timeline server

2021-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386359#comment-17386359 ] ASF GitHub Bot commented on HUDI-1138: -- hudi-bot edited a comment on pull request #32

[GitHub] [hudi] hudi-bot edited a comment on pull request #3233: [HUDI-1138] Add timeline-server-based marker file strategy for improving marker-related latency

2021-07-23 Thread GitBox
hudi-bot edited a comment on pull request #3233: URL: https://github.com/apache/hudi/pull/3233#issuecomment-875280958 ## CI report: * 2d22335c215ed620ce20018b1c83be189b7c70c6 UNKNOWN * 230205edfab190cfaf687d0323ae8d704f425e1d UNKNOWN * 618bf170c38cbd8e25b3e72e8a4d5a5c7d87aee

  1   2   3   >