[GitHub] [hudi] amrishlal commented on a diff in pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

2023-07-10 Thread via GitHub
amrishlal commented on code in PR #9131: URL: https://github.com/apache/hudi/pull/9131#discussion_r1258932073 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestUpdateTable.scala: ## @@ -48,235 +48,14 @@ class TestUpdateTable extends

[GitHub] [hudi] yihua commented on a diff in pull request #9149: [HUDI-6511] Mark more recently added configs advanced as appropriate

2023-07-10 Thread via GitHub
yihua commented on code in PR #9149: URL: https://github.com/apache/hudi/pull/9149#discussion_r1258946099 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java: ## @@ -251,50 +253,58 @@ public final class HoodieMetadataConfig extends HoodieConfig

[GitHub] [hudi] hudi-bot commented on pull request #9164: [HUDI-6516] Correct the use of hoodie.bootstrap.mode.selector

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9164: URL: https://github.com/apache/hudi/pull/9164#issuecomment-1629997571 ## CI report: * 5861f76644649791a3b050c8fb0baa12d0c4135a Azure:

[GitHub] [hudi] Zouxxyy commented on pull request #9164: [HUDI-6516] Correct the use of hoodie.bootstrap.mode.selector

2023-07-10 Thread via GitHub
Zouxxyy commented on PR #9164: URL: https://github.com/apache/hudi/pull/9164#issuecomment-1630137355 CI failure is caused by other, Can you help with a review? @codope -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] hudi-bot commented on pull request #9167: [HUDI-6519] The default value of read.streaming.enabled is determined by execution.runtime-mode

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9167: URL: https://github.com/apache/hudi/pull/9167#issuecomment-1629729800 ## CI report: * 7a8ef331fc7f4e4862d7b0f0cd6e8c35123e6c8a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] yihua commented on a diff in pull request #9149: [HUDI-6511] Mark more recently added configs advanced as appropriate

2023-07-10 Thread via GitHub
yihua commented on code in PR #9149: URL: https://github.com/apache/hudi/pull/9149#discussion_r1258950508 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala: ## @@ -124,10 +124,12 @@ object DataSourceReadOptions { val

[hudi] branch master updated (a9225bd9093 -> 5eeefaeccd5)

2023-07-10 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from a9225bd9093 [HUDI-6515] Fix bucket index row writer write record to wrong handle (#9156) add 5eeefaeccd5

[GitHub] [hudi] hudi-bot commented on pull request #8827: [DNM][Testing Only][HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #8827: URL: https://github.com/apache/hudi/pull/8827#issuecomment-1629904891 ## CI report: * 7f0cf39c716e969938e110e4de9b31e648eec83e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9168: [HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9168: URL: https://github.com/apache/hudi/pull/9168#issuecomment-1629905445 ## CI report: * 7f0cf39c716e969938e110e4de9b31e648eec83e Azure:

[GitHub] [hudi] danny0405 commented on a diff in pull request #9160: [HUDI-6501] HoodieHeartbeatClient should stop all heartbeats and not delete heartbeat files for close

2023-07-10 Thread via GitHub
danny0405 commented on code in PR #9160: URL: https://github.com/apache/hudi/pull/9160#discussion_r1259104247 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/HoodieClientTestBase.java: ## @@ -770,6 +770,7 @@ protected void

[GitHub] [hudi] hudi-bot commented on pull request #9149: [HUDI-6511] Mark more recently added configs advanced as appropriate

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9149: URL: https://github.com/apache/hudi/pull/9149#issuecomment-1629793740 ## CI report: * 1520d0f6f10b04cc69ec1392e42d8b5e3c1d3233 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9163: [MINOR] Fix spark2.4 do not support bucket index row writer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9163: URL: https://github.com/apache/hudi/pull/9163#issuecomment-1630128022 ## CI report: * 784dbe3012ec4312a4877b389994081272ad6506 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9168: [HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9168: URL: https://github.com/apache/hudi/pull/9168#issuecomment-1630128240 ## CI report: * 4dff8041f9695a645f35d846006610be6ec85963 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8827: [DNM][Testing Only][HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #8827: URL: https://github.com/apache/hudi/pull/8827#issuecomment-1630126159 ## CI report: * 4dff8041f9695a645f35d846006610be6ec85963 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should stop all heartbeats and not delete heartbeat files for close

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1629718295 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * c72150232719f86cc0a2ee1e429b81177698487b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9136: [HUDI-6509] Add GitHub CI for Java 17

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1629936547 ## CI report: * a0e7207fb19738237d56fa0060c91cb7865ae9c0 UNKNOWN * cb101756f1bb906839b8f135b618f26205e022a9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9080: [HUDI-6445] Making some of Spark DS tests as functional

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9080: URL: https://github.com/apache/hudi/pull/9080#issuecomment-1629936364 ## CI report: * d28ff949a1dd43456fda75e5624848bb63e030f4 UNKNOWN * 61f56fd22ac70d3ede05692efc86b201bc86326a Azure:

[GitHub] [hudi] bvaradar commented on pull request #8452: [HUDI-6077] Add more partition push down filters

2023-07-10 Thread via GitHub
bvaradar commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1629985656 @boneanxs : Can you kindly rebase the PR and fix the conflicts ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] bvaradar commented on a diff in pull request #8452: [HUDI-6077] Add more partition push down filters

2023-07-10 Thread via GitHub
bvaradar commented on code in PR #8452: URL: https://github.com/apache/hudi/pull/8452#discussion_r1259089188 ## hudi-common/src/main/java/org/apache/hudi/metadata/AbstractHoodieTableMetadata.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hudi] pftn commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
pftn commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-163795 > @big-doudou > > > * Replace partition files using the repairedOutputPath in step 2 > > Can you please share your Hudi-version + stack trace, thanks. Listed in the 1st

[GitHub] [hudi] voonhous commented on issue #9162: [SUPPORT] Permission denied access ckp_meta while start flink job

2023-07-10 Thread via GitHub
voonhous commented on issue #9162: URL: https://github.com/apache/hudi/issues/9162#issuecomment-1630006589 Are you restarting the job with the hadoop user that you initially created the table with? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] voonhous commented on issue #9162: [SUPPORT] Permission denied access ckp_meta while start flink job

2023-07-10 Thread via GitHub
voonhous commented on issue #9162: URL: https://github.com/apache/hudi/issues/9162#issuecomment-1630159382 Hmmm, possible to change the permission of `/path_to_hudi_table/.hoodie/.aux/ckp_meta` to `drwx--`? ```shell hdfs dfs -chmod -R 700

[GitHub] [hudi] hudi-bot commented on pull request #8827: [DNM][Testing Only][HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #8827: URL: https://github.com/apache/hudi/pull/8827#issuecomment-1629792954 ## CI report: * dac4ff91b1c79911ffd9bd787e8f772a716b241a Azure:

[GitHub] [hudi] yihua merged pull request #9166: [DOCS] Update video and blogs

2023-07-10 Thread via GitHub
yihua merged PR #9166: URL: https://github.com/apache/hudi/pull/9166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch asf-site updated: [DOCS] Updated video and blogs (#9166)

2023-07-10 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 161d2e4efa3 [DOCS] Updated video and blogs

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] [WIP] Codreview changes for Spark SQL Merge Into for pkless tables'

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1629891978 ## CI report: * d43ca83278ca3c0222ac3635ca7de0b9db6d0f71 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should stop all heartbeats and not delete heartbeat files for close

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1629892081 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * 6358410f19457f5fbe23b209b0b1b59faee0ade4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] [WIP] Codreview changes for Spark SQL Merge Into for pkless tables'

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1629947745 ## CI report: * 10f9adc8c62d6fae7219a4472cba010a1c1c0da0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9164: [HUDI-6516] Correct the use of hoodie.bootstrap.mode.selector

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9164: URL: https://github.com/apache/hudi/pull/9164#issuecomment-1629991794 ## CI report: * 5861f76644649791a3b050c8fb0baa12d0c4135a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1630030137 ## CI report: * 75157cf41f09e386d93f2e154553e9190731095c Azure:

[GitHub] [hudi] big-doudou commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
big-doudou commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1630036429 My TM is running on k8s, and the k8s exception caused TM to exit. The error seems to have occurred after this. I will sort out the details and post them here -- This is an

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1630035617 ## CI report: * 0fa3be40442ad7e9570a5f7b3f26a75618f0447e Azure:

[GitHub] [hudi] luongngochoa commented on issue #9132: [SUPPORT] hudi deltastreamer jsonkafka source schema registry fail

2023-07-10 Thread via GitHub
luongngochoa commented on issue #9132: URL: https://github.com/apache/hudi/issues/9132#issuecomment-1630043791 @danny0405 I tried to use with this configuration but it's still not work. (follow this issues [ ](https://github.com/apache/hudi/pull/7727))

[GitHub] [hudi] SteNicholas commented on a diff in pull request #9160: [HUDI-6501] HoodieHeartbeatClient should stop all heartbeats and not delete heartbeat files for close

2023-07-10 Thread via GitHub
SteNicholas commented on code in PR #9160: URL: https://github.com/apache/hudi/pull/9160#discussion_r1259146107 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/HoodieClientTestBase.java: ## @@ -770,6 +770,7 @@ protected void

[jira] [Created] (HUDI-6519) The default value of read.streaming.enabled is determined by execution.runtime-mode

2023-07-10 Thread Nicholas Jiang (Jira)
Nicholas Jiang created HUDI-6519: Summary: The default value of read.streaming.enabled is determined by execution.runtime-mode Key: HUDI-6519 URL: https://issues.apache.org/jira/browse/HUDI-6519

[GitHub] [hudi] hudi-bot commented on pull request #9080: [HUDI-6445] Making some of Spark DS tests as functional

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9080: URL: https://github.com/apache/hudi/pull/9080#issuecomment-1629739302 ## CI report: * d28ff949a1dd43456fda75e5624848bb63e030f4 UNKNOWN * 9dadc7f288b7aac6204637e4d292cc1fb59fa540 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9167: [HUDI-6519] The default value of read.streaming.enabled is determined by execution.runtime-mode

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9167: URL: https://github.com/apache/hudi/pull/9167#issuecomment-1629942478 ## CI report: * 7a8ef331fc7f4e4862d7b0f0cd6e8c35123e6c8a Azure:

[GitHub] [hudi] pftn commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
pftn commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1629995280 > Is this problem solved? I also got the same error Fix: 1. Stop the flink job 2. Execute command by hudi-cli: repair deduplicate --duplicatedPartitionPath 20220604

[GitHub] [hudi] voonhous commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
voonhous commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1630005948 @pftn I mean @big-doudou's logs. Is he running the same version as you? Also, if this error was thrown recently on his pipeline, is it possible for him to share his JM + TM logs

[GitHub] [hudi] chandu-1101 commented on issue #9141: [SUPPORT] Example from Hudi Quick start doesnt work!

2023-07-10 Thread via GitHub
chandu-1101 commented on issue #9141: URL: https://github.com/apache/hudi/issues/9141#issuecomment-1630119428 @ ad1happy2go thank you for the reply. I will check these and get back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] SteNicholas opened a new pull request, #9167: [HUDI-6519] The default value of read.streaming.enabled is determined by execution.runtime-mode

2023-07-10 Thread via GitHub
SteNicholas opened a new pull request, #9167: URL: https://github.com/apache/hudi/pull/9167 ### Change Logs The default value of `read.streaming.enabled` could be determined by `execution.runtime-mode` from which you can choose depending on the requirements of your use case and the

[jira] [Updated] (HUDI-6519) The default value of read.streaming.enabled is determined by execution.runtime-mode

2023-07-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6519: - Labels: pull-request-available (was: ) > The default value of read.streaming.enabled is

[GitHub] [hudi] hudi-bot commented on pull request #9167: [HUDI-6519] The default value of read.streaming.enabled is determined by execution.runtime-mode

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9167: URL: https://github.com/apache/hudi/pull/9167#issuecomment-1629739737 ## CI report: * 7a8ef331fc7f4e4862d7b0f0cd6e8c35123e6c8a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9145: [HUDI-6464] [WIP] Codreview changes for Spark SQL Merge Into for pkless tables'

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9145: URL: https://github.com/apache/hudi/pull/9145#issuecomment-1629854751 ## CI report: * d43ca83278ca3c0222ac3635ca7de0b9db6d0f71 Azure:

[GitHub] [hudi] yihua merged pull request #9152: [HUDI-6112] Basic and All configs pages generated from Config doc generation tool

2023-07-10 Thread via GitHub
yihua merged PR #9152: URL: https://github.com/apache/hudi/pull/9152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] sunneebaby commented on issue #9073: Spark-3.2 Insert Into Hudi Table UnsupportedOperationException: S3A streams are not Syncable

2023-07-10 Thread via GitHub
sunneebaby commented on issue #9073: URL: https://github.com/apache/hudi/issues/9073#issuecomment-1629982525 > @sunneebaby Did you got a chance to try it? Did it worked? Yes ,I tried it.But it didn't work too.insert failed. -- This is an automated message from the Apache Git

[GitHub] [hudi] big-doudou commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
big-doudou commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1629982551 Is this problem solved? I also got the same error -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] pftn commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
pftn commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1630034977 > @pftn I mean @big-doudou's logs. > > Is he running the same version as you? Also, if this error was thrown recently on his pipeline, is it possible for him to share his JM + TM

[GitHub] [hudi] hudi-bot commented on pull request #9080: [HUDI-6445] Making some of Spark DS tests as functional

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9080: URL: https://github.com/apache/hudi/pull/9080#issuecomment-1629729006 ## CI report: * d28ff949a1dd43456fda75e5624848bb63e030f4 UNKNOWN * 9dadc7f288b7aac6204637e4d292cc1fb59fa540 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1629793624 ## CI report: * 75157cf41f09e386d93f2e154553e9190731095c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9168: [HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9168: URL: https://github.com/apache/hudi/pull/9168#issuecomment-1629793858 ## CI report: * 7f0cf39c716e969938e110e4de9b31e648eec83e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9035: [HUDI-6416] Completion markers for handling execution engine (spark) …

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9035: URL: https://github.com/apache/hudi/pull/9035#issuecomment-1629793353 ## CI report: * b29e7cad9b2f4384dba40833e09b5885809b836f Azure:

[GitHub] [hudi] stream2000 commented on pull request #9163: [MINOR] Fix spark2.4 do not support bucket index row writer

2023-07-10 Thread via GitHub
stream2000 commented on PR #9163: URL: https://github.com/apache/hudi/pull/9163#issuecomment-1629898784 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #8827: [DNM][Testing Only][HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #8827: URL: https://github.com/apache/hudi/pull/8827#issuecomment-1629898651 ## CI report: * dac4ff91b1c79911ffd9bd787e8f772a716b241a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9168: [HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9168: URL: https://github.com/apache/hudi/pull/9168#issuecomment-1629899322 ## CI report: * 7f0cf39c716e969938e110e4de9b31e648eec83e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9163: [MINOR] Fix spark2.4 do not support bucket index row writer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9163: URL: https://github.com/apache/hudi/pull/9163#issuecomment-1629899255 ## CI report: * 784dbe3012ec4312a4877b389994081272ad6506 Azure:

[GitHub] [hudi] leesf commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should stop all heartbeats and not delete heartbeat files for close

2023-07-10 Thread via GitHub
leesf commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1629944171 LGTM, let's waiting for https://github.com/apache/hudi/pull/9163 to be merged to fix CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] nylqd commented on issue #9162: [SUPPORT] Permission denied access ckp_meta while start flink job

2023-07-10 Thread via GitHub
nylqd commented on issue #9162: URL: https://github.com/apache/hudi/issues/9162#issuecomment-1630060302 > Are you restarting the job with the hadoop user that you initially created the table with? If not, possible to ensure that the job is started with the same hadoop user as the one that

[hudi] branch master updated: [HUDI-6515] Fix Spark2 do not support bucket bulk insert (#9163)

2023-07-10 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 0258a89112a [HUDI-6515] Fix Spark2 do not support

[GitHub] [hudi] leesf commented on pull request #9163: [MINOR] Fix spark2.4 do not support bucket index row writer

2023-07-10 Thread via GitHub
leesf commented on PR #9163: URL: https://github.com/apache/hudi/pull/9163#issuecomment-1630177494 The second failure case passed in first CI. Let's merge it to unblock other PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] leesf merged pull request #9163: [MINOR] Fix spark2.4 do not support bucket index row writer

2023-07-10 Thread via GitHub
leesf merged PR #9163: URL: https://github.com/apache/hudi/pull/9163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] yihua opened a new pull request, #9168: [HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
yihua opened a new pull request, #9168: URL: https://github.com/apache/hudi/pull/9168 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] hudi-bot commented on pull request #9168: [HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9168: URL: https://github.com/apache/hudi/pull/9168#issuecomment-1629801524 ## CI report: * 7f0cf39c716e969938e110e4de9b31e648eec83e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9149: [HUDI-6511] Mark more recently added configs advanced as appropriate

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9149: URL: https://github.com/apache/hudi/pull/9149#issuecomment-1629801364 ## CI report: * 1520d0f6f10b04cc69ec1392e42d8b5e3c1d3233 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9165: [HUDI-6517] Throw error if deletion of invalid data file fails

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9165: URL: https://github.com/apache/hudi/pull/9165#issuecomment-1629801470 ## CI report: * a0055f7d26380a87af8c29ec2abb0d4d1e0606d2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9131: [HUDI-6315] Feature flag for disabling optimized update/delete code path.

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9131: URL: https://github.com/apache/hudi/pull/9131#issuecomment-1629801249 ## CI report: * 75157cf41f09e386d93f2e154553e9190731095c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8827: [DNM][Testing Only][HUDI-6276] Rename HoodieDeltaStreamer to HoodieStreamer

2023-07-10 Thread via GitHub
hudi-bot commented on PR #8827: URL: https://github.com/apache/hudi/pull/8827#issuecomment-1629800648 ## CI report: * dac4ff91b1c79911ffd9bd787e8f772a716b241a Azure:

[GitHub] [hudi] voonhous commented on issue #8892: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2023-07-10 Thread via GitHub
voonhous commented on issue #8892: URL: https://github.com/apache/hudi/issues/8892#issuecomment-1629996333 @big-doudou > * Replace partition files using the repairedOutputPath in step 2 Can you please share your Hudi-version + stack trace, thanks. -- This is an

[GitHub] [hudi] hudi-bot commented on pull request #9136: [HUDI-6509] Skip tests for Java 17

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1628530586 ## CI report: * a0e7207fb19738237d56fa0060c91cb7865ae9c0 UNKNOWN * 360e04ff0549c152e0847b71d8a9651507582ec8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1628530749 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * 1463b2113c2060191da6da8e63451d46665c89ec UNKNOWN Bot commands @hudi-bot supports the

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1628543952 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * 1463b2113c2060191da6da8e63451d46665c89ec Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1628543854 ## CI report: * 8bb1ed9a5c1198cfe8e0380c8bc2130858bbccb2 Azure:

[GitHub] [hudi] nylqd opened a new issue, #9162: [SUPPORT]

2023-07-10 Thread via GitHub
nylqd opened a new issue, #9162: URL: https://github.com/apache/hudi/issues/9162 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

[jira] [Closed] (HUDI-6515) Bucket index row writer write record to wrong handle

2023-07-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6515. Resolution: Fixed Fixed via master branch: a9225bd9093e8be8f9ccdd07b0bc542994ffc4c6 > Bucket index row

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1628636634 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * 1463b2113c2060191da6da8e63451d46665c89ec Azure:

[GitHub] [hudi] stream2000 commented on a diff in pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
stream2000 commented on code in PR #9160: URL: https://github.com/apache/hudi/pull/9160#discussion_r1258103624 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/heartbeat/HoodieHeartbeatClient.java: ## @@ -201,14 +196,6 @@ public void stop(String

[GitHub] [hudi] hudi-bot commented on pull request #9156: [HUDI-6515] Fix bucket index row writer write record to wrong handle

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9156: URL: https://github.com/apache/hudi/pull/9156#issuecomment-1628294697 ## CI report: * 4416219014959fe02290bf41695abb70c31c1715 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9158: [MINOR] Unpersist only relevent metadata table RDDs

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9158: URL: https://github.com/apache/hudi/pull/9158#issuecomment-1628350667 ## CI report: * 8bb1ed9a5c1198cfe8e0380c8bc2130858bbccb2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9136: [HUDI-6509] Skip tests for Java 17

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9136: URL: https://github.com/apache/hudi/pull/9136#issuecomment-1628350541 ## CI report: * a0e7207fb19738237d56fa0060c91cb7865ae9c0 UNKNOWN * d022005cba456d694d24614db357fbd538eb290d Azure:

[GitHub] [hudi] eyjian commented on issue #9154: [SUPPORT] org.apache.hudi.exception.HoodieIOException: Unable to create

2023-07-10 Thread via GitHub
eyjian commented on issue #9154: URL: https://github.com/apache/hudi/issues/9154#issuecomment-1628364954 > Did you configure the spillable map path explicitly? Wondering if it is related with the disk space or something. Thanks. The relevant configuration items are

[GitHub] [hudi] SteNicholas opened a new pull request, #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
SteNicholas opened a new pull request, #9160: URL: https://github.com/apache/hudi/pull/9160 ### Change Logs `HoodieHeartbeatClient` should close without stopping all heartbeats started via this instance of the client. Because the start of the heartbeat is executed in `startCommit`,

[GitHub] [hudi] xzwDavid commented on issue #9159: [SUPPORT] classnotfound error for Spark3_2Adapter

2023-07-10 Thread via GitHub
xzwDavid commented on issue #9159: URL: https://github.com/apache/hudi/issues/9159#issuecomment-1628555270 > @xzwDavid you need to give only one thing. remove bundle from --jars in spark-shell command. Also I see a space after comma in the --jars list. that can create issues as well. >

[jira] [Updated] (HUDI-6514) Remove hoodie.auto.commit

2023-07-10 Thread zouxxyy (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zouxxyy updated HUDI-6514: -- Description: The usage of `hoodie.auto.commit` is very disgusting, It currently has too much hack logic and

[GitHub] [hudi] SteNicholas closed pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
SteNicholas closed pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client URL: https://github.com/apache/hudi/pull/9160 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1628649335 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * 1463b2113c2060191da6da8e63451d46665c89ec Azure:

[GitHub] [hudi] stream2000 commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
stream2000 commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1628782275 We should also note that if we don't restart the job before the heartbeat expires, the cleaner will still roll back the inflight instant that should be recommitted. And we can't set the

[GitHub] [hudi] SteNicholas closed pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
SteNicholas closed pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client URL: https://github.com/apache/hudi/pull/9160 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] hudi-bot commented on pull request #9156: [HUDI-6515] Fix bucket index row writer write record to wrong handle

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9156: URL: https://github.com/apache/hudi/pull/9156#issuecomment-1628559974 ## CI report: * 5846f14798533566181bb166c08d06e24f316ba6 Azure:

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #6673: [HUDI-4785] Fix partition discovery in bootstrap operation

2023-07-10 Thread via GitHub
Zouxxyy commented on code in PR #6673: URL: https://github.com/apache/hudi/pull/6673#discussion_r1257984792 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/bootstrap/SparkBootstrapCommitActionExecutor.java: ## @@ -307,12 +321,21 @@ private Map>>>

[GitHub] [hudi] danny0405 merged pull request #9156: [HUDI-6515] Fix bucket index row writer write record to wrong handle

2023-07-10 Thread via GitHub
danny0405 merged PR #9156: URL: https://github.com/apache/hudi/pull/9156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated: [HUDI-6515] Fix bucket index row writer write record to wrong handle (#9156)

2023-07-10 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new a9225bd9093 [HUDI-6515] Fix bucket index row

[jira] [Updated] (HUDI-6515) Bucket index row writer write record to wrong handle

2023-07-10 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6515: - Fix Version/s: 0.14.0 > Bucket index row writer write record to wrong handle >

[GitHub] [hudi] hudi-bot commented on pull request #9160: [HUDI-6501] HoodieHeartbeatClient should close without stopping all heartbeats started via this instance of the client

2023-07-10 Thread via GitHub
hudi-bot commented on PR #9160: URL: https://github.com/apache/hudi/pull/9160#issuecomment-1628623844 ## CI report: * b2a0e7e0a2539122bc9178f0b2e2283a175c9de8 UNKNOWN * 1463b2113c2060191da6da8e63451d46665c89ec Azure:

[GitHub] [hudi] chandu-1101 commented on issue #9141: [SUPPORT] unable to write dataframe into s3. Getting: HoodieSparkSqlWriter$: UPSERT failed with errors

2023-07-10 Thread via GitHub
chandu-1101 commented on issue #9141: URL: https://github.com/apache/hudi/issues/9141#issuecomment-1628679634 Any update? is the above explanation missing something important? If i modify the above program to the below, the program returns in few seconds (the parquet file in the src is

[GitHub] [hudi] Zouxxyy opened a new issue, #9161: [SUPPORT] bootstrap mode modification and remove `hoodie.auto.commit`

2023-07-10 Thread via GitHub
Zouxxyy opened a new issue, #9161: URL: https://github.com/apache/hudi/issues/9161 **_Tips before filing an issue_** The following is the current configuration of bootstrap's mode selection: - hoodie.bootstrap.mode.selector: class - hoodie.bootstrap.mode.selector.regex -

[GitHub] [hudi] wqwl611 commented on issue #9138: [SUPPORT]Read Hive then write Hudi,but org.apache.hadoop.hive.ql.metadata.AuthorizationException: No privilege 'Update' found for outputs

2023-07-10 Thread via GitHub
wqwl611 commented on issue #9138: URL: https://github.com/apache/hudi/issues/9138#issuecomment-1628790274 > Seems an issue of privilege, did you try to grant the permission with cmd: `grant xxx_action on database xxx_database to user xxx_user;` @danny0405 thanks reply, there is a bug

[GitHub] [hudi] wqwl611 closed issue #9138: [SUPPORT]Read Hive then write Hudi,but org.apache.hadoop.hive.ql.metadata.AuthorizationException: No privilege 'Update' found for outputs

2023-07-10 Thread via GitHub
wqwl611 closed issue #9138: [SUPPORT]Read Hive then write Hudi,but org.apache.hadoop.hive.ql.metadata.AuthorizationException: No privilege 'Update' found for outputs URL: https://github.com/apache/hudi/issues/9138 -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] codope merged pull request #9157: [HUDI-6246] Adding tests to validate restore of compaction commit

2023-07-10 Thread via GitHub
codope merged PR #9157: URL: https://github.com/apache/hudi/pull/9157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated: [HUDI-6246] Add tests to validate restore of compaction commit (#9157)

2023-07-10 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 9744de26709 [HUDI-6246] Add tests to validate

[GitHub] [hudi] ad1happy2go commented on issue #9143: [SUPPORT] Failure to delete records with missing attributes from PostgresDebeziumSource

2023-07-10 Thread via GitHub
ad1happy2go commented on issue #9143: URL: https://github.com/apache/hudi/issues/9143#issuecomment-1628311249 @sydneyhoran Are you using master version of code? Now sure if This PR - https://github.com/apache/hudi/pull/8690 will fix this issue. Are you facing this issue with this PR also?

  1   2   >