[GitHub] [hudi] ad1happy2go commented on issue #7363: [SUPPORT] how to get hudi table schema and get table list under the same database

ad1happy2go commented on issue #7363: URL: https://github.com/apache/hudi/issues/7363#issuecomment-1529375213 @zengqinchris Closing this issue as we have the explanation and workaround also. Please reopen in case you have any more issues on the same. -- This is an automated message from

[GitHub] [hudi] ad1happy2go commented on issue #7249: [SUPPORT] How to run cleaner table service on DFS source of DeltaStreamer ?

ad1happy2go commented on issue #7249: URL: https://github.com/apache/hudi/issues/7249#issuecomment-1529367537 @rtdt99 Closing this bug as above comment clarifies. Please reopen in case of any more concerns on same. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] ad1happy2go commented on issue #7242: [SUPPORT] Partition field value lost in table column

ad1happy2go commented on issue #7242: URL: https://github.com/apache/hudi/issues/7242#issuecomment-1529365385 @Priyanka128 @ROOBALJINDAL The correct way is to add a new date column before writing to hudi. For the same you can use "SqlQueryBasedTransformer". -- This is an automated

[GitHub] [hudi] hudi-bot commented on pull request #8611: [HUDI-6157] Fix potential data loss for flink streaming source from table with multi writer

hudi-bot commented on PR #8611: URL: https://github.com/apache/hudi/pull/8611#issuecomment-1529347685 ## CI report: * b184b111c6928408d082ce73486f5bd3ae7c6683 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8611: [HUDI-6157] Fix potential data loss for flink streaming source from table with multi writer

hudi-bot commented on PR #8611: URL: https://github.com/apache/hudi/pull/8611#issuecomment-1529344411 ## CI report: * b184b111c6928408d082ce73486f5bd3ae7c6683 Azure:

[GitHub] [hudi] ad1happy2go commented on issue #7141: [SUPPORT] Question on Bootstrapped hudi table

ad1happy2go commented on issue #7141: URL: https://github.com/apache/hudi/issues/7141#issuecomment-1529333245 @rtdt99 Currently HoodieSnapshotExporter doesn't provide that functionality. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] hudi-bot commented on pull request #8613: [HUDI-6158] Strengthen Flink clustering commit and rollback strategy

hudi-bot commented on PR #8613: URL: https://github.com/apache/hudi/pull/8613#issuecomment-1529319205 ## CI report: * ff24199cad215049cc4274aae3a1008bf7053c90 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8613: [HUDI-6158] Strengthen Flink clustering commit and rollback strategy

hudi-bot commented on PR #8613: URL: https://github.com/apache/hudi/pull/8613#issuecomment-1529315186 ## CI report: * ff24199cad215049cc4274aae3a1008bf7053c90 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Commented] (HUDI-2751) To avoid the duplicates for streaming read MOR table

[ https://issues.apache.org/jira/browse/HUDI-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718151#comment-17718151 ] Danny Chen commented on HUDI-2751: -- > So, no records from the new base parquet file created from

[jira] [Closed] (HUDI-2751) To avoid the duplicates for streaming read MOR table

[ https://issues.apache.org/jira/browse/HUDI-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-2751. Fix Version/s: 0.12.0 0.11.0 Resolution: Fixed > To avoid the duplicates for

[jira] [Updated] (HUDI-6158) Strengthen Flink clustering commit and rollback strategy

[ https://issues.apache.org/jira/browse/HUDI-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6158: - Labels: pull-request-available (was: ) > Strengthen Flink clustering commit and rollback

[GitHub] [hudi] SteNicholas opened a new pull request, #8613: [HUDI-6158] Strengthen Flink clustering commit and rollback strategy

SteNicholas opened a new pull request, #8613: URL: https://github.com/apache/hudi/pull/8613 ### Change Logs `ClusteringCommitSink` could strengthen commit and rollback strategy from two solutions: - Commit: Introduces `clusteringPlanCache` that caches to store clustering plan

[jira] [Created] (HUDI-6158) Strengthen Flink clustering commit and rollback strategy

Nicholas Jiang created HUDI-6158: Summary: Strengthen Flink clustering commit and rollback strategy Key: HUDI-6158 URL: https://issues.apache.org/jira/browse/HUDI-6158 Project: Apache Hudi

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

nsivabalan commented on code in PR #7826: URL: https://github.com/apache/hudi/pull/7826#discussion_r1181278240 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/functional/TestHoodieSparkMergeOnReadTableRollback.java: ## @@ -912,6 +914,62 @@ public void

[GitHub] [hudi] nsivabalan commented on pull request #7826: [HUDI-5675] fix lazy clean schedule rollback on completed instant

nsivabalan commented on PR #7826: URL: https://github.com/apache/hudi/pull/7826#issuecomment-1529103489 sorry, dropped the ball. reviewing it again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Commented] (HUDI-3694) Not use magic number of next block to determine current log block

[ https://issues.apache.org/jira/browse/HUDI-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718089#comment-17718089 ] sivabalan narayanan commented on HUDI-3694: --- This might potentially be an issue only for hdfs

[jira] [Commented] (HUDI-2751) To avoid the duplicates for streaming read MOR table

[ https://issues.apache.org/jira/browse/HUDI-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718088#comment-17718088 ] sivabalan narayanan commented on HUDI-2751: --- we do preserve commit metadata for compaction. So,

[GitHub] [hudi] samserpoosh commented on issue #8521: [SUPPORT] Deltastreamer not recognizing config `hoodie.deltastreamer.source.kafka.value.deserializer.class` with PostgresDebeziumSource

samserpoosh commented on issue #8521: URL: https://github.com/apache/hudi/issues/8521#issuecomment-1529090886 @sydneyhoran You might have seen this already, but just in case, I stumbled upon [this comment](https://github.com/apache/hudi/issues/6348#issuecomment-1223742672) which mentioned

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8604: [HUDI-6151] Rollback previously applied commits to MDT when operations are retried.

nsivabalan commented on code in PR #8604: URL: https://github.com/apache/hudi/pull/8604#discussion_r1181266570 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieBackedTableMetadataWriter.java: ## @@ -161,27 +161,28 @@ protected void

[hudi] branch master updated (1fa9e37df92 -> 04e54a6187d)

This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 1fa9e37df92 [HUDI-6031] fix bug: checkpoint lost after changing cow to mor (#8378) add 04e54a6187d Revert

[GitHub] [hudi] xushiyan merged pull request #8612: Revert "[MINOR] enable publish test results"

xushiyan merged PR #8612: URL: https://github.com/apache/hudi/pull/8612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8493: [HUDI-6098] Use bulk insert prepped for the initial write into MDT.

nsivabalan commented on code in PR #8493: URL: https://github.com/apache/hudi/pull/8493#discussion_r1181264755 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1004,6 +1004,57 @@ public static int mapRecordKeyToFileGroupIndex(String

[GitHub] [hudi] xushiyan opened a new pull request, #8612: Revert "[MINOR] enable publish test results"

xushiyan opened a new pull request, #8612: URL: https://github.com/apache/hudi/pull/8612 Reverts apache/hudi#7943 as this is actually costing money -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[hudi] branch revert-7943-publish_test_results created (now c7f96b31612)

This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a change to branch revert-7943-publish_test_results in repository https://gitbox.apache.org/repos/asf/hudi.git at c7f96b31612 Revert "[MINOR] Enable Azure CI to publish test results (#7943)" This branch

[hudi] 01/01: Revert "[MINOR] Enable Azure CI to publish test results (#7943)"

This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch revert-7943-publish_test_results in repository https://gitbox.apache.org/repos/asf/hudi.git commit c7f96b316128ff7647e8e296fde1ee90689053de Author: Shiyan Xu

[GitHub] [hudi] ad1happy2go commented on issue #6881: Processing time is increased with hudi metadata enable

ad1happy2go commented on issue #6881: URL: https://github.com/apache/hudi/issues/6881#issuecomment-1529064535 @koochiswathiTR Can you test out with latest version of hudi. We have done lots of improvement related to Metadata server. Let us know if you still see the error. -- This is an

[GitHub] [hudi] ad1happy2go commented on issue #6750: [SUPPORT] SqlQueryBasedTransformer causes memory issues

ad1happy2go commented on issue #6750: URL: https://github.com/apache/hudi/issues/6750#issuecomment-1529062128 @tzhang-fetch Couldn't reproduce this issue as SQL transformer is working fine. Are you saying with same executor and driver memory , hudi job got killed when using sql

[GitHub] [hudi] ad1happy2go commented on issue #6596: [SUPPORT] with Impala 4.0 Records lost

ad1happy2go commented on issue #6596: URL: https://github.com/apache/hudi/issues/6596#issuecomment-1529061441 @zhengyuan-cn Are you still facing this issue with latest Hudi version. Can you test out with any version after 0.12 and let us know. -- This is an automated message from the

[GitHub] [hudi] ad1happy2go commented on issue #5687: [SUPPORT]hudi sql parser ignores all exceptions of spark sql parser

ad1happy2go commented on issue #5687: URL: https://github.com/apache/hudi/issues/5687#issuecomment-1529058559 @melin Closing this issue as couldn't reproduce it. In both the cases (spark-sql and spark-shell), only spark exception informations is been thrown. Please reopen if the

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8527: [HUDI-6117] Parallelize the initial creation of file groups for a new MDT partition.

nsivabalan commented on code in PR #8527: URL: https://github.com/apache/hudi/pull/8527#discussion_r1181247986 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -731,21 +733,40 @@ public void

[GitHub] [hudi] hudi-bot commented on pull request #8596: [BUG-FIX] use try with resource to close stream

hudi-bot commented on PR #8596: URL: https://github.com/apache/hudi/pull/8596#issuecomment-1528982268 ## CI report: * 0c8c7d99fc250191a7eba156052f01371e431a30 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8611: [HUDI-6157] Fix potential data loss for flink streaming source from table with multi writer

hudi-bot commented on PR #8611: URL: https://github.com/apache/hudi/pull/8611#issuecomment-1528970694 ## CI report: * b184b111c6928408d082ce73486f5bd3ae7c6683 Azure:

[hudi] branch master updated: [HUDI-6031] fix bug: checkpoint lost after changing cow to mor (#8378)

This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1fa9e37df92 [HUDI-6031] fix bug: checkpoint lost

[GitHub] [hudi] bvaradar merged pull request #8378: [HUDI-6031] fix bug: checkpoint lost after changing cow to mor

bvaradar merged PR #8378: URL: https://github.com/apache/hudi/pull/8378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: