Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1823920564 ## CI report: * 190b9df539423cb5da8f01b400426d9e97f7bab4 Azure:

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10002: URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823920316 ## CI report: * 35fed0de0587b411f9470e1c69db43501df5a725 Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1823920594 ## CI report: * 3b24d4130099aab67c76de81f77701c730f2e78a Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
empcl commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1823916790 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10002: URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823913218 ## CI report: * 35fed0de0587b411f9470e1c69db43501df5a725 Azure:

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823907351 ## CI report: * 868ba59ecf1a08d7b73a7121429103c2134b291f Azure:

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10002: URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823907225 ## CI report: * 35fed0de0587b411f9470e1c69db43501df5a725 Azure:

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
danny0405 commented on code in PR #10002: URL: https://github.com/apache/hudi/pull/10002#discussion_r1402973135 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/RocksDbBasedFileSystemView.java: ## @@ -553,6 +553,10 @@ protected void

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
danny0405 commented on code in PR #10002: URL: https://github.com/apache/hudi/pull/10002#discussion_r1402973135 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/RocksDbBasedFileSystemView.java: ## @@ -553,6 +553,10 @@ protected void

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
danny0405 commented on code in PR #10002: URL: https://github.com/apache/hudi/pull/10002#discussion_r1402972184 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java: ## @@ -202,6 +207,13 @@ private Map

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
danny0405 commented on code in PR #10002: URL: https://github.com/apache/hudi/pull/10002#discussion_r1402970319 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -598,35 +603,49 @@ private FileSlice

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-22 Thread via GitHub
danny0405 commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1823886771 Still got some compile issues: ```scala Error: /home/runner/work/hudi/hudi/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestHoodieFileIndex.scala:249:

Re: [I] [SUPPORT] Flink SQL client cow table query error "org/apache/parquet/column/ColumnDescriptor" (but mor table query normal) [hudi]

2023-11-22 Thread via GitHub
xiaolan-bit commented on issue #6297: URL: https://github.com/apache/hudi/issues/6297#issuecomment-1823881483 is the jar about parquet have question? my flink version is 1.17.1 but the parquet version is 1.13.0 -- This is an automated message from the Apache Git Service. To respond to

Re: [I] [SUPPORT] Flink SQL client cow table query error "org/apache/parquet/column/ColumnDescriptor" (but mor table query normal) [hudi]

2023-11-22 Thread via GitHub
xiaolan-bit commented on issue #6297: URL: https://github.com/apache/hudi/issues/6297#issuecomment-1823880643 when i use select * ,an error appear:java.lang.LinkageError: org/apache/parquet/column/ColumnDescriptor at

Re: [I] [SUPPORT] Flink SQL client cow table query error "org/apache/parquet/column/ColumnDescriptor" (but mor table query normal) [hudi]

2023-11-22 Thread via GitHub
xiaolan-bit commented on issue #6297: URL: https://github.com/apache/hudi/issues/6297#issuecomment-1823878299 how to slove this question? add or replace any jar? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10002: URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823878156 ## CI report: * fc27baa8c2df9135bc6e4b0d14e50a127ecb434f Azure:

Re: [I] [SUPPORT] The INSERT records are marked as UPDATE [hudi]

2023-11-22 Thread via GitHub
zdl1 commented on issue #10156: URL: https://github.com/apache/hudi/issues/10156#issuecomment-1823873879 > there is no way to figure out whether a key has been written to an existing bucket before, except the first file slice, so all the records are updates. Thanks for the

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1823873637 ## CI report: * 032ad417971148eec41a5d41066b37d238ecf70a Azure:

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1823868662 ## CI report: * 032ad417971148eec41a5d41066b37d238ecf70a Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
zhangyue19921010 commented on code in PR #10157: URL: https://github.com/apache/hudi/pull/10157#discussion_r1402949484 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HoodieHiveCatalog.java: ## @@ -510,6 +511,9 @@ private void

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823839975 ## CI report: * 91b8b5ff8242d5fa0f01fc78ba55f70d458e58c9 Azure:

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
nsivabalan commented on code in PR #10158: URL: https://github.com/apache/hudi/pull/10158#discussion_r1402927461 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java: ## @@ -801,24 +765,25 @@ private HoodieWriteConfig

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823835454 ## CI report: * 91b8b5ff8242d5fa0f01fc78ba55f70d458e58c9 Azure:

(hudi) branch master updated: [HUDI-7120] Performance improvements in deltastreamer executor code path (#10135)

2023-11-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b77eff2522a [HUDI-7120] Performance

Re: [PR] [HUDI-7120] Performance improvements in deltastreamer executor code path [hudi]

2023-11-22 Thread via GitHub
nsivabalan merged PR #10135: URL: https://github.com/apache/hudi/pull/10135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(hudi) branch master updated: [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) (#10095)

2023-11-22 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 405be173664 [MINOR] Making misc fixes to

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-22 Thread via GitHub
codope merged PR #10095: URL: https://github.com/apache/hudi/pull/10095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]

2023-11-22 Thread via GitHub
danny0405 commented on PR #10002: URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823820723 Thanks for the contribution, I have reviewed and created a patch: [7041.patch.zip](https://github.com/apache/hudi/files/13446123/7041.patch.zip) -- This is an automated

(hudi) branch master updated (72ff9a7f0c9 -> 3d212853724)

2023-11-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 72ff9a7f0c9 [HUDI-7052] Fix partition key validation for custom key generators. (#10014) add 3d212853724

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-22 Thread via GitHub
nsivabalan merged PR #10122: URL: https://github.com/apache/hudi/pull/10122 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-22 Thread via GitHub
nsivabalan commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1823818490 https://github.com/apache/hudi/assets/513218/43a50fef-afef-4a80-b54a-75d5fe1260d3;> -- This is an automated message from the Apache Git Service. To respond to the message, please

(hudi) branch master updated: [HUDI-7052] Fix partition key validation for custom key generators. (#10014)

2023-11-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 72ff9a7f0c9 [HUDI-7052] Fix partition key

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-22 Thread via GitHub
nsivabalan merged PR #10014: URL: https://github.com/apache/hudi/pull/10014 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-22 Thread via GitHub
nsivabalan commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1823817928 https://github.com/apache/hudi/assets/513218/f0efc544-a78a-4ee3-bed7-f403aea335fb;> -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1823807872 ## CI report: * c8c49d513c8b91b2ff8462f6db25203ba563d39a Azure:

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1823804491 ## CI report: * c8c49d513c8b91b2ff8462f6db25203ba563d39a Azure:

Re: [I] [SUPPORT] Spark job stuck after completion, due to some non daemon threads still running [hudi]

2023-11-22 Thread via GitHub
zyclove commented on issue #9826: URL: https://github.com/apache/hudi/issues/9826#issuecomment-1823781426 Hi, this issue occurs frequently, has it been resolved? As https://issues.apache.org/jira/browse/HUDI-6980 is not closed. When will version 0.14.1 be released? There is an urgent

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823776282 ## CI report: * 91b8b5ff8242d5fa0f01fc78ba55f70d458e58c9 Azure:

Re: [I] Cannot encode decimal with precision 15 as max precision 14 [hudi]

2023-11-22 Thread via GitHub
njalan closed issue #10160: Cannot encode decimal with precision 15 as max precision 14 URL: https://github.com/apache/hudi/issues/10160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-22 Thread via GitHub
majian1998 commented on code in PR #10120: URL: https://github.com/apache/hudi/pull/10120#discussion_r1402861158 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ShowMetadataTableColumnStatsProcedure.scala: ## @@ -0,0 +1,169 @@ +/*

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-22 Thread via GitHub
stream2000 commented on code in PR #10120: URL: https://github.com/apache/hudi/pull/10120#discussion_r1402852595 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ShowMetadataTableColumnStatsProcedure.scala: ## @@ -0,0 +1,169 @@ +/*

[jira] [Closed] (HUDI-7110) Add call procedure for show column stats information

2023-11-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7110. Resolution: Fixed Fixed via master branch: 8d6d04387753662a5bb41f35874c6bbdd7021b36 > Add call procedure

[jira] [Updated] (HUDI-7110) Add call procedure for show column stats information

2023-11-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7110: - Fix Version/s: 1.0.0 > Add call procedure for show column stats information >

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-22 Thread via GitHub
danny0405 merged PR #10120: URL: https://github.com/apache/hudi/pull/10120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(hudi) branch master updated: [HUDI-7110] Add call procedure for show column stats information (#10120)

2023-11-22 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8d6d0438775 [HUDI-7110] Add call procedure for

Re: [I] Cannot encode decimal with precision 15 as max precision 14 [hudi]

2023-11-22 Thread via GitHub
njalan commented on issue #10160: URL: https://github.com/apache/hudi/issues/10160#issuecomment-1823733255 @ad1happy2go It is already merged in 0.13.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10095: URL: https://github.com/apache/hudi/pull/10095#issuecomment-1823714576 ## CI report: * c1b5bd41ac1f4be476fb69f84f7197a27733eb23 Azure:

(hudi) branch master updated: [MINOR] Remove unused import (#10159)

2023-11-22 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new aabaa9947fc [MINOR] Remove unused import (#10159)

Re: [PR] [MINOR] Remove unused import [hudi]

2023-11-22 Thread via GitHub
leesf merged PR #10159: URL: https://github.com/apache/hudi/pull/10159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[I] [SUPPORT] Async Clustering: Seeking Help on Specific Partitioning and Regex Pattern [hudi]

2023-11-22 Thread via GitHub
soumilshah1995 opened a new issue, #10165: URL: https://github.com/apache/hudi/issues/10165 Subject : Async Clustering: Seeking Help on Specific Partitioning and Regex Pattern I'm currently exploring async clustering in Apache Hudi, and this is also intended for a community

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823692854 ## CI report: * 48df6bbec2473dbbbedb1b723896acb17056e80f Azure:

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
rmahindra123 commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823692075 Approved -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [HUDI-7086] Scaling gcs event source [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10073: URL: https://github.com/apache/hudi/pull/10073#issuecomment-1823688411 ## CI report: * 48df6bbec2473dbbbedb1b723896acb17056e80f Azure:

[PR] [MINOR] update disaster recovery docs [hudi]

2023-11-22 Thread via GitHub
sagarlakshmipathy opened a new pull request, #10164: URL: https://github.com/apache/hudi/pull/10164 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ > added for loop to avoid copy pasting code > added note to make sure users

Re: [PR] Asf site update disaster recovery doc [hudi]

2023-11-22 Thread via GitHub
sagarlakshmipathy closed pull request #10163: Asf site update disaster recovery doc URL: https://github.com/apache/hudi/pull/10163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1823617478 ## CI report: * 5e60b3d12b40a04006d3697fa99538e9e494b96c Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1823590565 ## CI report: * cae921ac9d016d28b87139b5c0fd24debadf1592 Azure:

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-22 Thread via GitHub
nsivabalan commented on code in PR #10095: URL: https://github.com/apache/hudi/pull/10095#discussion_r1402767000 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/S3EventsHoodieIncrSource.java: ## @@ -70,6 +72,7 @@ public class S3EventsHoodieIncrSource extends

[PR] Asf site update disaster recovery doc [hudi]

2023-11-22 Thread via GitHub
sagarlakshmipathy opened a new pull request, #10163: URL: https://github.com/apache/hudi/pull/10163 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ > added for loops in 2 places to avoid copy pasting effort > fixed indentation

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10095: URL: https://github.com/apache/hudi/pull/10095#issuecomment-1823585243 ## CI report: * a6476f06265d7600755e5597af173fea6db2954f Azure:

Re: [PR] [MINOR] Making misc fixes to deltastreamer sources(S3 and GCS) [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10095: URL: https://github.com/apache/hudi/pull/10095#issuecomment-1823579673 ## CI report: * a6476f06265d7600755e5597af173fea6db2954f Azure:

Re: [PR] [HUDI-6734] Add back HUDI-5409: Avoid file index and use fs view cache in COW input format [hudi]

2023-11-22 Thread via GitHub
nsivabalan commented on code in PR #9567: URL: https://github.com/apache/hudi/pull/9567#discussion_r1402761264 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java: ## @@ -241,31 +246,86 @@ private List listStatusForSnapshotMode(JobConf

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1823515572 ## CI report: * 597f6d7bd7134d635ad5a675bd398ba03faafef8 Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1823471080 ## CI report: * 597f6d7bd7134d635ad5a675bd398ba03faafef8 Azure:

Re: [PR] [HUDI-7136] in the dfs catalog scenario, solve the problem of Primary key definit… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10162: URL: https://github.com/apache/hudi/pull/10162#issuecomment-1823447272 ## CI report: * 64589da09eb106b1fc771ca77b64d30c81ae5970 Azure:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1823446823 ## CI report: * 80725367a7e21160545ffa27ec1275a32e47e7c4 Azure:

Re: [PR] [HUDI-7052] Fix partition key validation for custom key generators. [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10014: URL: https://github.com/apache/hudi/pull/10014#issuecomment-1823391577 ## CI report: * 80725367a7e21160545ffa27ec1275a32e47e7c4 Azure:

[I] [SUPPORT] Schema evolution error: promoted data type from integer to double [hudi]

2023-11-22 Thread via GitHub
kenny291 opened a new issue, #3558: URL: https://github.com/apache/hudi/issues/3558 **Description** Hi all, I tested schema evolution change data type from int to double, but it did not work with Hudi. (hudi doc:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1823352506 ## CI report: * 3b24d4130099aab67c76de81f77701c730f2e78a Azure:

(hudi) branch master updated: [HUDI-7123] Improve CI scripts (#10136)

2023-11-22 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f88a73f09e7 [HUDI-7123] Improve CI scripts

Re: [PR] [HUDI-7123] Improve CI scripts [hudi]

2023-11-22 Thread via GitHub
yihua merged PR #10136: URL: https://github.com/apache/hudi/pull/10136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7136] in the dfs catalog scenario, solve the problem of Primary key definit… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10162: URL: https://github.com/apache/hudi/pull/10162#issuecomment-1823196032 ## CI report: * 64589da09eb106b1fc771ca77b64d30c81ae5970 Azure:

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1823195959 ## CI report: * 1ecd7d0aaf9a406be3d134a0202911a7b32f05bd Azure:

[jira] [Updated] (HUDI-7135) Spark reads hudi table error when flink creates the table without preCombine fields by catalog or factory

2023-11-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7135: - Labels: pull-request-available (was: ) > Spark reads hudi table error when flink creates the

Re: [PR] [HUDI-7135] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1823184561 ## CI report: * 1ecd7d0aaf9a406be3d134a0202911a7b32f05bd Azure:

Re: [PR] [HUDI-7136] in the dfs catalog scenario, solve the problem of Primary key definit… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10162: URL: https://github.com/apache/hudi/pull/10162#issuecomment-1823184657 ## CI report: * 64589da09eb106b1fc771ca77b64d30c81ae5970 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [MINOR] Remove unused import [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10159: URL: https://github.com/apache/hudi/pull/10159#issuecomment-1823173019 ## CI report: * 72e6a610b88f3d269477fd967b970c48fbc6f387 Azure:

[jira] [Updated] (HUDI-7136) in the dfs catalog scenario, solve the problem of Primary key definition is missing

2023-11-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7136: - Labels: pull-request-available (was: ) > in the dfs catalog scenario, solve the problem of

[PR] [HUDI-7136] in the dfs catalog scenario, solve the problem of Primary key definit… [hudi]

2023-11-22 Thread via GitHub
empcl opened a new pull request, #10162: URL: https://github.com/apache/hudi/pull/10162 …ion is missing ### Change Logs in the dfs catalog scenario, solve the problem of Primary key definition is missing ### Impact no ### Risk level (write none, low medium or

[jira] [Updated] (HUDI-7136) in the dfs catalog scenario, solve the problem of Primary key definition is missing

2023-11-22 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 陈磊 updated HUDI-7136: - Description: in the dfs catalog scenario, solve the problem of Primary key definition is missing demo: {code:java} //

[jira] [Updated] (HUDI-7136) in the dfs catalog scenario, solve the problem of Primary key definition is missing

2023-11-22 Thread Jira
[ https://issues.apache.org/jira/browse/HUDI-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 陈磊 updated HUDI-7136: - Description: in the dfs catalog scenario, solve the problem of Primary key definition is missing demo: {code:java} //

[jira] [Created] (HUDI-7136) in the dfs catalog scenario, solve the problem of Primary key definition is missing

2023-11-22 Thread Jira
陈磊 created HUDI-7136: Summary: in the dfs catalog scenario, solve the problem of Primary key definition is missing Key: HUDI-7136 URL: https://issues.apache.org/jira/browse/HUDI-7136 Project: Apache Hudi

Re: [I] Cannot encode decimal with precision 15 as max precision 14 [hudi]

2023-11-22 Thread via GitHub
ad1happy2go commented on issue #10160: URL: https://github.com/apache/hudi/issues/10160#issuecomment-1823026376 @njalan I remember a similar issue before also, This issue got fixed in this PR - https://github.com/apache/hudi/pull/8063 -- This is an automated message from the Apache

[jira] [Created] (HUDI-7135) Spark reads hudi table error when flink creates the table without preCombine fields by catalog or factory

2023-11-22 Thread Jira
陈磊 created HUDI-7135: Summary: Spark reads hudi table error when flink creates the table without preCombine fields by catalog or factory Key: HUDI-7135 URL: https://issues.apache.org/jira/browse/HUDI-7135

Re: [PR] [HUDI-7034] Refresh index fix - remove cached file slices within part… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10151: URL: https://github.com/apache/hudi/pull/10151#issuecomment-1822983761 ## CI report: * 190b9df539423cb5da8f01b400426d9e97f7bab4 Azure:

(hudi) branch master updated: [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources (#10152)

2023-11-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 38c87b7ebe1 [HUDI-7004] Add support of

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-11-22 Thread via GitHub
nsivabalan merged PR #10152: URL: https://github.com/apache/hudi/pull/10152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(hudi) branch master updated (cda9dbca206 -> d0edfb55ca2)

2023-11-22 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from cda9dbca206 [HUDI-7129] Fix bug when upgrade from table version three using UpgradeOrDowngradeProcedure (#10147)

Re: [PR] [HUDI-6961] Fixing DefaultHoodieRecordPayload to honor deletion based on meta field as well as custome delete marker [hudi]

2023-11-22 Thread via GitHub
nsivabalan merged PR #10150: URL: https://github.com/apache/hudi/pull/10150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Spark reads hudi table error when flink creates the table without pre… [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10157: URL: https://github.com/apache/hudi/pull/10157#issuecomment-1822968869 ## CI report: * 1ecd7d0aaf9a406be3d134a0202911a7b32f05bd Azure:

Re: [PR] [HUDI-7110] Add call procedure for show column stats information [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10120: URL: https://github.com/apache/hudi/pull/10120#issuecomment-1822968498 ## CI report: * a7f986bd546e2c38c241ee743734dbec491b0351 Azure:

Re: [PR] in the dfs catalog scenario, solve the problem of Primary key definit… [hudi]

2023-11-22 Thread via GitHub
empcl closed pull request #10161: in the dfs catalog scenario, solve the problem of Primary key definit… URL: https://github.com/apache/hudi/pull/10161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] in the dfs catalog scenario, solve the problem of Primary key definit… [hudi]

2023-11-22 Thread via GitHub
empcl opened a new pull request, #10161: URL: https://github.com/apache/hudi/pull/10161 …ion is missing ### Change Logs in the dfs catalog scenario, solve the problem of Primary key definition is missing ### Impact no ### Risk level (write none, low medium

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1822878847 ## CI report: * c8c49d513c8b91b2ff8462f6db25203ba563d39a Azure:

Re: [PR] [HUDI-7112] Reuse existing timeline server and performance improvements [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10122: URL: https://github.com/apache/hudi/pull/10122#issuecomment-1822849025 ## CI report: * 597f6d7bd7134d635ad5a675bd398ba03faafef8 Azure:

[I] Cannot encode decimal with precision 15 as max precision 14 [hudi]

2023-11-22 Thread via GitHub
njalan opened a new issue, #10160: URL: https://github.com/apache/hudi/issues/10160 Got below error message when try to load data from postgresql into hudi, But it is working fine on hudi 0.9. Caused by: org.apache.hudi.exception.HoodieException:

Re: [PR] [MINOR] Remove unused import [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10159: URL: https://github.com/apache/hudi/pull/10159#issuecomment-1822776587 ## CI report: * 72e6a610b88f3d269477fd967b970c48fbc6f387 Azure:

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1822776507 ## CI report: * c8c49d513c8b91b2ff8462f6db25203ba563d39a Azure:

Re: [PR] [MINOR] Remove unused import [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10159: URL: https://github.com/apache/hudi/pull/10159#issuecomment-1822763645 ## CI report: * 72e6a610b88f3d269477fd967b970c48fbc6f387 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7006] Reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-11-22 Thread via GitHub
hudi-bot commented on PR #10158: URL: https://github.com/apache/hudi/pull/10158#issuecomment-1822763563 ## CI report: * c8c49d513c8b91b2ff8462f6db25203ba563d39a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

  1   2   >