Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua merged PR #10615: URL: https://github.com/apache/hudi/pull/10615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2052767220 ## CI report: * 805ba35b65afbb1daccbcf00291fd520a69c5584 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2052703726 ## CI report: * dfab8e1285bf0241eea2e71f9d85607c647446d7 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2052699814 ## CI report: * dfab8e1285bf0241eea2e71f9d85607c647446d7 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1563323590 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -530,6 +539,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1563245298 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestSparkSqlWithCustomKeyGenerator.scala: ## @@ -0,0 +1,571 @@ +/* + * Licensed to the

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1563198254 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -530,6 +539,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1563196323 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -530,6 +539,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1563151868 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -528,6 +536,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
yihua commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2052277245 > I like that this has the benefit of not breaking tables with their existing hoodie.table.recordkey.fields, but I am curious about any other approaches you thought about. From you test

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
jonvex commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1562569055 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala: ## @@ -201,8 +201,26 @@ object HoodieWriterUtils {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-12 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2051092856 ## CI report: * dfab8e1285bf0241eea2e71f9d85607c647446d7 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050971324 ## CI report: * 989ffd5220e4f5ae666a05afdd0e7de3c6543972 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050965097 ## CI report: * 989ffd5220e4f5ae666a05afdd0e7de3c6543972 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050776752 ## CI report: * 989ffd5220e4f5ae666a05afdd0e7de3c6543972 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050723314 ## CI report: * 50b27846bf118909f3fd69f20cf5d7654d8a87c7 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050717174 ## CI report: * 50b27846bf118909f3fd69f20cf5d7654d8a87c7 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
codope commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1561326209 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -528,6 +536,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
yihua commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1561319630 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -528,6 +536,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
yihua commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050092146 > 1. is there any change to partitions in `hoodie.proerties`? Do we now write it as `field1:type,field2:type2` when using CustomKeyGenerator? There is no change to the table configs

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2049197186 ## CI report: * 50b27846bf118909f3fd69f20cf5d7654d8a87c7 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2049032647 ## CI report: * c376900f104a979535fe7b4b9bb7e9a2d236a2b9 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-11 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2049023818 ## CI report: * c376900f104a979535fe7b4b9bb7e9a2d236a2b9 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-10 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2048869318 ## CI report: * c376900f104a979535fe7b4b9bb7e9a2d236a2b9 Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-10 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2048834962 ## CI report: * 185d0fc1b26344563514603f9f5e600972feaaac Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-10 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2048829426 ## CI report: * 185d0fc1b26344563514603f9f5e600972feaaac Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-04-04 Thread via GitHub
codope commented on code in PR #10615: URL: https://github.com/apache/hudi/pull/10615#discussion_r1551075779 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -528,6 +536,40 @@ object ProvidesHoodieConfig {

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-03-28 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2024589288 ## CI report: * 185d0fc1b26344563514603f9f5e600972feaaac Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-03-28 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2024516329 ## CI report: * afc107a681bb6df8e1b856239a811ccac6b3b3db Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-03-28 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2024506879 ## CI report: * afc107a681bb6df8e1b856239a811ccac6b3b3db Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-02-03 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-1925553485 ## CI report: * afc107a681bb6df8e1b856239a811ccac6b3b3db Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-02-03 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-1925509035 ## CI report: * afc107a681bb6df8e1b856239a811ccac6b3b3db Azure:

Re: [PR] [HUDI-7378] Fix Spark SQL DML with custom key generator [hudi]

2024-02-03 Thread via GitHub
hudi-bot commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-1925507573 ## CI report: * afc107a681bb6df8e1b856239a811ccac6b3b3db UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run