[GitHub] [hudi] yihua commented on a diff in pull request #7528: [HUDI-5443] Fixing exception trying to read MOR table after `NestedSchemaPruning` rule has been applied

2023-01-20 Thread GitBox
yihua commented on code in PR #7528: URL: https://github.com/apache/hudi/pull/7528#discussion_r1082866912 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieCatalystExpressionUtils.scala: ## @@ -78,14 +80,52 @@ object HoodieCatalystExpressionUtils {

[GitHub] [hudi] hudi-bot commented on pull request #7719: [HUDI-5584] When the table to be synchronized already exists in hive,…

2023-01-20 Thread GitBox
hudi-bot commented on PR #7719: URL: https://github.com/apache/hudi/pull/7719#issuecomment-1398739070 ## CI report: * b29c08828fe9a7ab027c4de3c5055ac88413b97d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398738962 ## CI report: * 860b46bcff9d06d71c4cd453700bb4160ae6a61a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-20 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398738792 ## CI report: * 1b075e25aa5811f36e83e12bfba11a08bc929bf1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-20 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398738476 ## CI report: * 48c0f695a3b9aade6fc3439a8d53433019b95e89 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-20 Thread GitBox
hudi-bot commented on PR #7607: URL: https://github.com/apache/hudi/pull/7607#issuecomment-1398738311 ## CI report: * 32033e4a4ed91005a237aa88afa2c6adcb51169f UNKNOWN * 6e67e79228d1e4d165af0faf5905e216153a80e3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-20 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398737116 ## CI report: * 031dc62b21fc55546243a8fea450138ef94f3405 Azure:

[GitHub] [hudi] LinMingQiang opened a new pull request, #7719: [HUDI-5584] When the table to be synchronized already exists in hive,…

2023-01-20 Thread GitBox
LinMingQiang opened a new pull request, #7719: URL: https://github.com/apache/hudi/pull/7719 … need to update serde/table properties ### Change Logs HiveSyncTool#syncSchema _Describe context and summary for this change. Highlight if any code was copied._ ### Impact

[GitHub] [hudi] hudi-bot commented on pull request #7709: [HUDI-5582] Do not let users override internal metadata configs

2023-01-20 Thread GitBox
hudi-bot commented on PR #7709: URL: https://github.com/apache/hudi/pull/7709#issuecomment-1398731340 ## CI report: * 8048762c6d565c09289564d812c4bba77cc90d61 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-20 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398730409 ## CI report: * 48c0f695a3b9aade6fc3439a8d53433019b95e89 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-20 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398731123 ## CI report: * 1b075e25aa5811f36e83e12bfba11a08bc929bf1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-20 Thread GitBox
hudi-bot commented on PR #7607: URL: https://github.com/apache/hudi/pull/7607#issuecomment-1398730183 ## CI report: * 32033e4a4ed91005a237aa88afa2c6adcb51169f UNKNOWN * 6e67e79228d1e4d165af0faf5905e216153a80e3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-20 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398728655 ## CI report: * 031dc62b21fc55546243a8fea450138ef94f3405 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398722329 ## CI report: * 860b46bcff9d06d71c4cd453700bb4160ae6a61a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7709: [HUDI-5582] Do not let users override internal metadata configs

2023-01-20 Thread GitBox
hudi-bot commented on PR #7709: URL: https://github.com/apache/hudi/pull/7709#issuecomment-1398722262 ## CI report: * 8048762c6d565c09289564d812c4bba77cc90d61 Azure:

[GitHub] [hudi] alexeykudinkin merged pull request #7423: [HUDI-5384] Adding optimization rule to appropriately push down filters into the `HoodieFileIndex`

2023-01-20 Thread GitBox
alexeykudinkin merged PR #7423: URL: https://github.com/apache/hudi/pull/7423 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] alexeykudinkin commented on pull request #7423: [HUDI-5384] Adding optimization rule to appropriately push down filters into the `HoodieFileIndex`

2023-01-20 Thread GitBox
alexeykudinkin commented on PR #7423: URL: https://github.com/apache/hudi/pull/7423#issuecomment-1398631718 CI is green: https://user-images.githubusercontent.com/428277/213751799-9d943b81-62a6-482c-94eb-c558a2fbb736.png;>

[GitHub] [hudi] soumilshah1995 commented on issue #2544: [SUPPORT]failed to read timestamp column in version 0.7.0 even when HIVE_SUPPORT_TIMESTAMP is enabled

2023-01-20 Thread GitBox
soumilshah1995 commented on issue #2544: URL: https://github.com/apache/hudi/issues/2544#issuecomment-1398626568 > find the commit for this fix? ru using latest version of hudi ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398619566 ## CI report: * 45e4b8e0ed22052683a7982c48444f7fc43b767d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7718: [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider …

2023-01-20 Thread GitBox
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1398609592 ## CI report: * 0da7f5ff6c70c773c035aae60d84ec460252aae6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398609328 ## CI report: * 45e4b8e0ed22052683a7982c48444f7fc43b767d Azure:

[GitHub] [hudi] LinMingQiang commented on pull request #7718: [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider …

2023-01-20 Thread GitBox
LinMingQiang commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1398604505 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-20 Thread GitBox
hudi-bot commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1398597214 ## CI report: * 860cf0ff1a05c3e79ccc57c48efa44894a916c4f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398596964 ## CI report: * 45e4b8e0ed22052683a7982c48444f7fc43b767d Azure:

[GitHub] [hudi] SteNicholas commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-20 Thread GitBox
SteNicholas commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1398542077 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398497794 ## CI report: * 45e4b8e0ed22052683a7982c48444f7fc43b767d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7712: [DNM] Check CI timeout

2023-01-20 Thread GitBox
hudi-bot commented on PR #7712: URL: https://github.com/apache/hudi/pull/7712#issuecomment-1398487649 ## CI report: * 45e4b8e0ed22052683a7982c48444f7fc43b767d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7718: [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider …

2023-01-20 Thread GitBox
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1398346998 ## CI report: * 0da7f5ff6c70c773c035aae60d84ec460252aae6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-20 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398346271 ## CI report: * 48c0f695a3b9aade6fc3439a8d53433019b95e89 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-20 Thread GitBox
hudi-bot commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1398267238 ## CI report: * 860cf0ff1a05c3e79ccc57c48efa44894a916c4f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-20 Thread GitBox
hudi-bot commented on PR #7607: URL: https://github.com/apache/hudi/pull/7607#issuecomment-1398266742 ## CI report: * 32033e4a4ed91005a237aa88afa2c6adcb51169f UNKNOWN * 6e67e79228d1e4d165af0faf5905e216153a80e3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-20 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398259582 ## CI report: * 1b075e25aa5811f36e83e12bfba11a08bc929bf1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7718: [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider …

2023-01-20 Thread GitBox
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1398187113 ## CI report: * 0da7f5ff6c70c773c035aae60d84ec460252aae6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-20 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398184942 ## CI report: * 031dc62b21fc55546243a8fea450138ef94f3405 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7718: [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider …

2023-01-20 Thread GitBox
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1398178136 ## CI report: * 0da7f5ff6c70c773c035aae60d84ec460252aae6 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7703: [HUDI-1575][DO NOT MERGE] Testing early conflict detection with feature flag enabled by default

2023-01-20 Thread GitBox
hudi-bot commented on PR #7703: URL: https://github.com/apache/hudi/pull/7703#issuecomment-1398177925 ## CI report: * 0fe1eddd4034e3861ff2519dc21d7a008b10d74d UNKNOWN * a1fdd1603b1fb1f59ce04990601c2e99f00cc9af Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7512: [HUDI-5417] support to read avro from non-legacy map/list in parquet log

2023-01-20 Thread GitBox
hudi-bot commented on PR #7512: URL: https://github.com/apache/hudi/pull/7512#issuecomment-1398177022 ## CI report: * 49fab36027c88f5235ce360a374e98a3b8f1a1d2 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-20 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398175391 ## CI report: * 97af2458373c47dff52bc8e2a8cd63099461ff67 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2023-01-20 Thread GitBox
hudi-bot commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1398174461 ## CI report: * dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN * 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN * 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #7713: [HUDI-5589] Fix Hudi config inference

2023-01-20 Thread GitBox
hudi-bot commented on PR #7713: URL: https://github.com/apache/hudi/pull/7713#issuecomment-1398164784 ## CI report: * 4ef51f2e03159eda252b15f90993069d257923f6 UNKNOWN * a55175d7d5bf775ed16ccac6d859ce6619c6 Azure:

[GitHub] [hudi] LinMingQiang opened a new pull request, #7718: [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider …

2023-01-20 Thread GitBox
LinMingQiang opened a new pull request, #7718: URL: https://github.com/apache/hudi/pull/7718 …parameter HIVE_SYNC_TABLE_STRATEGY ### Change Logs HoodieSparkSqlWriter#getHiveTableNames _Describe context and summary for this change. Highlight if any code was copied._ ###

[GitHub] [hudi] wzx140 commented on a diff in pull request #7714: [HUDI-5549] SparkRecordManager support avro data block

2023-01-20 Thread GitBox
wzx140 commented on code in PR #7714: URL: https://github.com/apache/hudi/pull/7714#discussion_r1082263118 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java: ## @@ -192,16 +198,25 @@ public boolean hasNext() { } @Override -

[GitHub] [hudi] hudi-bot commented on pull request #7714: [HUDI-5549] SparkRecordManager support avro data block

2023-01-20 Thread GitBox
hudi-bot commented on PR #7714: URL: https://github.com/apache/hudi/pull/7714#issuecomment-1398082322 ## CI report: * 462a4041585cfeb6c16354bbc8b964bdd08ed301 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-20 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398082191 ## CI report: * 1b075e25aa5811f36e83e12bfba11a08bc929bf1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-20 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398081852 ## CI report: * 48c0f695a3b9aade6fc3439a8d53433019b95e89 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-20 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398080581 ## CI report: * 97af2458373c47dff52bc8e2a8cd63099461ff67 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-20 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398072303 ## CI report: * 1b075e25aa5811f36e83e12bfba11a08bc929bf1 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-20 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398071952 ## CI report: * 48c0f695a3b9aade6fc3439a8d53433019b95e89 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7423: [HUDI-5384] Adding optimization rule to appropriately push down filters into the `HoodieFileIndex`

2023-01-20 Thread GitBox
hudi-bot commented on PR #7423: URL: https://github.com/apache/hudi/pull/7423#issuecomment-1398071383 ## CI report: * 78a6da0b0d5d65f8e7f4c59b495a2820e1f9877f UNKNOWN * 296dadf9e961375e4a81d35f87fef55ce8a1d860 UNKNOWN * 7c0c8a22940ae822b51b0848d97dea6fafd5216f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7232: [HUDI-5235] clustering target size should larger than small file limit

2023-01-20 Thread GitBox
hudi-bot commented on PR #7232: URL: https://github.com/apache/hudi/pull/7232#issuecomment-1398071029 ## CI report: * 08239e5b8d4d49da4b5b3d814233251f81b3d0b0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-20 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398070394 ## CI report: * 97af2458373c47dff52bc8e2a8cd63099461ff67 Azure:

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-19 Thread GitBox
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1082189809 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List

[GitHub] [hudi] SteNicholas commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-19 Thread GitBox
SteNicholas commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1398017208 @danny0405, this pull request explicitly declares the `serialVersionUID` for hudi-flink module. PTAL. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] hudi-bot commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-19 Thread GitBox
hudi-bot commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1398012661 ## CI report: * 860cf0ff1a05c3e79ccc57c48efa44894a916c4f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-19 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398012526 ## CI report: * 1b075e25aa5811f36e83e12bfba11a08bc929bf1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-19 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398012310 ## CI report: * 48c0f695a3b9aade6fc3439a8d53433019b95e89 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-19 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398011318 ## CI report: * 97af2458373c47dff52bc8e2a8cd63099461ff67 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] abhishekshenoy opened a new issue, #7717: [SUPPORT] org.apache.avro.SchemaParseException: Can't redefine: array When there are Top level variables , Struct and Array[struct] (no compl

2023-01-19 Thread GitBox
abhishekshenoy opened a new issue, #7717: URL: https://github.com/apache/hudi/issues/7717 ### Describe the problem you faced When storing a data structure with the following layout into a copy-on-write table: ``` root |-- personDetails: struct (nullable = true) ||--

[GitHub] [hudi] alexeykudinkin commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-19 Thread GitBox
alexeykudinkin commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398007962 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] alexeykudinkin commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-19 Thread GitBox
alexeykudinkin commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398007837 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] alexeykudinkin commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-19 Thread GitBox
alexeykudinkin commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398007610 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-19 Thread GitBox
hudi-bot commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1398007450 ## CI report: * 860cf0ff1a05c3e79ccc57c48efa44894a916c4f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-19 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398007308 ## CI report: * 384a9774018272e13b967817b0e48b1596a23dcc UNKNOWN * ae1bcf3c42da3945c843864cdeac7f8cb89ef088 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-19 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398007081 ## CI report: * b11fa6b2246e4f02f1da12487093a9b5bfaf2149 UNKNOWN * a37aa62fd9ea8d3b70a06e181237df23097d90a4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2023-01-19 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1398006500 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-19 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1398006155 ## CI report: * 13fb78850890b96b86b66d7df060feb11950ec0c UNKNOWN * 031dc62b21fc55546243a8fea450138ef94f3405 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7711: [HUDI-5569] Files written by first commit/delta commit if it failed are detected as valid data files

2023-01-19 Thread GitBox
hudi-bot commented on PR #7711: URL: https://github.com/apache/hudi/pull/7711#issuecomment-1398001739 ## CI report: * fb1f2609baf5b3f12a4ca5243f5205f2ba8f6367 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-19 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1398001638 ## CI report: * 384a9774018272e13b967817b0e48b1596a23dcc UNKNOWN * ae1bcf3c42da3945c843864cdeac7f8cb89ef088 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-19 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1398001285 ## CI report: * b11fa6b2246e4f02f1da12487093a9b5bfaf2149 UNKNOWN * a37aa62fd9ea8d3b70a06e181237df23097d90a4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-19 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-139731 ## CI report: * 13fb78850890b96b86b66d7df060feb11950ec0c UNKNOWN * 031dc62b21fc55546243a8fea450138ef94f3405 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6384: [HUDI-4613] Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

2023-01-19 Thread GitBox
hudi-bot commented on PR #6384: URL: https://github.com/apache/hudi/pull/6384#issuecomment-1397999485 ## CI report: * 5c72287193c51530504e19b69e81f877bd03c675 Azure:

[GitHub] [hudi] TengHuo commented on pull request #7626: [HUDI-5516] Reduce memory footprint on workload with thousand active partitions

2023-01-19 Thread GitBox
TengHuo commented on PR #7626: URL: https://github.com/apache/hudi/pull/7626#issuecomment-1397989682 > @TengHuo I tried the following workload with MOR table, 2000 partitions and compaction (checkpoint here triggers compaction) Got it, thanks so much @trushev -- This is an

[GitHub] [hudi] nsivabalan commented on issue #7628: [SUPPORT] Hudi Metadata Column Stats Fail

2023-01-19 Thread GitBox
nsivabalan commented on issue #7628: URL: https://github.com/apache/hudi/issues/7628#issuecomment-1397984406 yes, integer might have problems if you use it as record keys. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] fengjian428 commented on issue #7654: [SUPPORT] Starvation on Hudi Java Client in OCC mode

2023-01-19 Thread GitBox
fengjian428 commented on issue #7654: URL: https://github.com/apache/hudi/issues/7654#issuecomment-1397981689 > @fengjian428 , sure.. Please feel free to include the test code. Thanks for the quick fix.. I tried it out a snapshot built from your branch clean_deadlock, did not come across

[GitHub] [hudi] SteNicholas opened a new pull request, #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-19 Thread GitBox
SteNicholas opened a new pull request, #7716: URL: https://github.com/apache/hudi/pull/7716 ### Change Logs `Serializable` interface implementation don't explicitly declare `serialVersionUID`, which causes the `InvalidClassException` for the deserialization. `Serializable` interface

[GitHub] [hudi] danny0405 commented on a diff in pull request #7633: Fix Deletes issued without any prior commits

2023-01-19 Thread GitBox
danny0405 commented on code in PR #7633: URL: https://github.com/apache/hudi/pull/7633#discussion_r1082144102 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -1637,8 +1637,6 @@ protected void

[GitHub] [hudi] koochiswathiTR commented on issue #7708: Parquet files are in small size

2023-01-19 Thread GitBox
koochiswathiTR commented on issue #7708: URL: https://github.com/apache/hudi/issues/7708#issuecomment-1397968358 I we use clustering will it slow down ingestion? @danny0405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] koochiswathiTR commented on issue #7708: Parquet files are in small size

2023-01-19 Thread GitBox
koochiswathiTR commented on issue #7708: URL: https://github.com/apache/hudi/issues/7708#issuecomment-1397968093 We use inline, we dont use clustering @danny0405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] danny0405 commented on issue #7708: Parquet files are in small size

2023-01-19 Thread GitBox
danny0405 commented on issue #7708: URL: https://github.com/apache/hudi/issues/7708#issuecomment-1397967114 Did you try the inline or async clustering. the clustering service constantly merges small files into large ones. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] danny0405 commented on a diff in pull request #7710: [Doc] [minor] remove duplicated match clause in MergeInto syntax

2023-01-19 Thread GitBox
danny0405 commented on code in PR #7710: URL: https://github.com/apache/hudi/pull/7710#discussion_r1082141639 ## website/versioned_docs/version-0.12.2/quick-start-guide.md: ## @@ -761,7 +761,6 @@ MERGE INTO tableIdentifier AS target_alias USING (sub_query | tableIdentifier) AS

[GitHub] [hudi] danny0405 commented on pull request #7706: [HUDI-5585][flink]Fix flink creates and writes the table, the spark alter table reports an error

2023-01-19 Thread GitBox
danny0405 commented on PR #7706: URL: https://github.com/apache/hudi/pull/7706#issuecomment-1397964369 Thanks for the fix @waywtdcc , can we describe in high level what we are fixing here? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] hudi-bot commented on pull request #7703: [HUDI-1575][DO NOT MERGE] Testing early conflict detection with feature flag enabled by default

2023-01-19 Thread GitBox
hudi-bot commented on PR #7703: URL: https://github.com/apache/hudi/pull/7703#issuecomment-1397954902 ## CI report: * 0fe1eddd4034e3861ff2519dc21d7a008b10d74d UNKNOWN * e7ea55af65e8b0af7024268b652db37561eb501f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-19 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1397954859 ## CI report: * 384a9774018272e13b967817b0e48b1596a23dcc UNKNOWN * 45b15748e01531fb144e37f5b04b34b811ab1474 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-19 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1397954692 ## CI report: * b11fa6b2246e4f02f1da12487093a9b5bfaf2149 UNKNOWN * a4afea9412a655aee083524e56b6d75e56720bc4 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-19 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1397953949 ## CI report: * 13fb78850890b96b86b66d7df060feb11950ec0c UNKNOWN * 29de073c80985fa18576e7a01ca47b61d32ac944 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2023-01-19 Thread GitBox
hudi-bot commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1397953606 ## CI report: * dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN * 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN * 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN *

[GitHub] [hudi] danny0405 commented on pull request #7687: Update to handle deletes in postgres debezium

2023-01-19 Thread GitBox
danny0405 commented on PR #7687: URL: https://github.com/apache/hudi/pull/7687#issuecomment-1397953163 > > Thanks @BalaMahesh can we fire a JIRA issue and change the commit title to: [HUDI-${JIRA_ID}] ${you commit title} > > @danny0405 - How do I get access to jira to create the

[GitHub] [hudi] danny0405 commented on issue #7715: [SUPPORT] HoodieDeltaStreamer gives an errror when reading from Redpanda Avro topics

2023-01-19 Thread GitBox
danny0405 commented on issue #7715: URL: https://github.com/apache/hudi/issues/7715#issuecomment-1397951254 Seems the Redpanda returns the null for earliest offset of each partition:

[GitHub] [hudi] nfarah86 commented on pull request #7687: Update to handle deletes in postgres debezium

2023-01-19 Thread GitBox
nfarah86 commented on PR #7687: URL: https://github.com/apache/hudi/pull/7687#issuecomment-1397951198 helping with jira -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #7703: [HUDI-1575][DO NOT MERGE] Testing early conflict detection with feature flag enabled by default

2023-01-19 Thread GitBox
hudi-bot commented on PR #7703: URL: https://github.com/apache/hudi/pull/7703#issuecomment-1397949951 ## CI report: * d5a19b738146a7003c7957b3a1cd63c5cfbd348d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7702: [HUDI-5579] Fixing Kryo registration to be properly wired into Spark sessions

2023-01-19 Thread GitBox
hudi-bot commented on PR #7702: URL: https://github.com/apache/hudi/pull/7702#issuecomment-1397949920 ## CI report: * 384a9774018272e13b967817b0e48b1596a23dcc UNKNOWN * 45b15748e01531fb144e37f5b04b34b811ab1474 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7694: [HUDI-5572] Flink write need to skip check the compatibility of Schem…

2023-01-19 Thread GitBox
hudi-bot commented on PR #7694: URL: https://github.com/apache/hudi/pull/7694#issuecomment-1397949871 ## CI report: * 97fdc558722b8d5152f9e21112045adb73eca9fe Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7642: [HUDI-5534][Stacked on 6815] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table

2023-01-19 Thread GitBox
hudi-bot commented on PR #7642: URL: https://github.com/apache/hudi/pull/7642#issuecomment-1397949726 ## CI report: * dd82fb612d88d6c9ba4f06be2989ec5061052047 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6815: [HUDI-4937][Stacked on 7702] Fix `HoodieTable` injecting non-reusable `HoodieBackedTableMetadata` aggressively flushing MT readers

2023-01-19 Thread GitBox
hudi-bot commented on PR #6815: URL: https://github.com/apache/hudi/pull/6815#issuecomment-1397949027 ## CI report: * 13fb78850890b96b86b66d7df060feb11950ec0c UNKNOWN * 29de073c80985fa18576e7a01ca47b61d32ac944 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2023-01-19 Thread GitBox
hudi-bot commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1397948658 ## CI report: * dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN * 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN * 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #7661: [DO NOT MERGE] Release testing record merger

2023-01-19 Thread GitBox
hudi-bot commented on PR #7661: URL: https://github.com/apache/hudi/pull/7661#issuecomment-1397945732 ## CI report: * f698f26db2314cbbbee30d37df0d6fd343317796 UNKNOWN * 4a2dbb50cff97211589a22059ac7fb1ffcf605a8 UNKNOWN * 9dc9aed49c0797ba20dd716bab973ed6cfc803a4 Azure:

[GitHub] [hudi] afuyo opened a new issue, #7715: [SUPPORT] HoodieDeltaStreamer gives an errror when reading from Redpanda Avro topics

2023-01-19 Thread GitBox
afuyo opened a new issue, #7715: URL: https://github.com/apache/hudi/issues/7715 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? Yes **Describe the problem you faced** I have replaced Apache Kafka with

[GitHub] [hudi] hudi-bot commented on pull request #7713: [HUDI-5589] Fix Hudi config inference

2023-01-19 Thread GitBox
hudi-bot commented on PR #7713: URL: https://github.com/apache/hudi/pull/7713#issuecomment-1397911609 ## CI report: * 198a828b76b654e4b8f3ef8ac133f672a682cdf8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7703: [HUDI-1575][DO NOT MERGE] Testing early conflict detection with feature flag enabled by default

2023-01-19 Thread GitBox
hudi-bot commented on PR #7703: URL: https://github.com/apache/hudi/pull/7703#issuecomment-1397911575 ## CI report: * d5a19b738146a7003c7957b3a1cd63c5cfbd348d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7512: [HUDI-5417] support to read avro from non-legacy map/list in parquet log

2023-01-19 Thread GitBox
hudi-bot commented on PR #7512: URL: https://github.com/apache/hudi/pull/7512#issuecomment-1397911293 ## CI report: * b248f60720e34316217dadc67882ff2417cf6781 Azure:

  1   2   3   4   5   6   7   8   9   10   >