Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567888070 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1567730841 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -278,14 +286,26 @@ public void setConf(Object newConf) { @Override public List

[PR] Test: Add test to partition on field with a dot [iceberg-python]

2024-04-16 Thread via GitHub
Fokko opened a new pull request, #610: URL: https://github.com/apache/iceberg-python/pull/610 This is not allowed in Avro, just checking if it works. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Iceberg/Comet integration POC [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #9841: URL: https://github.com/apache/iceberg/pull/9841#discussion_r1568005309 ## spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/comet/CometIcebergColumnarBatchReader.java: ## @@ -0,0 +1,303 @@ +/* + * Licensed to the

Re: [I] CodePointCharStream not found while using the latest snapshot version [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2383: URL: https://github.com/apache/iceberg/issues/2383#issuecomment-2060100511 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Is Apache Iceberg table support spatial data? How to store spatial data in iceberg tables? [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2378: URL: https://github.com/apache/iceberg/issues/2378#issuecomment-2060100495 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Commit failed due to Internal error processing lock in metastore DB [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2387: Commit failed due to Internal error processing lock in metastore DB URL: https://github.com/apache/iceberg/issues/2387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Commit failed due to Internal error processing lock in metastore DB [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2387: URL: https://github.com/apache/iceberg/issues/2387#issuecomment-2060100523 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] CodePointCharStream not found while using the latest snapshot version [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2383: CodePointCharStream not found while using the latest snapshot version URL: https://github.com/apache/iceberg/issues/2383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Integrate Parquet bloomfilter feature [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2391: URL: https://github.com/apache/iceberg/issues/2391#issuecomment-2060100541 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Integrate Parquet bloomfilter feature [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2391: Integrate Parquet bloomfilter feature URL: https://github.com/apache/iceberg/issues/2391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Flink cdc events with update or delete doesn't work in 0.11.0 branch [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2409: URL: https://github.com/apache/iceberg/issues/2409#issuecomment-2060100555 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Flink cdc events with update or delete doesn't work in 0.11.0 branch [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2409: Flink cdc events with update or delete doesn't work in 0.11.0 branch URL: https://github.com/apache/iceberg/issues/2409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Incorrect Metrics Calculation for Iceberg Table Due to Column Name Transformation with Special Characters [iceberg]

2024-04-16 Thread via GitHub
szehon-ho commented on issue #10115: URL: https://github.com/apache/iceberg/issues/10115#issuecomment-2059772658 I see, is it because of the dot character? I havent looked deeply at the problem but both solutions make sense, I think the first one may be preferable because then there is

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1567967380 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -1796,6 +1799,48 @@ public void

Re: [PR] Add Refs metadata table [iceberg-python]

2024-04-16 Thread via GitHub
Fokko commented on PR #602: URL: https://github.com/apache/iceberg-python/pull/602#issuecomment-2059988471 @geruh Thanks for creating this PR! Can you resolve the conflicts so we can get this in? Thanks! -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Add Refs metadata table [iceberg-python]

2024-04-16 Thread via GitHub
Fokko commented on code in PR #602: URL: https://github.com/apache/iceberg-python/pull/602#discussion_r1567985291 ## pyiceberg/table/__init__.py: ## @@ -3410,6 +3410,32 @@ def _readable_metrics_struct(bound_type: PrimitiveType) -> pa.StructType:

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567892975 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
igoradulian commented on PR #9793: URL: https://github.com/apache/iceberg/pull/9793#issuecomment-2059879886 @nastra please review the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1567967116 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -1796,6 +1799,48 @@ public void

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1567966520 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkSessionCatalog.java: ## @@ -98,4 +103,20 @@ public void testLoadFunction() { // TODO:

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1568016911 ## data/src/main/java/org/apache/iceberg/data/MigrationService.java: ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1568016911 ## data/src/main/java/org/apache/iceberg/data/MigrationService.java: ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
wForget commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1568109837 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkSessionCatalog.java: ## @@ -98,4 +103,20 @@ public void testLoadFunction() { // TODO: fix

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1567855351 ## core/src/test/java/org/apache/iceberg/rest/TestRESTViewCatalog.java: ## @@ -144,6 +151,36 @@ public void closeCatalog() throws Exception { } } + @Test +

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1567854386 ## core/src/test/java/org/apache/iceberg/rest/responses/TestListTablesResponse.java: ## @@ -105,9 +105,32 @@ public void testBuilderDoesNotCreateInvalidObjects() {

Re: [I] Rotation of encryption keys [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2373: URL: https://github.com/apache/iceberg/issues/2373#issuecomment-2060100420 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Rotation of encryption keys [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2373: Rotation of encryption keys URL: https://github.com/apache/iceberg/issues/2373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] SparkSessionCatalog Drop Issues [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2374: SparkSessionCatalog Drop Issues URL: https://github.com/apache/iceberg/issues/2374 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] SparkSessionCatalog Drop Issues [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2374: URL: https://github.com/apache/iceberg/issues/2374#issuecomment-2060100437 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] How to update number of buckets in partition spec ? [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2375: How to update number of buckets in partition spec ? URL: https://github.com/apache/iceberg/issues/2375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] How to update number of buckets in partition spec ? [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2375: URL: https://github.com/apache/iceberg/issues/2375#issuecomment-2060100456 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] How to use credential in Apache iceberg? [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2377: How to use credential in Apache iceberg? URL: https://github.com/apache/iceberg/issues/2377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] How to use credential in Apache iceberg? [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] commented on issue #2377: URL: https://github.com/apache/iceberg/issues/2377#issuecomment-2060100470 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Is Apache Iceberg table support spatial data? How to store spatial data in iceberg tables? [iceberg]

2024-04-16 Thread via GitHub
github-actions[bot] closed issue #2378: Is Apache Iceberg table support spatial data? How to store spatial data in iceberg tables? URL: https://github.com/apache/iceberg/issues/2378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567888070 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567888070 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567888070 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1568016911 ## data/src/main/java/org/apache/iceberg/data/MigrationService.java: ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [I] Iceberg Spark Extensions conflict with Paimon [iceberg]

2024-04-16 Thread via GitHub
wForget commented on issue #10143: URL: https://github.com/apache/iceberg/issues/10143#issuecomment-2060201859 > Never mind. I think what you mean it can be fixed either by Iceberg or by paimon? Yes, I'm not sure if my idea is acceptable, so I submitted issues on both sides hoping

Re: [I] Metadata table queries fail if a partition column was reused in V2 [iceberg]

2024-04-16 Thread via GitHub
d4r3topk commented on issue #4661: URL: https://github.com/apache/iceberg/issues/4661#issuecomment-2059838101 This issue has not been resolved yet for V2 API. I have a table where one of the columns is "region" and "region" is also part of the partition spec. I run a

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567891507 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

Re: [I] Incorrect Metrics Calculation for Iceberg Table Due to Column Name Transformation with Special Characters [iceberg]

2024-04-16 Thread via GitHub
Fokko commented on issue #10115: URL: https://github.com/apache/iceberg/issues/10115#issuecomment-2059779295 Related: https://github.com/apache/iceberg/issues/10120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] [feature request] Allow Java Iceberg library to write parquet files with special character column names [iceberg]

2024-04-16 Thread via GitHub
Fokko commented on issue #10120: URL: https://github.com/apache/iceberg/issues/10120#issuecomment-2059777982 I can confirm this, when creating a table: ```sql CREATE TABLE default.abc(`a.b+c` string); %%sql INSERT INTO default.abc VALUES ('a'), ('b') ``` The field name is

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1567853170 ## core/src/test/java/org/apache/iceberg/rest/responses/TestListNamespacesResponse.java: ## @@ -83,9 +83,32 @@ public void testBuilderDoesNotCreateInvalidObjects() {

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
javrasya commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1567888070 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software

Re: [PR] Hive: turn off the stats gathering when iceberg.hive.keep.stats is false [iceberg]

2024-04-16 Thread via GitHub
stargrey102 commented on PR #10148: URL: https://github.com/apache/iceberg/pull/10148#issuecomment-2059891565 Hi @deniskuzZ: for more info: we identified the autogather executed (controlled by hive.stats.autogather) to collect stats from Hive when committing a new Iceberg table to Hive

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1567963497 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkSessionCatalog.java: ## @@ -275,18 +275,20 @@ public Table alterTable(Identifier ident,

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
aokolnychyi commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1567983615 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkSessionCatalog.java: ## @@ -98,4 +103,20 @@ public void testLoadFunction() { // TODO:

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
stevenzwu commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1568040503 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Don't fail to serialize IcebergSourceSplit when there is too many delete files [iceberg]

2024-04-16 Thread via GitHub
stevenzwu commented on code in PR #9464: URL: https://github.com/apache/iceberg/pull/9464#discussion_r1568039670 ## flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/split/SerializerHelper.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
rahil-c commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1568086253 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -1796,6 +1799,48 @@ public void

Re: [PR] Sql catalog [iceberg-rust]

2024-04-16 Thread via GitHub
JanKaul commented on PR #229: URL: https://github.com/apache/iceberg-rust/pull/229#issuecomment-2060377330 I have to add a couple of more changes. I'll notify you when I'm finished. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Drop table purge issue for parquet tables with SparkSessionCatalog [iceberg]

2024-04-16 Thread via GitHub
wForget commented on issue #10157: URL: https://github.com/apache/iceberg/issues/10157#issuecomment-2060344320 FYI #10128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
wForget commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1568220924 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkSessionCatalog.java: ## @@ -98,4 +103,20 @@ public void testLoadFunction() { // TODO: fix

Re: [PR] Updates flink versioning information in our docs [iceberg]

2024-04-16 Thread via GitHub
pvary merged PR #10155: URL: https://github.com/apache/iceberg/pull/10155 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Test: Add test to partition on field with a dot [iceberg-python]

2024-04-16 Thread via GitHub
HonahX commented on code in PR #610: URL: https://github.com/apache/iceberg-python/pull/610#discussion_r1568179853 ## tests/integration/test_reads.py: ## @@ -473,6 +474,31 @@ def test_sanitize_character(catalog: Catalog) -> None: assert arrow_table.schema.names[0] ==

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568265618 ## data/src/test/java/org/apache/iceberg/data/TestDataFileIndexStatsFilters.java: ## @@ -94,24 +93,23 @@ public void createTableAndData() throws IOException {

Re: [I] Add metadata tables [iceberg-python]

2024-04-16 Thread via GitHub
rahil-c commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2060301685 Hey @Fokko I can try working on `Manifests` table if no one is assigned it? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
wForget commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1568234803 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkSessionCatalog.java: ## @@ -275,18 +275,20 @@ public Table alterTable(Identifier ident,

Re: [I] rewriting manifest can rewrite based on filter? [iceberg]

2024-04-16 Thread via GitHub
chenwyi2 commented on issue #10129: URL: https://github.com/apache/iceberg/issues/10129#issuecomment-2060385444 https://github.com/apache/iceberg/pull/9447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] rewriting manifest can rewrite based on filter? [iceberg]

2024-04-16 Thread via GitHub
chenwyi2 closed issue #10129: rewriting manifest can rewrite based on filter? URL: https://github.com/apache/iceberg/issues/10129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568263803 ## data/src/test/java/org/apache/iceberg/data/DeleteReadTests.java: ## @@ -45,10 +38,12 @@ import org.apache.iceberg.util.Pair; import

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568264243 ## data/src/test/java/org/apache/iceberg/data/TestDataFileIndexStatsFilters.java: ## @@ -58,7 +57,7 @@ public class TestDataFileIndexStatsFilters {

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568262204 ## data/src/test/java/org/apache/iceberg/data/DataTestHelpers.java: ## @@ -84,16 +86,15 @@ private static void assertEquals(Type type, Object expected, Object actual)

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568264984 ## data/src/test/java/org/apache/iceberg/data/TestDataFileIndexStatsFilters.java: ## @@ -68,9 +67,9 @@ public class TestDataFileIndexStatsFilters { private DataFile

[PR] Docs: Remove link to Flink unit test [iceberg]

2024-04-16 Thread via GitHub
manuzhang opened a new pull request, #10160: URL: https://github.com/apache/iceberg/pull/10160 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Spark 3.5: Check table existence to determine which catalog for drop table [iceberg]

2024-04-16 Thread via GitHub
wForget commented on code in PR #10128: URL: https://github.com/apache/iceberg/pull/10128#discussion_r1568227470 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestSparkSessionCatalog.java: ## @@ -98,4 +103,20 @@ public void testLoadFunction() { // TODO: fix

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568261117 ## data/src/test/java/org/apache/iceberg/data/DataTest.java: ## @@ -21,6 +21,7 @@ import static org.apache.iceberg.types.Types.NestedField.optional; import static

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568260531 ## data/src/test/java/org/apache/iceberg/data/TestGenericReaderDeletes.java: ## @@ -18,28 +18,23 @@ */ package org.apache.iceberg.data; -import static

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568266969 ## data/src/test/java/org/apache/iceberg/data/TestDataFileIndexStatsFilters.java: ## @@ -137,9 +135,11 @@ public void testPositionDeletePlanningPath() throws

Re: [PR] Updates flink versioning information in our docs [iceberg]

2024-04-16 Thread via GitHub
manuzhang commented on code in PR #10155: URL: https://github.com/apache/iceberg/pull/10155#discussion_r1568215330 ## docs/docs/flink-writes.md: ## @@ -103,7 +103,7 @@ FlinkSink.forRowData(input) env.execute("Test Iceberg DataStream"); ``` -The iceberg API also allows users

Re: [PR] Updates flink versioning information in our docs [iceberg]

2024-04-16 Thread via GitHub
pvary commented on PR #10155: URL: https://github.com/apache/iceberg/pull/10155#issuecomment-2060366264 Thanks for the PR @rodmeneses and @Fokko and @manuzhang for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Docs: Remove link to Flink unit test [iceberg]

2024-04-16 Thread via GitHub
pvary commented on PR #10160: URL: https://github.com/apache/iceberg/pull/10160#issuecomment-2060367028 @manuzhang: Could you please rebase? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568263144 ## data/src/test/java/org/apache/iceberg/data/DataTestHelpers.java: ## @@ -84,16 +86,15 @@ private static void assertEquals(Type type, Object expected, Object actual)

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568261644 ## data/src/test/java/org/apache/iceberg/data/DataTestHelpers.java: ## @@ -43,7 +42,9 @@ public static void assertEquals(Types.StructType struct, Record expected,

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568260729 ## data/src/test/java/org/apache/iceberg/data/TestGenericRecord.java: ## @@ -35,7 +34,7 @@ public void testGetNullValue() { GenericRecord record =

Re: [PR] #9073 Junit 4 tests switched to JUnit 5 [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9793: URL: https://github.com/apache/iceberg/pull/9793#discussion_r1568261957 ## data/src/test/java/org/apache/iceberg/data/DataTestHelpers.java: ## @@ -56,7 +57,8 @@ public static void assertEquals(Types.ListType list, List expected, List a

Re: [PR] Sql catalog [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 commented on PR #229: URL: https://github.com/apache/iceberg-rust/pull/229#issuecomment-2059212759 cc @JanKaul Is this pr ready for review or you need to do more updates? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Flink: Adds support for Flink 1.19 version [iceberg]

2024-04-16 Thread via GitHub
pvary merged PR #10112: URL: https://github.com/apache/iceberg/pull/10112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Discussion: Next steps / requirements to support `append` files [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 commented on issue #329: URL: https://github.com/apache/iceberg-rust/issues/329#issuecomment-2059202270 I think to implement appending data file, there are two main tasks: 1. Implement transaction api to append data file 2. Implement file writer to write record

Re: [PR] feat: Convert predicate to arrow filter and push down to parquet reader [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 commented on code in PR #295: URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1567468384 ## crates/iceberg/src/arrow/reader.rs: ## @@ -186,4 +216,399 @@ impl ArrowReader { Ok(ProjectionMask::leaves(parquet_schema, indices)) }

Re: [PR] Flink: Adds support for Flink 1.19 version [iceberg]

2024-04-16 Thread via GitHub
rodmeneses commented on PR #10112: URL: https://github.com/apache/iceberg/pull/10112#issuecomment-2059306340 > > @nastra: Any comments? I would like to merge this soon, as any merge to Flink code path will make this PR stale, and @rodmeneses needs to recreate the whole PR. > > Thanks,

Re: [PR] Core: Fix JDBC Catalog table commit when migrating from schema V0 to V1 (#101111) [iceberg]

2024-04-16 Thread via GitHub
nastra merged PR #10152: URL: https://github.com/apache/iceberg/pull/10152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Flink: Adds support for Flink 1.19 version [iceberg]

2024-04-16 Thread via GitHub
pvary commented on PR #10112: URL: https://github.com/apache/iceberg/pull/10112#issuecomment-2059345426 > Why do we remove Flink 1.1.6 in this PR? @manuzhang: This is how we usually do these changes. We support the 3 last version of Flink, so when we add a new version, we remove the

Re: [PR] Tests: Unify the test catalog setting [iceberg-python]

2024-04-16 Thread via GitHub
HonahX merged PR #609: URL: https://github.com/apache/iceberg-python/pull/609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Flink: Adds support for Flink 1.19 version [iceberg]

2024-04-16 Thread via GitHub
pvary commented on PR #10112: URL: https://github.com/apache/iceberg/pull/10112#issuecomment-2059391305 Merged to main. Thanks for the PR @rodmeneses and @nastra for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] chore(deps): Bump apache/skywalking-eyes from 0.5.0 to 0.6.0 [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 commented on PR #328: URL: https://github.com/apache/iceberg-rust/pull/328#issuecomment-2058314718 Thanks @dependabot for working on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] chore(deps): Update volo-thrift requirement from 0.9.2 to 0.10.0 [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 closed pull request #326: chore(deps): Update volo-thrift requirement from 0.9.2 to 0.10.0 URL: https://github.com/apache/iceberg-rust/pull/326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Tests: Unify the test catalog setting [iceberg-python]

2024-04-16 Thread via GitHub
HonahX commented on code in PR #609: URL: https://github.com/apache/iceberg-python/pull/609#discussion_r1566772517 ## tests/conftest.py: ## @@ -2144,3 +2144,31 @@ def arrow_table_with_only_nulls(pa_schema: "pa.Schema") -> "pa.Table": import pyarrow as pa return

Re: [PR] chore(deps): Bump apache/skywalking-eyes from 0.5.0 to 0.6.0 [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 merged PR #328: URL: https://github.com/apache/iceberg-rust/pull/328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] chore(deps): Update volo-thrift requirement from 0.9.2 to 0.10.0 [iceberg-rust]

2024-04-16 Thread via GitHub
dependabot[bot] commented on PR #326: URL: https://github.com/apache/iceberg-rust/pull/326#issuecomment-2058313835 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version,

Re: [PR] Add Partitions Metadata Table [iceberg-python]

2024-04-16 Thread via GitHub
HonahX merged PR #603: URL: https://github.com/apache/iceberg-python/pull/603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] chore(deps): Update volo-thrift requirement from 0.9.2 to 0.10.0 [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 closed pull request #326: chore(deps): Update volo-thrift requirement from 0.9.2 to 0.10.0 URL: https://github.com/apache/iceberg-rust/pull/326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] chore(deps): Update volo-thrift requirement from 0.9.2 to 0.10.0 [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 commented on PR #326: URL: https://github.com/apache/iceberg-rust/pull/326#issuecomment-2058338661 I think we may need to upgrade this manually since it contains api change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-16 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1566871505 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,317 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [I] Create iceberg table from existsing parquet files with slightly different schemas (schemas merge is possible). [iceberg-python]

2024-04-16 Thread via GitHub
sergun commented on issue #601: URL: https://github.com/apache/iceberg-python/issues/601#issuecomment-2058456159 > One thing I wonder is if PyIceberg can handle schema evolution of nested structs. Looks like it can. From https://py.iceberg.apache.org/api/#add-column: ```

Re: [PR] Add Pagination To List Apis [iceberg]

2024-04-16 Thread via GitHub
nastra commented on code in PR #9782: URL: https://github.com/apache/iceberg/pull/9782#discussion_r1566939950 ## core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java: ## @@ -2329,6 +2374,57 @@ public void multipleDiffsAgainstMultipleTablesLastFails() {

Re: [PR] chore(deps): Update pilota requirement from 0.10.0 to 0.11.0 [iceberg-rust]

2024-04-16 Thread via GitHub
liurenjie1024 merged PR #327: URL: https://github.com/apache/iceberg-rust/pull/327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add Partitions Metadata Table [iceberg-python]

2024-04-16 Thread via GitHub
HonahX commented on PR #603: URL: https://github.com/apache/iceberg-python/pull/603#issuecomment-2058317984 Merged! Thanks @syun64 for working on this and thanks @Fokko for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   3   >