Re: [PR] Partitioned Append on Identity Transform [iceberg-python]

2024-04-05 Thread via GitHub
Fokko merged PR #555: URL: https://github.com/apache/iceberg-python/pull/555 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553215173 ## crates/iceberg/src/expr/predicate.rs: ## @@ -217,6 +249,10 @@ impl Display for SetExpression { /// Unbound predicate expression before binding to a

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553220669 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [I] Support creating tags [iceberg-python]

2024-04-05 Thread via GitHub
Fokko commented on issue #573: URL: https://github.com/apache/iceberg-python/issues/573#issuecomment-2039233700 @enkidulan Thanks for reaching out here. Are you interested in creating the API for this? :) -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1553285045 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] Support identifier warehouses [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 merged PR #308: URL: https://github.com/apache/iceberg-rust/pull/308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10088: URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553073213 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1318,8 +1324,16 @@ public void

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10088: URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553073692 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1341,10 +1355,17 @@ public void

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10088: URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553071451 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -900,7 +906,7 @@ public void dropV1View() { String v1View

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1553151792 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553253806 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553279624 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] feat: Project transform [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on PR #309: URL: https://github.com/apache/iceberg-rust/pull/309#issuecomment-2039348517 > Hi, @marvinlanhenke Thanks for pr, it looks great! I have some small suggestion to restructure the code to make it easier for review. Really greatful for these tests!

Re: [PR] feat: Project transform [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 merged PR #309: URL: https://github.com/apache/iceberg-rust/pull/309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Implement `project` for `Transform`. [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 closed issue #264: Implement `project` for `Transform`. URL: https://github.com/apache/iceberg-rust/issues/264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nastra commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553429104 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -970,6 +975,47 @@ public void testCatalogWithCustomMetricsReporter() throws IOException {

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nastra commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553435328 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveClientPool.java: ## @@ -116,6 +119,52 @@ public void testGetTablesFailsForNonReconnectableException()

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553297280 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553430668 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveClientPool.java: ## @@ -116,6 +119,52 @@ public void testGetTablesFailsForNonReconnectableException()

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10088: URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553067799 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -62,6 +64,9 @@ public class TestViews extends

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553098425 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,363 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] feat: Project transform [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #309: URL: https://github.com/apache/iceberg-rust/pull/309#discussion_r1553262971 ## crates/iceberg/src/spec/transform.rs: ## @@ -261,6 +269,323 @@ impl Transform { _ => self == other, } } + +/// Projects a

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nastra commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553049382 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -970,6 +976,51 @@ public void testCatalogWithCustomMetricsReporter() throws IOException {

Re: [PR] Spark: Hive-View Integration Tests [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10088: URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553075946 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java: ## @@ -1421,38 +1442,38 @@ public void showViews() throws

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nk1506 commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553083646 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -970,6 +976,51 @@ public void testCatalogWithCustomMetricsReporter() throws IOException {

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553082179 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,363 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nastra commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553236192 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -970,6 +976,51 @@ public void testCatalogWithCustomMetricsReporter() throws IOException {

Re: [PR] feat: Project transform [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #309: URL: https://github.com/apache/iceberg-rust/pull/309#discussion_r1553275061 ## crates/iceberg/src/spec/transform.rs: ## @@ -261,6 +269,323 @@ impl Transform { _ => self == other, } } + +/// Projects a

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553077742 ## crates/iceberg/src/expr/predicate.rs: ## @@ -217,6 +249,10 @@ impl Display for SetExpression { /// Unbound predicate expression before binding to a schema.

Re: [PR] feat: Project transform [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #309: URL: https://github.com/apache/iceberg-rust/pull/309#discussion_r1553272539 ## crates/iceberg/src/spec/transform.rs: ## @@ -261,6 +269,323 @@ impl Transform { _ => self == other, } } + +/// Projects a

Re: [PR] feat: Project transform [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #309: URL: https://github.com/apache/iceberg-rust/pull/309#discussion_r1553271703 ## crates/iceberg/src/spec/transform.rs: ## @@ -261,6 +269,323 @@ impl Transform { _ => self == other, } } + +/// Projects a

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1552969104 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,363 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nastra commented on code in PR #10082: URL: https://github.com/apache/iceberg/pull/10082#discussion_r1553049382 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -970,6 +976,51 @@ public void testCatalogWithCustomMetricsReporter() throws IOException {

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553297280 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553653540 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [I] In case of Positional Deletes "file_path" in lowerbound & upperbound do not have full file_path Buffer [iceberg]

2024-04-05 Thread via GitHub
singhpk234 commented on issue #10064: URL: https://github.com/apache/iceberg/issues/10064#issuecomment-2040031695 which iceberg version ? seems like the file_path metrics were truncated : > This is because these metrics were truncated, Iceberg's default metrics mode for

Re: [PR] Add option to delete datafiles [iceberg-python]

2024-04-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1553588970 ## pyiceberg/table/__init__.py: ## @@ -2726,6 +2731,112 @@ def _commit(self) -> UpdatesAndRequirements: ) +class DeleteFiles(_MergingSnapshotProducer):

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1553740208 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] Core: Add explicit JSON parser for ConfigResponse [iceberg]

2024-04-05 Thread via GitHub
amogh-jahagirdar commented on code in PR #9952: URL: https://github.com/apache/iceberg/pull/9952#discussion_r1553791769 ## core/src/main/java/org/apache/iceberg/rest/responses/ConfigResponseParser.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation

Re: [I] Add metadata tables [iceberg-python]

2024-04-05 Thread via GitHub
syun64 commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-2040023902 Hi @Fokko and folks, we are interested in implementing the Partitions metadata table for our use case if it hasn't been picked up already -- This is an automated message from

Re: [PR] Hive, JDBC: Add null check before matching the error message [iceberg]

2024-04-05 Thread via GitHub
nastra merged PR #10082: URL: https://github.com/apache/iceberg/pull/10082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add option to delete datafiles [iceberg-python]

2024-04-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1553580296 ## pyiceberg/table/__init__.py: ## @@ -2726,6 +2731,112 @@ def _commit(self) -> UpdatesAndRequirements: ) +class DeleteFiles(_MergingSnapshotProducer):

Re: [PR] Add option to delete datafiles [iceberg-python]

2024-04-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1553580296 ## pyiceberg/table/__init__.py: ## @@ -2726,6 +2731,112 @@ def _commit(self) -> UpdatesAndRequirements: ) +class DeleteFiles(_MergingSnapshotProducer):

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554176015 ## crates/iceberg/src/spec/schema.rs: ## @@ -137,9 +142,55 @@ impl SchemaBuilder { name_to_id, lowercase_name_to_id, id_to_name,

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554174651 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

[PR] Change DataScan to accept Metadata and io [iceberg-python]

2024-04-05 Thread via GitHub
Fokko opened a new pull request, #581: URL: https://github.com/apache/iceberg-python/pull/581 For the partial deletes I want to do a scan on in memory metadata. Changing this API allows this. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554161905 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] Support client-side purge in REST catalog [iceberg]

2024-04-05 Thread via GitHub
danielcweeks commented on issue #10089: URL: https://github.com/apache/iceberg/issues/10089#issuecomment-2040535320 @flyrain I'm a little confused, how can the REST Server not have access to the files? Currently the server needs access to at least the metadata files. Are you considering

Re: [I] zorder does not work with sub fields [iceberg]

2024-04-05 Thread via GitHub
cccs-jc commented on issue #10017: URL: https://github.com/apache/iceberg/issues/10017#issuecomment-2040368228 Seems like it would. I'm not a reviewer but I do want to the fix :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554163204 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] In case of Positional Deletes "file_path" in lowerbound & upperbound do not have full file_path Buffer [iceberg]

2024-04-05 Thread via GitHub
singhpk234 commented on issue #10064: URL: https://github.com/apache/iceberg/issues/10064#issuecomment-2040468114 are you using presto as a writer as well ? if you check it's still 16 characters so the writer which is generating the delete files is generating wrong metrics i.e still using

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554175122 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Core: Add explicit JSON parser for ConfigResponse [iceberg]

2024-04-05 Thread via GitHub
nastra commented on PR #9952: URL: https://github.com/apache/iceberg/pull/9952#issuecomment-2040195604 thanks for the review @amogh-jahagirdar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: Add explicit JSON parser for ConfigResponse [iceberg]

2024-04-05 Thread via GitHub
nastra merged PR #9952: URL: https://github.com/apache/iceberg/pull/9952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554171392 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,114 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] In case of Positional Deletes "file_path" in lowerbound & upperbound do not have full file_path Buffer [iceberg]

2024-04-05 Thread via GitHub
agrawalreetika commented on issue #10064: URL: https://github.com/apache/iceberg/issues/10064#issuecomment-2040458075 Thank you for your response @singhpk234 So we are using iceberg version 1.4.3 in Presto. And I checked https://github.com/apache/iceberg/pull/6313 looks like it's

Re: [I] In case of Positional Deletes "file_path" in lowerbound & upperbound do not have full file_path Buffer [iceberg]

2024-04-05 Thread via GitHub
singhpk234 commented on issue #10064: URL: https://github.com/apache/iceberg/issues/10064#issuecomment-2040485782 seems like we always get default config setting in presto

Re: [I] Support client-side purge in REST catalog [iceberg]

2024-04-05 Thread via GitHub
flyrain commented on issue #10089: URL: https://github.com/apache/iceberg/issues/10089#issuecomment-2040599592 That's right. In our case, the rest server cannot access all table files due to following reasons: 1. The rest catalog or any other catalog isn't allowed to access users' data

Re: [PR] Extend HTTPClient Builder to allow setting a proxy server [iceberg]

2024-04-05 Thread via GitHub
harishch1998 commented on code in PR #10052: URL: https://github.com/apache/iceberg/pull/10052#discussion_r1554322900 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -468,6 +483,19 @@ public Builder uri(String path) { return this; } +

Re: [PR] Extend HTTPClient Builder to allow setting a proxy server [iceberg]

2024-04-05 Thread via GitHub
harishch1998 commented on code in PR #10052: URL: https://github.com/apache/iceberg/pull/10052#discussion_r1554325691 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPClient.java: ## @@ -121,6 +128,92 @@ public void testHeadFailure() throws JsonProcessingException {

Re: [PR] Introduce two properties for reading the connection timeout and socke… [iceberg]

2024-04-05 Thread via GitHub
harishch1998 commented on code in PR #10053: URL: https://github.com/apache/iceberg/pull/10053#discussion_r1554368145 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPClient.java: ## @@ -133,6 +136,59 @@ public void testDynamicHttpRequestInterceptorLoading() {

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1554438555 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [I] Move site-related things to a separate repo [iceberg]

2024-04-05 Thread via GitHub
github-actions[bot] commented on issue #2446: URL: https://github.com/apache/iceberg/issues/2446#issuecomment-2040804831 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] Class not found error when use sink [iceberg]

2024-04-05 Thread via GitHub
github-actions[bot] commented on issue #2455: URL: https://github.com/apache/iceberg/issues/2455#issuecomment-2040804841 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [I] Build fails due to inaccessible palantir dependencies [iceberg]

2024-04-05 Thread via GitHub
github-actions[bot] commented on issue #2462: URL: https://github.com/apache/iceberg/issues/2462#issuecomment-2040804859 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1554519334 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

[PR] Validate overwrite filter [iceberg-python]

2024-04-05 Thread via GitHub
jqin61 opened a new pull request, #582: URL: https://github.com/apache/iceberg-python/pull/582 **Background** When we are doing a static overwrite, we could choose to overwrite the full table or overwrite some partitions of the table. Example spark sql counterpart in [iceberg

Re: [I] [BUG] Valid column characters fail on to_arrow() or to_pandas() ArrowInvalid: No match for FieldRef.Name [iceberg-python]

2024-04-05 Thread via GitHub
gwindes commented on issue #584: URL: https://github.com/apache/iceberg-python/issues/584#issuecomment-2040825581 Further research shows that when I use [daft](https://www.getdaft.io/projects/docs/en/latest/user_guide/integrations/iceberg.html#reading-a-table) that I'm able to read and use

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554520971 ## crates/iceberg/src/expr/accessor.rs: ## @@ -0,0 +1,119 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] [WIP] Integration with Datafusion [iceberg-rust]

2024-04-05 Thread via GitHub
marvinlanhenke commented on PR #324: URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2040961635 @liurenjie1024 @ZENOTME @Fokko PTAL and let me know what you think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [WIP] Integration with Datafusion [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on PR #324: URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2040961957 Thanks @marvinlanhenke This is amazing, I'll take a review later! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Extend HTTPClient Builder to allow setting a proxy server [iceberg]

2024-04-05 Thread via GitHub
harishch1998 commented on code in PR #10052: URL: https://github.com/apache/iceberg/pull/10052#discussion_r1554343246 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -468,6 +483,19 @@ public Builder uri(String path) { return this; } +

Re: [PR] Introduce two properties for reading the connection timeout and socke… [iceberg]

2024-04-05 Thread via GitHub
harishch1998 commented on code in PR #10053: URL: https://github.com/apache/iceberg/pull/10053#discussion_r1554368145 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPClient.java: ## @@ -133,6 +136,59 @@ public void testDynamicHttpRequestInterceptorLoading() {

[I] Hive Catalog cannot create table with TimestamptzType field [iceberg-python]

2024-04-05 Thread via GitHub
HonahX opened a new issue, #583: URL: https://github.com/apache/iceberg-python/issues/583 ### Apache Iceberg version 0.6.0 (latest release) ### Please describe the bug  This bug was reported by Jorge Arada on slack:

[I] [BUG] Valid column characters fail on to_arrow() or to_pandas() ArrowInvalid: No match for FieldRef.Name [iceberg-python]

2024-04-05 Thread via GitHub
gwindes opened a new issue, #584: URL: https://github.com/apache/iceberg-python/issues/584 ### Apache Iceberg version 0.6.0 (latest release) ### Please describe the bug  Related to: https://github.com/apache/iceberg-python/issues/81 Python `v3.12` ```

Re: [PR] Introduce two properties for reading the connection timeout and socke… [iceberg]

2024-04-05 Thread via GitHub
harishch1998 commented on code in PR #10053: URL: https://github.com/apache/iceberg/pull/10053#discussion_r1544095514 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -78,6 +80,8 @@ public class HTTPClient implements RESTClient { private static final int

Re: [PR] Add Struct Accessors to BoundReferences [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #317: URL: https://github.com/apache/iceberg-rust/pull/317#discussion_r1554425354 ## crates/iceberg/src/spec/schema.rs: ## @@ -137,9 +142,55 @@ impl SchemaBuilder { name_to_id, lowercase_name_to_id, id_to_name,

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
sdd commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1554441701 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] Add `BoundPredicateVisitor` trait [iceberg-rust]

2024-04-05 Thread via GitHub
liurenjie1024 commented on code in PR #320: URL: https://github.com/apache/iceberg-rust/pull/320#discussion_r1554519498 ## crates/iceberg/src/expr/visitors/bound_predicate_visitor.rs: ## @@ -0,0 +1,366 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

Re: [PR] [Bug Fix] Allow HiveCatalog to create table with TimestamptzType [iceberg-python]

2024-04-05 Thread via GitHub
HonahX commented on code in PR #585: URL: https://github.com/apache/iceberg-python/pull/585#discussion_r1554526443 ## pyiceberg/catalog/hive.py: ## @@ -103,22 +104,6 @@ import pyarrow as pa -# Replace by visitor -hive_types = { Review Comment: This is not used so I

Re: [PR] [Bug Fix] Allow HiveCatalog to create table with TimestamptzType [iceberg-python]

2024-04-05 Thread via GitHub
HonahX commented on code in PR #585: URL: https://github.com/apache/iceberg-python/pull/585#discussion_r1554524560 ## pyiceberg/catalog/hive.py: ## @@ -199,6 +184,7 @@ def _annotate_namespace(database: HiveDatabase, properties: Properties) -> HiveD DateType: "date",

[PR] [0.6.x] Backport #585 [iceberg-python]

2024-04-05 Thread via GitHub
HonahX opened a new pull request, #586: URL: https://github.com/apache/iceberg-python/pull/586 Backport #585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] SparkSessionCatalog with JDBC catalog: SHOW TABLES IN ... returns error but table exists in JDBC catalog [iceberg]

2024-04-05 Thread via GitHub
matepek commented on issue #10003: URL: https://github.com/apache/iceberg/issues/10003#issuecomment-2040974202 Thank you for the answers. > fundamentally that issue is the same Yes I think I understand that. What I'm surprised that why for that table creation call stack I

Re: [PR] [Bug Fix] Allow HiveCatalog to create table with TimestamptzType [iceberg-python]

2024-04-05 Thread via GitHub
HonahX commented on code in PR #585: URL: https://github.com/apache/iceberg-python/pull/585#discussion_r1554526443 ## pyiceberg/catalog/hive.py: ## @@ -103,22 +104,6 @@ import pyarrow as pa -# Replace by visitor -hive_types = { Review Comment: This is not used so I

Re: [PR] [Bug Fix] Allow HiveCatalog to create table with TimestamptzType [iceberg-python]

2024-04-05 Thread via GitHub
HonahX commented on code in PR #585: URL: https://github.com/apache/iceberg-python/pull/585#discussion_r1554526443 ## pyiceberg/catalog/hive.py: ## @@ -103,22 +104,6 @@ import pyarrow as pa -# Replace by visitor -hive_types = { Review Comment: This is not used so I