[GitHub] [incubator-iceberg] chenjunjiedada commented on issue #415: supports unpartitioned table in partitionDF

chenjunjiedada commented on issue #415: supports unpartitioned table in partitionDF URL: https://github.com/apache/incubator-iceberg/issues/415#issuecomment-525157629 Close this first, will add some doc to `partitionDF` to address it.

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r317910185 ## File path: api/src/main/java/org/apache/iceberg/expressions/ExpressionVisitors.java ## @@ -51,6 +53,10 @@

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

aokolnychyi commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r317964972 ## File path:

[GitHub] [incubator-iceberg] chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table

chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318171088 ## File path: spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala ## @@ -297,5

[GitHub] [incubator-iceberg] chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table

chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318174610 ## File path: spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala ## @@ -297,5

[GitHub] [incubator-iceberg] chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table

chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318176144 ## File path: spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala ## @@ -297,5

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r318178750 ## File path: api/src/test/java/org/apache/iceberg/expressions/TestInclusiveMetricsEvaluator.java

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318295030 ## File path: site/mkdocs.yml ## @@ -52,6 +52,7 @@ nav: - Quickstart:

[GitHub] [incubator-iceberg] rdblue opened a new pull request #420: Add commit.manifest-merge.enabled table property

rdblue opened a new pull request #420: Add commit.manifest-merge.enabled table property URL: https://github.com/apache/incubator-iceberg/pull/420 This property can disable manifest merging. This is intended to be used with `RewriteManifests`. Instead of automatic merging,

[GitHub] [incubator-iceberg] rdblue commented on issue #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on issue #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#issuecomment-525518658 Merged. Thanks @moulimukherjee! This is an automated

[GitHub] [incubator-iceberg] rdblue merged pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue merged pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318291645 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,149 @@ +# Custom

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318268122 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,149 @@ +# Custom

[GitHub] [incubator-iceberg] rdblue commented on issue #419: Adding common checks in commit()

rdblue commented on issue #419: Adding common checks in commit() URL: https://github.com/apache/incubator-iceberg/pull/419#issuecomment-525481988 Looks good to me. I'll merge once tests are passing. This is an automated

[GitHub] [incubator-iceberg] rdblue opened a new pull request #421: Add friendly names to catalog tables

rdblue opened a new pull request #421: Add friendly names to catalog tables URL: https://github.com/apache/incubator-iceberg/pull/421 This adds an abstract `name` method to `BaseMetastoreCatalog` and a new method to generate full table names from a catalog.

[GitHub] [incubator-iceberg] moulimukherjee opened a new pull request #419: Adding common checks in commit()

moulimukherjee opened a new pull request #419: Adding common checks in commit() URL: https://github.com/apache/incubator-iceberg/pull/419 Adding common checks in commit() From conversation at https://github.com/apache/incubator-iceberg/pull/416/files#r318243051 r? @rdblue

[GitHub] [incubator-iceberg] rdblue commented on issue #419: Adding common checks in commit()

rdblue commented on issue #419: Adding common checks in commit() URL: https://github.com/apache/incubator-iceberg/pull/419#issuecomment-525498398 Merged. Thanks @moulimukherjee! This is an automated message from the Apache

[GitHub] [incubator-iceberg] rdblue merged pull request #419: Adding common checks in commit()

rdblue merged pull request #419: Adding common checks in commit() URL: https://github.com/apache/incubator-iceberg/pull/419 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-iceberg] rdblue commented on issue #420: Add commit.manifest-merge.enabled table property

rdblue commented on issue #420: Add commit.manifest-merge.enabled table property URL: https://github.com/apache/incubator-iceberg/pull/420#issuecomment-525503667 @bryanck, here's the PR to disable manifest merging. This is an

[GitHub] [incubator-iceberg] rdblue merged pull request #386: Fix copies in partition data and field summaries

rdblue merged pull request #386: Fix copies in partition data and field summaries URL: https://github.com/apache/incubator-iceberg/pull/386 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

moulimukherjee commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318324317 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,139 @@ +# Custom

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318324025 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,139 @@ +# Custom Catalog

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318324221 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,139 @@ +# Custom Catalog

[GitHub] [incubator-iceberg] chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table

chenjunjiedada commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318368167 ## File path: spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala ## @@ -297,5

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #421: Add friendly names to catalog tables

rdsr commented on a change in pull request #421: Add friendly names to catalog tables URL: https://github.com/apache/incubator-iceberg/pull/421#discussion_r318396277 ## File path: hive/src/main/java/org/apache/iceberg/hive/HiveCatalog.java ## @@ -43,6 +43,11 @@ public

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402782 ## File path: api/src/test/java/org/apache/iceberg/expressions/TestEvaluator.java ## @@ -362,4 +368,197 @@ public

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402850 ## File path: parquet/src/main/java/org/apache/iceberg/parquet/ParquetMetricsRowGroupFilter.java ## @@ -19,19

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402818 ## File path: api/src/test/java/org/apache/iceberg/expressions/TestEvaluator.java ## @@ -362,4 +368,197 @@ public

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402697 ## File path: api/src/test/java/org/apache/iceberg/expressions/TestEvaluator.java ## @@ -362,4 +368,197 @@ public

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402737 ## File path: api/src/test/java/org/apache/iceberg/expressions/TestEvaluator.java ## @@ -362,4 +368,197 @@ public

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402641 ## File path: api/src/main/java/org/apache/iceberg/expressions/UnboundPredicate.java ## @@ -125,13 +149,40 @@

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402583 ## File path: api/src/main/java/org/apache/iceberg/expressions/UnboundPredicate.java ## @@ -125,13 +149,40 @@

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402556 ## File path: api/src/main/java/org/apache/iceberg/expressions/Predicate.java ## @@ -39,37 +37,35 @@ public R

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402469 ## File path: api/src/main/java/org/apache/iceberg/expressions/BoundSetPredicate.java ## @@ -0,0 +1,176 @@ +/* +

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318402380 ## File path: api/src/main/java/org/apache/iceberg/expressions/BoundSetPredicate.java ## @@ -0,0 +1,176 @@ +/* +

[GitHub] [incubator-iceberg] jun-he commented on a change in pull request #357: Add in and not in predicates

jun-he commented on a change in pull request #357: Add in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r318403061 ## File path: api/src/main/java/org/apache/iceberg/expressions/BoundSetPredicate.java ## @@ -0,0 +1,176 @@ +/* +

[GitHub] [incubator-iceberg] manishmalhotrawork commented on issue #280: Add persistent IDs to partition fields

manishmalhotrawork commented on issue #280: Add persistent IDs to partition fields URL: https://github.com/apache/incubator-iceberg/issues/280#issuecomment-525415789 @rdblue thanks. Let me take a stab on this, may be as you mentioned in description. Passing initial value for the

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318243051 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,149 @@ +# Custom Catalog

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318243051 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,149 @@ +# Custom Catalog

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318243982 ## File path: site/mkdocs.yml ## @@ -52,6 +52,7 @@ nav: - Quickstart:

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318244402 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,149 @@ +# Custom Catalog

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg

rdblue commented on a change in pull request #416: Adding docs on how to use custom catalog with iceberg URL: https://github.com/apache/incubator-iceberg/pull/416#discussion_r318252717 ## File path: site/docs/custom-catalog.md ## @@ -0,0 +1,149 @@ +# Custom Catalog

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #374: Migrate spark table to iceberg table

rdblue commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318254975 ## File path: spark/src/test/java/org/apache/iceberg/spark/source/TestSparkTableUtil.java ## @@ -116,4

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #374: Migrate spark table to iceberg table

rdblue commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318254975 ## File path: spark/src/test/java/org/apache/iceberg/spark/source/TestSparkTableUtil.java ## @@ -116,4

[GitHub] [incubator-iceberg] rdblue commented on issue #280: Add persistent IDs to partition fields

rdblue commented on issue #280: Add persistent IDs to partition fields URL: https://github.com/apache/incubator-iceberg/issues/280#issuecomment-525451795 > And also as TableMetadata knows how many fields are in partition, so can maintain the nextIDValue as well. The next partition

[GitHub] [incubator-iceberg] rdblue edited a comment on issue #280: Add persistent IDs to partition fields

rdblue edited a comment on issue #280: Add persistent IDs to partition fields URL: https://github.com/apache/incubator-iceberg/issues/280#issuecomment-525451795 > And also as TableMetadata knows how many fields are in partition, so can maintain the nextIDValue as well. The next

[GitHub] [incubator-iceberg] rdblue merged pull request #349: Remove unreferenced hash computations

rdblue merged pull request #349: Remove unreferenced hash computations URL: https://github.com/apache/incubator-iceberg/pull/349 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-iceberg] rdblue commented on issue #349: Remove unreferenced hash computations

rdblue commented on issue #349: Remove unreferenced hash computations URL: https://github.com/apache/incubator-iceberg/pull/349#issuecomment-525455644 +1 Thanks for fixing this @jbapple! This is an automated message

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #374: Migrate spark table to iceberg table

rdblue commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318200212 ## File path: spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala ## @@ -297,5 +302,81 @@

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r318191642 ## File path: api/src/test/java/org/apache/iceberg/expressions/TestInclusiveMetricsEvaluator.java

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r318196009 ## File path: parquet/src/main/java/org/apache/iceberg/parquet/ParquetMetricsRowGroupFilter.java

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

rdblue commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r318197341 ## File path: parquet/src/test/java/org/apache/iceberg/parquet/TestDictionaryRowGroupFilter.java

[GitHub] [incubator-iceberg] rdblue commented on issue #398: Push down StringStartsWith in Spark IcebergSource

rdblue commented on issue #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#issuecomment-525398380 +1 There were a couple of minor points, but I think we can fix those in a follow-up.

[GitHub] [incubator-iceberg] rdblue closed issue #397: Push down StringStartsWith in Spark IcebergSource

rdblue closed issue #397: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/issues/397 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [incubator-iceberg] rdblue merged pull request #398: Push down StringStartsWith in Spark IcebergSource

rdblue merged pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] rdblue merged pull request #388: Handle rollback in snapshot expiration

rdblue merged pull request #388: Handle rollback in snapshot expiration URL: https://github.com/apache/incubator-iceberg/pull/388 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [incubator-iceberg] rdblue commented on issue #411: Ignore unsupported partition transforms for forward compatibility

rdblue commented on issue #411: Ignore unsupported partition transforms for forward compatibility URL: https://github.com/apache/incubator-iceberg/pull/411#issuecomment-525412584 @aokolnychyi, can you review this? This is an

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

aokolnychyi commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r317970914 ## File path:

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource

aokolnychyi commented on a change in pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#discussion_r318107696 ## File path: spark/src/main/java/org/apache/iceberg/spark/SparkFilters.java ## @@

[GitHub] [incubator-iceberg] aokolnychyi closed pull request #398: Push down StringStartsWith in Spark IcebergSource

aokolnychyi closed pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] aokolnychyi opened a new pull request #398: Push down StringStartsWith in Spark IcebergSource

aokolnychyi opened a new pull request #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398 This PR adds support for pushdown of `StringStartsWith` predicates in Spark and resolves #397.

[GitHub] [incubator-iceberg] aokolnychyi commented on issue #398: Push down StringStartsWith in Spark IcebergSource

aokolnychyi commented on issue #398: Push down StringStartsWith in Spark IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/398#issuecomment-525331795 I extended the scope of this PR to also cover filtering of manifests.