[GitHub] [incubator-iceberg] ravichinoy opened a new issue #741: Spark DataFrame write fails if input dataframe has columns in different order than iceberg schema

ravichinoy opened a new issue #741: Spark DataFrame write fails if input dataframe has columns in different order than iceberg schema URL: https://github.com/apache/incubator-iceberg/issues/741 For this test case,

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #735: Refactor FilteredManifest and ManifestGroup

rdblue commented on a change in pull request #735: Refactor FilteredManifest and ManifestGroup URL: https://github.com/apache/incubator-iceberg/pull/735#discussion_r368159068 ## File path: core/src/main/java/org/apache/iceberg/FilteredManifest.java ## @@ -193,4 +164,19 @@

[GitHub] [incubator-iceberg] samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads

samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r36810 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/ArrowSchemaUtil.java ## @@ -0,0

[GitHub] [incubator-iceberg] samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads

samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r368105495 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/ArrowSchemaUtil.java ## @@ -0,0

[GitHub] [incubator-iceberg] samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads

samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r36810 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/ArrowSchemaUtil.java ## @@ -0,0

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #735: Refactor FilteredManifest and ManifestGroup

rdblue commented on a change in pull request #735: Refactor FilteredManifest and ManifestGroup URL: https://github.com/apache/incubator-iceberg/pull/735#discussion_r368158161 ## File path: core/src/main/java/org/apache/iceberg/FilteredManifest.java ## @@ -95,60 +95,31 @@

[GitHub] [incubator-iceberg] samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads

samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r368171781 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #723: Arrow changes for supporting vectorized reads

rdblue commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r368132505 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/ArrowSchemaUtil.java ## @@ -0,0

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #723: Arrow changes for supporting vectorized reads

rdblue commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r368132185 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/ArrowSchemaUtil.java ## @@ -0,0

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #735: Refactor FilteredManifest and ManifestGroup

rdblue commented on a change in pull request #735: Refactor FilteredManifest and ManifestGroup URL: https://github.com/apache/incubator-iceberg/pull/735#discussion_r368158676 ## File path: core/src/main/java/org/apache/iceberg/FilteredManifest.java ## @@ -95,60 +95,31 @@

[GitHub] [incubator-iceberg] prodeezy commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

prodeezy commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368160148 ## File path: core/src/main/java/org/apache/iceberg/TableMetadata.java ## @@ -589,4 +603,20 @@ private

[GitHub] [incubator-iceberg] samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads

samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r368177847 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorHolder.java ##

[GitHub] [incubator-iceberg] samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads

samarthjain commented on a change in pull request #723: Arrow changes for supporting vectorized reads URL: https://github.com/apache/incubator-iceberg/pull/723#discussion_r368105495 ## File path: arrow/src/main/java/org/apache/iceberg/arrow/ArrowSchemaUtil.java ## @@ -0,0

[GitHub] [incubator-iceberg] prodeezy commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

prodeezy commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368130670 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] prodeezy commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

prodeezy commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368156637 ## File path: core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java ## @@ -55,4 +56,28 @@ private

[GitHub] [incubator-iceberg] aokolnychyi commented on issue #682: Add Parquet user-defined predicate to push down StartsWith

aokolnychyi commented on issue #682: Add Parquet user-defined predicate to push down StartsWith URL: https://github.com/apache/incubator-iceberg/issues/682#issuecomment-575520094 I think #327 provided a basis and #398 implemented `startsWith` in our dictionary and metrics filters in

[GitHub] [incubator-iceberg] aokolnychyi closed issue #399: Add timeout for acquiring locks in HiveTableOperations

aokolnychyi closed issue #399: Add timeout for acquiring locks in HiveTableOperations URL: https://github.com/apache/incubator-iceberg/issues/399 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] aokolnychyi commented on issue #399: Add timeout for acquiring locks in HiveTableOperations

aokolnychyi commented on issue #399: Add timeout for acquiring locks in HiveTableOperations URL: https://github.com/apache/incubator-iceberg/issues/399#issuecomment-575518495 Thanks for fixing it, @waterlx! This is an

[GitHub] [incubator-iceberg] chenjunjiedada commented on a change in pull request #738: Collect row stats while writing manifests

chenjunjiedada commented on a change in pull request #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#discussion_r367865824 ## File path: core/src/test/java/org/apache/iceberg/TestManifestWriter.java ## @@ -0,0

[GitHub] [incubator-iceberg] rdblue closed issue #683: Add Parquet user-defined predicates to push down IN and NOT_IN

rdblue closed issue #683: Add Parquet user-defined predicates to push down IN and NOT_IN URL: https://github.com/apache/incubator-iceberg/issues/683 This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #738: Collect row stats while writing manifests

rdblue commented on a change in pull request #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#discussion_r368042226 ## File path: api/src/main/java/org/apache/iceberg/ManifestFile.java ## @@ -42,7 +42,10 @@

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #738: Collect row stats while writing manifests

rdblue commented on a change in pull request #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#discussion_r368043053 ## File path: core/src/test/java/org/apache/iceberg/TestManifestWriter.java ## @@ -0,0 +1,65 @@ +/*

[GitHub] [incubator-iceberg] rdblue commented on issue #738: Collect row stats while writing manifests

rdblue commented on issue #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#issuecomment-575713745 Looks good to me. This is an automated message from the Apache Git

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368044348 ## File path: api/src/main/java/org/apache/iceberg/Table.java ## @@ -201,11 +201,16 @@ default AppendFiles

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368044582 ## File path: api/src/main/java/org/apache/iceberg/Table.java ## @@ -201,11 +201,16 @@ default AppendFiles

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368044977 ## File path: api/src/main/java/org/apache/iceberg/Table.java ## @@ -201,11 +201,16 @@ default AppendFiles

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368045957 ## File path: core/src/main/java/org/apache/iceberg/RollbackToSnapshot.java ## @@ -19,61 +19,21 @@

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368047513 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368048071 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368050164 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368051340 ## File path: core/src/main/java/org/apache/iceberg/TableMetadata.java ## @@ -589,4 +603,20 @@ private

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368054530 ## File path: spark/src/main/java/org/apache/iceberg/spark/source/Writer.java ## @@ -108,6 +109,7 @@

[GitHub] [incubator-iceberg] maqroll commented on issue #682: Add Parquet user-defined predicate to push down StartsWith

maqroll commented on issue #682: Add Parquet user-defined predicate to push down StartsWith URL: https://github.com/apache/incubator-iceberg/issues/682#issuecomment-575724144 Sorry to jump in but maybe this issue refers to handle STARTS_WITH case with an UserDefinedPredicate

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368055946 ## File path: core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java ## @@ -55,4 +56,28 @@ private

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368056858 ## File path: core/src/main/java/org/apache/iceberg/TableMetadata.java ## @@ -391,17 +401,34 @@ public

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368056543 ## File path: core/src/main/java/org/apache/iceberg/TableMetadata.java ## @@ -436,7 +454,7 @@ public

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368056349 ## File path: core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java ## @@ -55,4 +56,28 @@ private

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368062720 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #738: Collect row stats while writing manifests

aokolnychyi commented on a change in pull request #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#discussion_r368062884 ## File path: api/src/main/java/org/apache/iceberg/ManifestFile.java ## @@ -42,7 +42,10 @@

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368063917 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature

rdblue commented on a change in pull request #695: [WIP] Cherrypick snapshot feature URL: https://github.com/apache/incubator-iceberg/pull/695#discussion_r368064413 ## File path: core/src/main/java/org/apache/iceberg/SnapshotManager.java ## @@ -0,0 +1,152 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #738: Collect row stats while writing manifests

rdblue commented on a change in pull request #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#discussion_r368065049 ## File path: api/src/main/java/org/apache/iceberg/ManifestFile.java ## @@ -42,7 +42,10 @@

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #738: Collect row stats while writing manifests

aokolnychyi commented on a change in pull request #738: Collect row stats while writing manifests URL: https://github.com/apache/incubator-iceberg/pull/738#discussion_r368065778 ## File path: api/src/main/java/org/apache/iceberg/ManifestFile.java ## @@ -42,7 +42,10 @@

[GitHub] [incubator-iceberg] rdblue commented on issue #682: Add Parquet user-defined predicate to push down StartsWith

rdblue commented on issue #682: Add Parquet user-defined predicate to push down StartsWith URL: https://github.com/apache/incubator-iceberg/issues/682#issuecomment-575734167 When I opened this issue, I was thinking about the old read path. That's the one that @maqroll correctly pointed

[GitHub] [incubator-iceberg] rdblue edited a comment on issue #682: Add Parquet user-defined predicate to push down StartsWith

rdblue edited a comment on issue #682: Add Parquet user-defined predicate to push down StartsWith URL: https://github.com/apache/incubator-iceberg/issues/682#issuecomment-575734167 When I opened this issue, I was thinking about the old read path. That's the one that @maqroll correctly

[GitHub] [incubator-iceberg] rdblue commented on issue #682: Add Parquet user-defined predicate to push down StartsWith

rdblue commented on issue #682: Add Parquet user-defined predicate to push down StartsWith URL: https://github.com/apache/incubator-iceberg/issues/682#issuecomment-575735179 I'm going to close this one since I think it covers a case that we don't really need to worry about.