[GitHub] [incubator-iceberg] prodeezy commented on issue #312: Add batch sizing by num rows as config for Vectorized reads

2019-08-07 Thread GitBox
prodeezy commented on issue #312: Add batch sizing by num rows as config for Vectorized reads URL: https://github.com/apache/incubator-iceberg/issues/312#issuecomment-518967213 yes. thank you @rdblue This is an automated

[GitHub] [incubator-iceberg] jun-he opened a new pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
jun-he opened a new pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357 Implements #39 . This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] jun-he commented on issue #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
jun-he commented on issue #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#issuecomment-518973038 I took a different approach than the previous incomplete PR (Netflix/iceberg#81). Instead of adding `CollectionLiteral` and `CollectionPredicate`

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
aokolnychyi commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311501300 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0

[GitHub] [incubator-iceberg] prabcs commented on issue #179: Use Iceberg tables as sources for Spark Structured Streaming

2019-08-07 Thread GitBox
prabcs commented on issue #179: Use Iceberg tables as sources for Spark Structured Streaming URL: https://github.com/apache/incubator-iceberg/issues/179#issuecomment-519074048 Ok, take it up. Thanks On Wed, Aug 7, 2019 at 5:27 PM Li Cheng wrote: > No work done yet. Feel

[GitHub] [incubator-iceberg] aokolnychyi edited a comment on issue #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
aokolnychyi edited a comment on issue #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#issuecomment-519054168 I see the benefit of reusing `OverwriteFiles`, so let's think it through. We have the following methods in

[GitHub] [incubator-iceberg] aokolnychyi edited a comment on issue #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
aokolnychyi edited a comment on issue #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#issuecomment-519054168 I see the benefit of reusing `OverwriteFiles`, so let's think it through. We have the following methods in

[GitHub] [incubator-iceberg] timmylicheng commented on issue #179: Use Iceberg tables as sources for Spark Structured Streaming

2019-08-07 Thread GitBox
timmylicheng commented on issue #179: Use Iceberg tables as sources for Spark Structured Streaming URL: https://github.com/apache/incubator-iceberg/issues/179#issuecomment-519064936 No work done yet. Feel free to take this up if you already have something done. Just keep us posted. :)

[GitHub] [incubator-iceberg] rdblue opened a new issue #358: Add sequence numbers to metadata

2019-08-07 Thread GitBox
rdblue opened a new issue #358: Add sequence numbers to metadata URL: https://github.com/apache/incubator-iceberg/issues/358 Row-level deletes require a way to determine whether a given delete file applies to a data file. The scope of a delete file will be limited in 2 ways: by time

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311678614 ## File path: api/src/main/java/org/apache/iceberg/expressions/UnboundPredicate.java ## @@ -125,13 +132,26 @@

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311633613 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0 +1,114

[GitHub] [incubator-iceberg] rdblue opened a new issue #359: Spec: Add file and position delete files

2019-08-07 Thread GitBox
rdblue opened a new issue #359: Spec: Add file and position delete files URL: https://github.com/apache/incubator-iceberg/issues/359 One option for encoding row-level deletes is to track rows by file and row position in the file. This format should be documented in the spec.

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311676526 ## File path: api/src/main/java/org/apache/iceberg/expressions/Expressions.java ## @@ -105,6 +105,16 @@ public

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311681044 ## File path: api/src/main/java/org/apache/iceberg/expressions/Predicate.java ## @@ -19,15 +19,44 @@ package

[GitHub] [incubator-iceberg] rdblue commented on issue #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
rdblue commented on issue #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#issuecomment-519167629 > we need to track the base snapshot id when the data was read and it should not change during retries Do we need to add

[GitHub] [incubator-iceberg] rdblue opened a new issue #361: Spec: Add delete file partition and sequence number scope rules

2019-08-07 Thread GitBox
rdblue opened a new issue #361: Spec: Add delete file partition and sequence number scope rules URL: https://github.com/apache/incubator-iceberg/issues/361 This is an automated message from the Apache Git Service. To

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311674751 ## File path: api/src/main/java/org/apache/iceberg/expressions/ExpressionVisitors.java ## @@ -89,11 +91,11 @@

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311680177 ## File path: api/src/main/java/org/apache/iceberg/expressions/Evaluator.java ## @@ -134,13 +135,13 @@ public

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311673642 ## File path: api/src/main/java/org/apache/iceberg/expressions/Predicate.java ## @@ -62,10 +95,10 @@ public

[GitHub] [incubator-iceberg] TGooch44 commented on issue #355: python: Remove unused deps

2019-08-07 Thread GitBox
TGooch44 commented on issue #355: python: Remove unused deps URL: https://github.com/apache/incubator-iceberg/pull/355#issuecomment-519192573 Should be fine we talked about it over slack. Six isn't needed anymore, pytest isn't an install dependency and the bandit command was referencing

[GitHub] [incubator-iceberg] rdblue commented on issue #355: python: Remove unused deps

2019-08-07 Thread GitBox
rdblue commented on issue #355: python: Remove unused deps URL: https://github.com/apache/incubator-iceberg/pull/355#issuecomment-519204487 Thanks for reviewing, @TGooch44! I'm merging this. This is an automated message from

[GitHub] [incubator-iceberg] rdblue commented on issue #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on issue #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#issuecomment-519204167 Nice work, @jun-he! I think this is looking good, but there are a few areas to fix. Thanks for working on this!

[GitHub] [incubator-iceberg] rdblue merged pull request #355: python: Remove unused deps

2019-08-07 Thread GitBox
rdblue merged pull request #355: python: Remove unused deps URL: https://github.com/apache/incubator-iceberg/pull/355 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-iceberg] rdsr commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
rdsr commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519263693 All good points. > I'd use Iceberg's Record interface instead. GenericRecord is the implementation class.

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311684316 ## File path: api/src/main/java/org/apache/iceberg/expressions/Expressions.java ## @@ -105,6 +105,16 @@ public

[GitHub] [incubator-iceberg] aokolnychyi opened a new pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
aokolnychyi opened a new pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362 This PR adds support for create and replace transactions in Catalog and resolves #261.

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311708146 ## File path: api/src/main/java/org/apache/iceberg/catalog/Catalog.java ## @@ -120,6 +120,14 @@

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
moulimukherjee commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311725364 ## File path: api/src/main/java/org/apache/iceberg/transforms/Truncate.java ## @@ -213,20 +213,35 @@

[GitHub] [incubator-iceberg] aokolnychyi commented on issue #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
aokolnychyi commented on issue #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#issuecomment-519237696 > Do we need to add this to the constructor? I think it should be possible to start an overwrite, then scan the same table

[GitHub] [incubator-iceberg] johnclara commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
johnclara commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311741878 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0

[GitHub] [incubator-iceberg] johnclara commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
johnclara commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311741878 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0

[GitHub] [incubator-iceberg] rdblue commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
rdblue commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519258569 > IcebergPigInputFormat relies on Java serialization via org.apache.pig.impl.util.ObjectSerializer, shall I

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311682632 ## File path: api/src/main/java/org/apache/iceberg/expressions/Predicate.java ## @@ -19,15 +19,44 @@ package

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
aokolnychyi commented on a change in pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362#discussion_r311696942 ## File path: api/src/main/java/org/apache/iceberg/catalog/Catalog.java ## @@ -97,6

[GitHub] [incubator-iceberg] sujithjay commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
sujithjay commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311713190 ## File path: api/src/main/java/org/apache/iceberg/transforms/Truncate.java ## @@ -213,20 +213,35 @@ public

[GitHub] [incubator-iceberg] sujithjay commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
sujithjay commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311715547 ## File path: api/src/main/java/org/apache/iceberg/transforms/Truncate.java ## @@ -243,17 +243,35 @@ public

[GitHub] [incubator-iceberg] moulimukherjee commented on issue #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
moulimukherjee commented on issue #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#issuecomment-519242836 Some nits, but the changes look good to me . This is an automated

[GitHub] [incubator-iceberg] johnclara commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
johnclara commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311741878 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0

[GitHub] [incubator-iceberg] johnclara commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
johnclara commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311741878 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0

[GitHub] [incubator-iceberg] sujithjay commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
sujithjay commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311752998 ## File path: api/src/test/java/org/apache/iceberg/transforms/TestStartsWith.java ## @@ -0,0 +1,99 @@ +/* +

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311685596 ## File path: api/src/main/java/org/apache/iceberg/expressions/Evaluator.java ## @@ -134,13 +135,13 @@ public

[GitHub] [incubator-iceberg] sujithjay commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
sujithjay commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311715547 ## File path: api/src/main/java/org/apache/iceberg/transforms/Truncate.java ## @@ -243,17 +243,35 @@ public

[GitHub] [incubator-iceberg] aokolnychyi commented on a change in pull request #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
aokolnychyi commented on a change in pull request #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r311756153 ## File path: core/src/main/java/org/apache/iceberg/DefaultModifyFiles.java ## @@ -0,0

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311762522 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81 @@

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311761154 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81 @@

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311761500 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81 @@

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311763069 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81 @@

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311762522 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81 @@

[GitHub] [incubator-iceberg] rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdsr commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311762522 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81 @@

[GitHub] [incubator-iceberg] rdblue opened a new issue #363: Python: Add Quickstart doc to iceberg.apache.org

2019-08-07 Thread GitBox
rdblue opened a new issue #363: Python: Add Quickstart doc to iceberg.apache.org URL: https://github.com/apache/incubator-iceberg/issues/363 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-iceberg] rdblue commented on issue #351: Provide an API to modify records within files

2019-08-07 Thread GitBox
rdblue commented on issue #351: Provide an API to modify records within files URL: https://github.com/apache/incubator-iceberg/pull/351#issuecomment-519261508 I think that plan sounds reasonable. > Could you elaborate a bit? I am not sure I got the use case. We need to make

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #357: Jun/in and not in predicates

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #357: Jun/in and not in predicates URL: https://github.com/apache/incubator-iceberg/pull/357#discussion_r311683906 ## File path: api/src/main/java/org/apache/iceberg/expressions/UnboundPredicate.java ## @@ -125,13 +132,26 @@

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
moulimukherjee commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311725953 ## File path: api/src/test/java/org/apache/iceberg/transforms/TestStartsWith.java ## @@ -0,0 +1,99 @@

[GitHub] [incubator-iceberg] guilload commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
guilload commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519238245 Hey, I started working on an MR InputFormat for Iceberg since there’s some overlap with the

[GitHub] [incubator-iceberg] moulimukherjee commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
moulimukherjee commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311731335 ## File path: api/src/test/java/org/apache/iceberg/transforms/TestStartsWith.java ## @@ -0,0 +1,99 @@

[GitHub] [incubator-iceberg] TGooch44 commented on a change in pull request #277: Moving/Renaming hadoop module to filesystem

2019-08-07 Thread GitBox
TGooch44 commented on a change in pull request #277: Moving/Renaming hadoop module to filesystem URL: https://github.com/apache/incubator-iceberg/pull/277#discussion_r311749843 ## File path: python/iceberg/core/filesystem/local_filesystem.py ## @@ -51,34 +63,19 @@ def

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311758102 ## File path: api/src/test/java/org/apache/iceberg/transforms/TestStartsWith.java ## @@ -0,0 +1,99 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #327: Introduce startsWith Predicate

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #327: Introduce startsWith Predicate URL: https://github.com/apache/incubator-iceberg/pull/327#discussion_r311758230 ## File path: api/src/test/java/org/apache/iceberg/transforms/TestStartsWith.java ## @@ -0,0 +1,99 @@ +/* + *

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311780357 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362#discussion_r311807833 ## File path: hive/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java

[GitHub] [incubator-iceberg] rdblue opened a new pull request #364: Clean up after create and replace transaction failures.

2019-08-07 Thread GitBox
rdblue opened a new pull request #364: Clean up after create and replace transaction failures. URL: https://github.com/apache/incubator-iceberg/pull/364 This is a follow-up to #352. The create and replace transactions did not delete deleted files. This also calls `cleanAll()` for each

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #350: Add dropTable purge option to Catalog API

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #350: Add dropTable purge option to Catalog API URL: https://github.com/apache/incubator-iceberg/pull/350#discussion_r311780233 ## File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java ## @@ -119,4 +133,81

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362#discussion_r311807434 ## File path: hive/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362#discussion_r311807710 ## File path: hive/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362#discussion_r311807151 ## File path: hive/src/test/java/org/apache/iceberg/hive/HiveCreateReplaceTableTest.java

[GitHub] [incubator-iceberg] guilload commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
guilload commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519342060 Ok, thanks for the pointers. That's helpful. I'll start with `HiveCatalogs` and we can revisit later.

[GitHub] [incubator-iceberg] guilload edited a comment on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
guilload edited a comment on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519342060 Ok, thanks for the pointers. That's helpful. I'll start with `HiveCatalogs` and we can revisit later.

[GitHub] [incubator-iceberg] guilload edited a comment on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
guilload edited a comment on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519342060 Ok, thanks for the pointers. That's helpful. I'll start with `HiveCatalogs` and we can revisit later.

[GitHub] [incubator-iceberg] rdsr edited a comment on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs

2019-08-07 Thread GitBox
rdsr edited a comment on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-519263693 All good points. > I'd use Iceberg's Record interface instead. GenericRecord is the implementation

[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog

2019-08-07 Thread GitBox
rdblue commented on a change in pull request #362: Support create and replace transactions in Catalog URL: https://github.com/apache/incubator-iceberg/pull/362#discussion_r311806870 ## File path: api/src/main/java/org/apache/iceberg/catalog/Catalog.java ## @@ -97,6 +98,42