> On July 21, 2016, 12:42 a.m., Lefty Leverenz wrote: > >
@Lefty, does the updated description for this config variable seem to explain better now? - Saket ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49766/#review143059 ----------------------------------------------------------- On July 27, 2016, 2:54 p.m., Saket Saurabh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49766/ > ----------------------------------------------------------- > > (Updated July 27, 2016, 2:54 p.m.) > > > Review request for hive and Eugene Koifman. > > > Repository: hive-git > > > Description > ------- > > https://issues.apache.org/jira/browse/HIVE-14035 > > In current Hive version, delta files created by ACID transactions do not > allow predicate pushdown if they contain any update/delete events. This is > done to preserve correctness when following a multi-version approach during > event collapsing, where an update event overwrites an existing insert event. > This JIRA proposes to split an update event into a combination of a delete > event followed by a new insert event, that can enable predicate push down to > all delta files without breaking correctness. To support backward > compatibility for this feature, this JIRA also proposes to add some sort of > versioning to ACID that can allow different versions of ACID transactions to > co-exist together. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e92466f > > hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java > 14f7316 > > hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java > 974c6b8 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java > ca2a912 > metastore/if/hive_metastore.thrift 4d92b73 > metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h ae14bd1 > metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp f982bf2 > > metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/hive_metastoreConstants.java > 5a666f2 > metastore/src/gen/thrift/gen-php/metastore/Types.php f505208 > metastore/src/gen/thrift/gen-py/hive_metastore/constants.py d1c07a5 > metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb eeccc84 > > metastore/src/java/org/apache/hadoop/hive/metastore/TransactionalValidationListener.java > 3e74675 > orc/src/java/org/apache/orc/impl/TreeReaderFactory.java c4a2093 > ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java db6848a > ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 57b6c67 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 23a13d6 > ql/src/java/org/apache/hadoop/hive/ql/io/AcidOutputFormat.java dd90a95 > ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java c150ec5 > ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 945b828 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 63d02fb > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 1a1af28 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 9d927bd > ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 8cf261d > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > 6caca98 > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java d48e441 > ql/src/test/org/apache/hadoop/hive/ql/io/TestAcidUtils.java b83cea4 > > Diff: https://reviews.apache.org/r/49766/diff/ > > > Testing > ------- > > Tests for the feature are in > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java. These are mostly > integration tests that test end-to-end insert/update/delete scenarios > followed by compaction and cleaning. > > > Thanks, > > Saket Saurabh > >
