Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2595/ --- (Updated 2011-10-30 07:34:36.866186) Review request for hive and namit jain. Changes --- Address review comments except the last one: Why have a for loop - you are returning inside if anyway the first time ? I think it is safe to return even for the first time. For any partition returned, it should either be archived or not. If the archive is based on its parent partition col, the partition should also be marked as archived, otherwise, it should not be there. Summary --- Support archiving for multiple partitions if the table is partitioned by multiple columns This addresses bug HIVE-2278. https://issues.apache.org/jira/browse/HIVE-2278 Diffs (updated) - trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1190490 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1190490 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1190490 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1190490 trunk/data/files/archive_corrupt.rc UNKNOWN trunk/metastore/if/hive_metastore.thrift 1190490 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1190490 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1190490 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1195116 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1190490 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1190490 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec5.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1190490 trunk/ql/src/test/results/clientnegative/archive2.q.out 1190490 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION
Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2595/ --- Review request for hive and namit jain. Summary --- Support archiving for multiple partitions if the table is partitioned by multiple columns This addresses bug HIVE-2278. https://issues.apache.org/jira/browse/HIVE-2278 Diffs - trunk/data/files/archive_corrupt.rc UNKNOWN trunk/metastore/if/hive_metastore.thrift 1188906 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1188906 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1188906 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1188906 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1188906 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1188906 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1188906 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1188906 trunk/ql/src/test/results/clientnegative/archive2.q.out 1188906 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/archive_corrupt.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/archive_multi.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2595/diff Testing --- Support archiving for multiple partitions if the table is partitioned by multiple columns Thanks, Yongqiang
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2595/#review2918 --- trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java https://reviews.apache.org/r/2595/#comment6526 In the example above, (hr='1', ds='2011-01-01') will work. It should not work - it would be simpler if you check the order also trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java https://reviews.apache.org/r/2595/#comment6527 add some comments trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java https://reviews.apache.org/r/2595/#comment6528 1 for loop instead of 2 for loops trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java https://reviews.apache.org/r/2595/#comment6529 I think there is a bug here - not sure Why have a for loop - you are returning inside if anyway the first time ? - namit On 2011-10-28 18:28:45, Yongqiang He wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2595/ --- (Updated 2011-10-28 18:28:45) Review request for hive and namit jain. Summary --- Support archiving for multiple partitions if the table is partitioned by multiple columns This addresses bug HIVE-2278. https://issues.apache.org/jira/browse/HIVE-2278 Diffs - trunk/data/files/archive_corrupt.rc UNKNOWN trunk/metastore/if/hive_metastore.thrift 1188906 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1188906 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1188906 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1188906 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1188906 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1188906 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1188906 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1188906 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1188906 trunk/ql/src/test/results/clientnegative/archive2.q.out 1188906 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-09-01 01:23:23.280266) Review request for hive, Paul Yang and namit jain. Changes --- Reverted accidentally deleted line. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271 trunk/data/files/archive_corrupt.rc UNKNOWN trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec3.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-11 23:31:47.018762) Review request for hive, Paul Yang and namit jain. Changes --- Fixed configuration (removed hook) Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/archive_corrupt.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1259/#review1359
---
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java
https://reviews.apache.org/r/1259/#comment3058
Do you want to return NULL for input like (hr='13') or return a non-null
PartSpecInfo but all fields (prefixFields and prefixValues) are null.
It seems the function implements the 2nd option. In this case how do you
distinguish the cases where partition spec is hr='13' and there is no
partition spec at all (meaning all partitions in the table)? Should we raise an
exception for the first case (hr='13') since it's not the correct usage?
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java
https://reviews.apache.org/r/1259/#comment3056
our coding convention is like:
if () {
} else {
}
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
https://reviews.apache.org/r/1259/#comment3059
Here the argument Map doesn't guarantee orders of the key-value pair.
You'll need to use LinkedHashMap for that purpose.
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
https://reviews.apache.org/r/1259/#comment3060
indentation
- Ning
On 2011-08-09 01:28:13, Marcin Kurczych wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1259/
---
(Updated 2011-08-09 01:28:13)
Review request for hive, Paul Yang and namit jain.
Summary
---
Allowing archiving at chosen level. When table is partitioned by ds, hr, min
it allows archiving at ds level, hr level and min level. Corresponding
syntaxes are:
ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08');
ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11');
ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30');
You cannot do much to archived partitions. You can read them. You cannot
write to them / overwrite them. You can drop single archived partitions, but
not parts of bigger archives.
Diffs
-
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java
1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271
trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271
trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java
PRE-CREATION
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271
trunk/data/conf/hive-site.xml 1153271
trunk/metastore/if/hive_metastore.thrift 1153271
trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271
trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271
trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java
1153271
trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php
1153271
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
1153271
trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/archive_partspec3.q
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
On 2011-08-03 22:39:04, Paul Yang wrote: trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java, lines 1172-1173 https://reviews.apache.org/r/1259/diff/1/?file=30272#file30272line1172 Should be info or debug, not error ok On 2011-08-03 22:39:04, Paul Yang wrote: trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java, line 1105 https://reviews.apache.org/r/1259/diff/1/?file=30272#file30272line1105 One possible issue is that if the user changes the value of this through the CLI (i.e. with a set xxx=yyy;), it wouldn't take effect. It should be read in the constructor or in the methods. ok - Marcin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/#review1278 --- On 2011-08-09 01:28:13, Marcin Kurczych wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-09 01:28:13) Review request for hive, Paul Yang and namit jain. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271 trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-09 21:46:08.628498) Review request for hive, Paul Yang and namit jain. Changes --- Changes requested by Ning. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs (updated) - trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271 trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec3.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-09 01:28:13.568217) Review request for hive, Paul Yang and namit jain. Changes --- Extracted some code into separate file with archiving utils, so it can be used in hooks more easily Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs (updated) - trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ArchiveUtils.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271 trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-04 22:27:36.104846) Review request for hive, Paul Yang and namit jain. Changes --- Archived partitions counter - contains archived partitions flag. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1153271 trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec3.q.out PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/#review1275 --- trunk/metastore/if/hive_metastore.thrift https://reviews.apache.org/r/1259/#comment2909 archiving_level seems to be a better name for this. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2910 use Path.SEPARATOR instead of '/' trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2911 '/' - Path.SEPARATOR trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2912 isn't the archive operation atomic? If an archive operation failed, and it's not recovered, does it mean the partitions will not be correct? trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java https://reviews.apache.org/r/1259/#comment2913 Change ArrayList... fields = to List... fields = . Variables should be declared as interface rather than implementation. trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java https://reviews.apache.org/r/1259/#comment2914 need some comments on this public function. - Ning On 2011-08-02 21:34:07, Marcin Kurczych wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-02 21:34:07) Review request for hive, Paul Yang and namit jain. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs - trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/#review1278 --- trunk/data/conf/hive-site.xml https://reviews.apache.org/r/1259/#comment2921 Why is a hook needed to update this count? trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2924 One possible issue is that if the user changes the value of this through the CLI (i.e. with a set xxx=yyy;), it wouldn't take effect. It should be read in the constructor or in the methods. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2923 Should be info or debug, not error trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2927 Why are we saving unboundKey? Can we just break on the first instance when partSpec does not contain fs.getName()? trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2928 Typo trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2929 Change to something like 'Cannot drop a subset of partitions in an archive' trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java https://reviews.apache.org/r/1259/#comment2930 This is going to be feature that we'll need to support. The script that drops based on retention can't unarchive as it will take too long. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java https://reviews.apache.org/r/1259/#comment2931 Need more context for these log messages - i.e. what kind of partitions were found? trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java https://reviews.apache.org/r/1259/#comment2934 The max parameter for getPartitions is not yet implemented. It needs to be or otherwise this will hang for large tables. trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java https://reviews.apache.org/r/1259/#comment2919 Comment from a different hook? trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java https://reviews.apache.org/r/1259/#comment2920 Formatting - add spaces trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java https://reviews.apache.org/r/1259/#comment2932 More context trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java https://reviews.apache.org/r/1259/#comment2933 Remove commented code - Paul On 2011-08-02 21:34:07, Marcin Kurczych wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-02 21:34:07) Review request for hive, Paul Yang and namit jain. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs - trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
On 2011-08-03 22:39:04, Paul Yang wrote: trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java, lines 1240-1249 https://reviews.apache.org/r/1259/diff/1/?file=30272#file30272line1240 Why are we saving unboundKey? Can we just break on the first instance when partSpec does not contain fs.getName()? No, we want to support prefix specs, so at the and there can be unbound keys. The error is thrown only when there's a hole inside of specification. On 2011-08-03 22:39:04, Paul Yang wrote: trunk/data/conf/hive-site.xml, line 142 https://reviews.apache.org/r/1259/diff/1/?file=30262#file30262line142 Why is a hook needed to update this count? Count is updated in archive, unarchive and drop. It has been decided, that it will be more clear to keep it in hook than add to drop (and maybe in future into other?) tasks. On 2011-08-03 22:39:04, Paul Yang wrote: trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java, line 119 https://reviews.apache.org/r/1259/diff/1/?file=30273#file30273line119 The max parameter for getPartitions is not yet implemented. It needs to be or otherwise this will hang for large tables. Well, you mean about class implementation or lower level one? I've added it to Hive class, but it works only under the assumption HiveMetaStoreClient.listPartitionsWithAuthInfo supports it (and it seems to depend on ThriftMetastoreIface.get_partitions_ps_with_auth). On 2011-08-03 22:39:04, Paul Yang wrote: trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java, line 3359 https://reviews.apache.org/r/1259/diff/1/?file=30272#file30272line3359 This is going to be feature that we'll need to support. The script that drops based on retention can't unarchive as it will take too long. Yes, but it's quite hard to keep it atomic (many changes in metastore and then archive deletion) and it will need refactoring and I don't want to make patch even bigger. When we use single archives everything should be ok, so we don't drop backward compatibility. - Marcin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/#review1278 --- On 2011-08-02 21:34:07, Marcin Kurczych wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-02 21:34:07) Review request for hive, Paul Yang and namit jain. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs - trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION
Re: Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- (Updated 2011-08-04 03:02:47.340003) Review request for hive, Paul Yang and namit jain. Changes --- * applied suggestions from the reviewers * pushing HAR URI through Path class to get consistent handling of ending slashes in location (problem appeared in the migration scripts) Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs (updated) - trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION
Review Request: Support archiving for multiple partitions if the table is partitioned by multiple columns
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1259/ --- Review request for hive, Paul Yang and namit jain. Summary --- Allowing archiving at chosen level. When table is partitioned by ds, hr, min it allows archiving at ds level, hr level and min level. Corresponding syntaxes are: ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11'); ALTER TABLE test ARCHIVE PARTITION (ds='2008-04-08', hr='11', min='30'); You cannot do much to archived partitions. You can read them. You cannot write to them / overwrite them. You can drop single archived partitions, but not parts of bigger archives. Diffs - trunk/data/conf/hive-site.xml 1153271 trunk/metastore/if/hive_metastore.thrift 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.h 1153271 trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_constants.cpp 1153271 trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java 1153271 trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php 1153271 trunk/metastore/src/gen/thrift/gen-py/hive_metastore/constants.py 1153271 trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_constants.rb 1153271 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MetaUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/UpdateArchivedCounter.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/DummyPartition.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1153271 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1153271 trunk/ql/src/test/queries/clientnegative/archive_insert1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_insert4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi3.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi4.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi5.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi6.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_multi7.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec1.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec2.q PRE-CREATION trunk/ql/src/test/queries/clientnegative/archive_partspec3.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_corrupt.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/archive_multi.q PRE-CREATION trunk/ql/src/test/results/clientnegative/archive1.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive2.q.out 1153271 trunk/ql/src/test/results/clientnegative/archive_insert1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_insert4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi3.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi4.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi5.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi6.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_multi7.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec1.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec2.q.out PRE-CREATION trunk/ql/src/test/results/clientnegative/archive_partspec3.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/archive_corrupt.q.out PRE-CREATION
