-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7893/
-----------------------------------------------------------
(Updated Nov. 6, 2012, 8:16 a.m.)
Review request for hcatalog and Francis Liu.
Changes
-------
Added recursive check in the applyGroupAndPerms() call, it was dropped by
mistake.
Description
-------
[HCATALOG-538] HCatalogStorer fails for 100GB of data with dynamic partitioning
(number of partition is 300)
This addresses bug HCATALOG-538.
https://issues.apache.org/jira/browse/HCATALOG-538
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/shims/src/20/java/org/apache/hcatalog/shims/HCatHadoopShims20S.java
1406021
http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/shims/src/23/java/org/apache/hcatalog/shims/HCatHadoopShims23.java
1406021
http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
1406021
http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/src/java/org/apache/hcatalog/shims/HCatHadoopShims.java
1406021
Diff: https://reviews.apache.org/r/7893/diff/
Testing
-------
branch-0.4|? grep 'Tests run:' build/test/logs/*
build/test/logs/TEST-org.apache.hcatalog.cli.TestPermsGrp.txt:Tests run: 1,
Failures: 0, Errors: 0, Time elapsed: 13.719 sec
build/test/logs/TEST-org.apache.hcatalog.cli.TestSemanticAnalysis.txt:Tests
run: 22, Failures: 0, Errors: 0, Time elapsed: 9.209 sec
build/test/logs/TEST-org.apache.hcatalog.cli.TestUseDatabase.txt:Tests run: 1,
Failures: 0, Errors: 0, Time elapsed: 5.329 sec
build/test/logs/TEST-org.apache.hcatalog.common.TestHCatUtil.txt:Tests run: 3,
Failures: 0, Errors: 0, Time elapsed: 0.116 sec
build/test/logs/TEST-org.apache.hcatalog.common.TestHiveClientCache.txt:Tests
run: 6, Failures: 0, Errors: 0, Time elapsed: 35.862 sec
build/test/logs/TEST-org.apache.hcatalog.data.TestDefaultHCatRecord.txt:Tests
run: 3, Failures: 0, Errors: 0, Time elapsed: 0.095 sec
build/test/logs/TEST-org.apache.hcatalog.data.TestHCatRecordSerDe.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 0.134 sec
build/test/logs/TEST-org.apache.hcatalog.data.TestJsonSerDe.txt:Tests run: 2,
Failures: 0, Errors: 0, Time elapsed: 0.163 sec
build/test/logs/TEST-org.apache.hcatalog.data.TestLazyHCatRecord.txt:Tests run:
11, Failures: 0, Errors: 0, Time elapsed: 0.158 sec
build/test/logs/TEST-org.apache.hcatalog.data.TestReaderWriter.txt:Tests run:
1, Failures: 0, Errors: 0, Time elapsed: 4.544 sec
build/test/logs/TEST-org.apache.hcatalog.data.schema.TestHCatSchema.txt:Tests
run: 2, Failures: 0, Errors: 0, Time elapsed: 0.008 sec
build/test/logs/TEST-org.apache.hcatalog.data.schema.TestHCatSchemaUtils.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 0.031 sec
build/test/logs/TEST-org.apache.hcatalog.listener.TestMsgBusConnection.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 7.47 sec
build/test/logs/TEST-org.apache.hcatalog.listener.TestNotificationListener.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 6.009 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatDynamicPartitioned.txt:Tests
run: 2, Failures: 0, Errors: 0, Time elapsed: 116.238 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatHiveCompatibility.txt:Tests
run: 2, Failures: 0, Errors: 0, Time elapsed: 31.278 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatMultiOutputFormat.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 28.907 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatNonPartitioned.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 13.494 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatOutputFormat.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 4.649 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatPartitionPublish.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 53.465 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestHCatPartitioned.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 90.65 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestMultiOutputFormat.txt:Tests
run: 2, Failures: 0, Errors: 0, Time elapsed: 59.965 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestPassProperties.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 8.024 sec
build/test/logs/TEST-org.apache.hcatalog.mapreduce.TestSequenceFileReadWrite.txt:Tests
run: 4, Failures: 0, Errors: 0, Time elapsed: 51.635 sec
build/test/logs/TEST-org.apache.hcatalog.pig.TestHCatLoader.txt:Tests run: 5,
Failures: 0, Errors: 0, Time elapsed: 55.243 sec
build/test/logs/TEST-org.apache.hcatalog.pig.TestHCatLoaderComplexSchema.txt:Tests
run: 3, Failures: 0, Errors: 0, Time elapsed: 135.339 sec
build/test/logs/TEST-org.apache.hcatalog.pig.TestHCatStorer.txt:Tests run: 15,
Failures: 0, Errors: 0, Time elapsed: 121.176 sec
build/test/logs/TEST-org.apache.hcatalog.pig.TestHCatStorerMulti.txt:Tests run:
3, Failures: 0, Errors: 0, Time elapsed: 28.09 sec
build/test/logs/TEST-org.apache.hcatalog.rcfile.TestRCFileMapReduceInputFormat.txt:Tests
run: 1, Failures: 0, Errors: 0, Time elapsed: 0.172 sec
build/test/logs/TEST-org.apache.hcatalog.security.TestHdfsAuthorizationProvider.txt:Tests
run: 34, Failures: 0, Errors: 0, Time elapsed: 19.837 sec
Thanks,
Arup Malakar
