-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59408/#review175622
-----------------------------------------------------------
Thanks for the patch Zsolt!
It is important to be able to check the data even if does not fit into memory!
Thanks for the catch!
As for the patch, few comments from YETUS:
- Checkstyle: I will not put all of them here, but there is a problem with the
indentation. Could you please check the checkstyle output of
mvn checkstyle:checkstyle -Dcheckstyle.consoleOutput=true -DskipTests
-Pitests -Ptest-patch
- ASF headers are missing:
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterable.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterator.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/EntityUpdater.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/IDataProvider.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntity.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementations.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationUpdater.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/RetrieverIterable.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/ReturnValue.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UpdateParams.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UriUpdateChecker.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterableTest.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/DataProviderStub.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/EntityUpdaterTest.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementationsTest.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationUpdaterTest.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/ReturnValueTest.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/UriUpdateCheckerTest.java
!?????
/Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/tools/HiveMetaToolTest.java
Thanks,
Peter
- Peter Vary
On May 19, 2017, 5:05 p.m., Zsolt Fekete wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59408/
> -----------------------------------------------------------
>
> (Updated May 19, 2017, 5:05 p.m.)
>
>
> Review request for hive.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Currently HiveMetaTool reads full tables (as DataNucleus entities) into
> memory by calling PersistenceManager's retrieveAll().
>
> See these methods of ObjectStore: updateMDatabaseURI, updateTblPropURI,
> updateMStorageDescriptorTblPropURI, updateMStorageDescriptorTblURI,
> updateSerdeURI.
>
> This might cause failure when the affected tables (SDS, DBS, TABLE_PARAMS,
> SD_PARAMS, SERDES, SERDE_PARAMS) are too big.
>
>
> Diffs
> -----
>
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
> b28983f
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterable.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterator.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/EntityUpdater.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/IDataProvider.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntity.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementations.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationUpdater.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/RetrieverIterable.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/ReturnValue.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UpdateParams.java
> PRE-CREATION
>
> metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UriUpdateChecker.java
> PRE-CREATION
> metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java
> 22e246f
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterableTest.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/DataProviderStub.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/EntityUpdaterTest.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementationsTest.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationUpdaterTest.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/ReturnValueTest.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/metatool/UriUpdateCheckerTest.java
> PRE-CREATION
>
> metastore/src/test/org/apache/hadoop/hive/metastore/tools/HiveMetaToolTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/59408/diff/2/
>
>
> Testing
> -------
>
> The new tests passed:
> mvn test
> -Dtest=BlockRetrieverIterableTest,EntityUpdaterTest,UriUpdateCheckerTest,LocationUpdaterTest,ReturnValueTest,LocationEntityImplementationsTest,IntegrationTest
>
>
> Thanks,
>
> Zsolt Fekete
>
>
