-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50586/
-----------------------------------------------------------
(Updated July 30, 2016, 8:22 p.m.)
Review request for hive.
Changes
-------
add new version of patch
Summary (updated)
-----------------
HIVE-14329 fix flapping qtests
Bugs: HIVE-14329
https://issues.apache.org/jira/browse/HIVE-14329
Repository: hive-git
Description (updated)
-------
fix hashmap order changes inside qtest outputs; most common is I think is
skewed key order differences.
main fix is some added sorting logic to MetaDataFormatUtils
i've seen some STATS_ACCURATE / COLUMN_STATS related order changes too; in that
class i've noticed some minor problems:
* fix for StatsSetupConst#setBasicStatsState: failed to upgrade&set basic stats
status
* removed: inconsitency: StatsSetupConst#setColumnStatsState would put
lowercase colnames if record is new; original colnames during update/addition
i've choosed to be keep the column names **case sensitive** - this is good?
* removed: upgrade-while-read from: StatsSetupConst#areBasicStatsUptoDate
even thru this may help avoiding STATS_ACCURATE related problems; it's not
bulletproof - jsonobject tries hard to keep the insertion order inside its
parts.
i'm not sure but: ...would it make sense to use jackson/ObjectMapper to parse
the params into a pojo; which then can be used more easily to read/write
colstats and other metadata like it more easily - i haven't looked into it; how
complicated would this be...
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 01e6010
common/src/test/org/apache/hadoop/hive/common/TestStatsSetupConst.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
03803bb
ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_1.q.out
00f3776
ql/src/test/results/clientpositive/create_alter_list_bucketing_table1.q.out
df4a75b
ql/src/test/results/clientpositive/create_skewed_table1.q.out fe5ea0f
ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out 8f40040
ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out
0acfe90
ql/src/test/results/clientpositive/extrapolate_part_stats_partial_ndv.q.out
70e4db3
ql/src/test/results/clientpositive/list_bucket_dml_10.q.out d4681b7
ql/src/test/results/clientpositive/list_bucket_dml_11.q.out ecf54a8
ql/src/test/results/clientpositive/list_bucket_dml_12.q.out 0e11f3f
ql/src/test/results/clientpositive/list_bucket_dml_13.q.out 93ebef0
ql/src/test/results/clientpositive/list_bucket_dml_2.q.out a29c224
ql/src/test/results/clientpositive/list_bucket_dml_4.q.out 5f0406a
ql/src/test/results/clientpositive/list_bucket_dml_5.q.out 09cb847
ql/src/test/results/clientpositive/list_bucket_dml_6.q.out e53fee7
ql/src/test/results/clientpositive/list_bucket_dml_8.q.out ee36d3f
ql/src/test/results/clientpositive/list_bucket_dml_9.q.out 81f3af3
ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out
492866a
ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out
cfb613a
ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out
6a5f8f5
ql/src/test/results/clientpositive/list_bucket_query_oneskew_3.q.out 0c6fb63
ql/src/test/results/clientpositive/spark/list_bucket_dml_10.q.out 9eca85a
ql/src/test/results/clientpositive/stats_list_bucket.q.out c66da97
Diff: https://reviews.apache.org/r/50586/diff/
Testing (updated)
-------
r2: added qtest changes to patch
Thanks,
Zoltan Haindrich
