> On Feb. 6, 2018, 12:33 a.m., Ashutosh Chauhan wrote: > > standalone-metastore/src/main/thrift/hive_metastore.thrift > > Lines 332 (patched) > > <https://reviews.apache.org/r/65130/diff/10/?file=1952598#file1952598line332> > > > > 1. Seems like you want to use this to allow/disallow load statements > > for malformed names. But seems like its not used anywhere. > > 2. You also need to provide upgrade scripts for all metastore for this > > new field in metastore. > > 2. In any case, I don't see a reason to store this in metadata. This > > should rather be a config variable which user may toggle to force malformed > > load. > > 3. Name expertMode is not descriptive. Pick better name : > > allowIllegalLoadInBucketedTable? > > Deepak Jaiswal wrote: > This is something which Gopal suggested. It is not supposed to block > loads. This config is set to true when user uses "load data" in bucketing > strict mode set to false. The plumbing is not complete so this field is just > a placeholder for the moment. > The idea is to check this value when user complaints about wrong results > when using bucket based joins as load data without data check can potentially > ruin the bucketing scheme. > Agree with name, will think of something more appropriate.
bq. The plumbing is not complete so this field is just a placeholder for the moment. If its not complete please don't include it in this patch. Add it later. Also, you are changing metastore schema, you need to provide upgrade scripts for all RDBMS - Ashutosh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65130/#review196844 ----------------------------------------------------------- On Feb. 7, 2018, 6:04 p.m., Deepak Jaiswal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65130/ > ----------------------------------------------------------- > > (Updated Feb. 7, 2018, 6:04 p.m.) > > > Review request for hive, Ashutosh Chauhan, Eugene Koifman, Gopal V, Jason > Dere, Sergey Shelukhin, and Thejas Nair. > > > Bugs: HIVE-18350 > https://issues.apache.org/jira/browse/HIVE-18350 > > > Repository: hive-git > > > Description > ------- > > Made changes for both bucketed and non-bucketed tables. > Added a positive test for non-bucketed table which renames the loaded file. > Added couple of negative tests for bucketed table which reject a load with > inconsistent file name. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomPartitionVertex.java > 26afe90faa > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomVertexConfiguration.java > ef5e7edcd6 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 9885038588 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 632a21390d > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java > dc698c8de8 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/metainfo/annotation/OpTraitsRulesProcFactory.java > 69d9f3125a > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java > bacc44482a > ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java > 54f5bab6de > ql/src/java/org/apache/hadoop/hive/ql/plan/OpTraits.java 9621c3be53 > ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java b5b478fbad > ql/src/test/queries/clientpositive/auto_sortmerge_join_2.q e5fdcb57e4 > ql/src/test/queries/clientpositive/auto_sortmerge_join_4.q abf09e5534 > ql/src/test/queries/clientpositive/auto_sortmerge_join_5.q b85c4a7aa3 > ql/src/test/queries/clientpositive/auto_sortmerge_join_7.q bd780861e3 > ql/src/test/results/clientnegative/bucket_mapjoin_mismatch1.q.out > b9c2e6f827 > ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out 5cfc35aa73 > ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out 0d586fd26b > ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out 45704d1253 > ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out 1959075912 > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out > 054b0d00be > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out > 95d329862c > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out > e711715aa5 > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out > 53c685cb11 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_2.q.out > 8cfa113794 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_4.q.out > fce5e0cfc4 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_5.q.out > 8250eca099 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_7.q.out > eb813c1734 > standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp > af0fd6b0e0 > standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h > 4c09bc8fe6 > standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp > aadf8f17c4 > > standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BucketingVersion.java > PRE-CREATION > > standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java > a132e5e838 > standalone-metastore/src/gen/thrift/gen-php/metastore/Types.php a5b578ef37 > standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py > 5598859042 > standalone-metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb > bc58cfe0ef > > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java > d58ed677f3 > > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/model/MTable.java > a38a1250e0 > standalone-metastore/src/main/thrift/hive_metastore.thrift 371b97590c > > standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStore.java > bd61df654a > > standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestTablesCreateDropAlterTruncate.java > 00f38eeec5 > > > Diff: https://reviews.apache.org/r/65130/diff/13/ > > > Testing > ------- > > > Thanks, > > Deepak Jaiswal > >
