Author: gunther
Date: Wed Sep 24 02:38:04 2014
New Revision: 1627194
URL: http://svn.apache.org/r1627194
Log:
HIVE-8179: Fetch task conversion: Remove some dependencies on AST (Gunther
Hagleitner, reviewed by Ashutosh Chauhan)
Modified:
hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out
hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out
hive/trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java
hive/trunk/ql/src/test/queries/positive/udf6.q
hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_part.q.out
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_table.q.out
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_union.q.out
hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out
hive/trunk/ql/src/test/results/clientpositive/combine2.q.out
hive/trunk/ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out
hive/trunk/ql/src/test/results/clientpositive/explain_logical.q.out
hive/trunk/ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out
hive/trunk/ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out
hive/trunk/ql/src/test/results/clientpositive/filter_numeric.q.out
hive/trunk/ql/src/test/results/clientpositive/input4.q.out
hive/trunk/ql/src/test/results/clientpositive/keyword_1.q.out
hive/trunk/ql/src/test/results/clientpositive/limit_partition_metadataonly.q.out
hive/trunk/ql/src/test/results/clientpositive/nonmr_fetch.q.out
hive/trunk/ql/src/test/results/clientpositive/orc_create.q.out
hive/trunk/ql/src/test/results/clientpositive/parquet_create.q.out
hive/trunk/ql/src/test/results/clientpositive/partition_wise_fileformat2.q.out
hive/trunk/ql/src/test/results/clientpositive/ppr_pushdown3.q.out
hive/trunk/ql/src/test/results/clientpositive/quote1.q.out
hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out
hive/trunk/ql/src/test/results/clientpositive/regex_col.q.out
hive/trunk/ql/src/test/results/clientpositive/schemeAuthority.q.out
hive/trunk/ql/src/test/results/clientpositive/schemeAuthority2.q.out
hive/trunk/ql/src/test/results/clientpositive/select_dummy_source.q.out
hive/trunk/ql/src/test/results/clientpositive/serde_user_properties.q.out
hive/trunk/ql/src/test/results/clientpositive/subquery_alias.q.out
hive/trunk/ql/src/test/results/clientpositive/tez/select_dummy_source.q.out
hive/trunk/ql/src/test/results/clientpositive/udf5.q.out
hive/trunk/ql/src/test/results/clientpositive/udf6.q.out
hive/trunk/ql/src/test/results/clientpositive/udf_current_database.q.out
hive/trunk/ql/src/test/results/clientpositive/udf_reflect2.q.out
hive/trunk/ql/src/test/results/clientpositive/udf_to_unix_timestamp.q.out
hive/trunk/ql/src/test/results/compiler/parse/udf6.q.out
hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml
hive/trunk/service/src/test/org/apache/hive/service/cli/CLIServiceTest.java
hive/trunk/service/src/test/org/apache/hive/service/cli/thrift/ThriftCLIServiceTest.java
Modified:
hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out
(original)
+++ hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out
Wed Sep 24 02:38:04 2014
@@ -25,36 +25,24 @@ SELECT example_add(1, 2),
FROM src LIMIT 1
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 3 (type: int), 6 (type: int), 10 (type: int),
3.3000000000000003 (type: double), 6.6 (type: double), 11.0 (type: double),
10.4 (type: double)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5,
_col6
- Statistics: Num rows: 500 Data size: 22000 Basic stats: COMPLETE
Column stats: COMPLETE
- Limit
- Number of rows: 1
- Statistics: Num rows: 1 Data size: 44 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 44 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: 1
Processor Tree:
- ListSink
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE
Column stats: COMPLETE
+ Select Operator
+ expressions: 3 (type: int), 6 (type: int), 10 (type: int),
3.3000000000000003 (type: double), 6.6 (type: double), 11.0 (type: double),
10.4 (type: double)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
+ Statistics: Num rows: 500 Data size: 22000 Basic stats: COMPLETE
Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 44 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: SELECT example_add(1, 2),
example_add(1, 2, 3),
Modified:
hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out
(original)
+++ hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out
Wed Sep 24 02:38:04 2014
@@ -19,36 +19,24 @@ SELECT example_format("abc"),
FROM src LIMIT 1
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 'abc' (type: string), '1.1' (type: string), '1.1
1.200000e+00' (type: string), 'a 12 10' (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 500 Data size: 182500 Basic stats:
COMPLETE Column stats: COMPLETE
- Limit
- Number of rows: 1
- Statistics: Num rows: 1 Data size: 365 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 365 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: 1
Processor Tree:
- ListSink
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE
Column stats: COMPLETE
+ Select Operator
+ expressions: 'abc' (type: string), '1.1' (type: string), '1.1
1.200000e+00' (type: string), 'a 12 10' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 500 Data size: 182500 Basic stats: COMPLETE
Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 365 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: SELECT example_format("abc"),
example_format("%1$s", 1.1),
Modified:
hive/trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
URL:
http://svn.apache.org/viewvc/hive/trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
---
hive/trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
(original)
+++
hive/trunk/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
Wed Sep 24 02:38:04 2014
@@ -139,7 +139,7 @@ public class TestHiveHistory extends Tes
SessionState.start(ss);
- String cmd = "select a.key from src a";
+ String cmd = "select a.key+1 from src a";
Driver d = new Driver(conf);
int ret = d.run(cmd).getResponseCode();
if (ret != 0) {
Modified:
hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
---
hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
(original)
+++
hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
Wed Sep 24 02:38:04 2014
@@ -44,9 +44,9 @@ import org.apache.hadoop.hive.ql.exec.Ut
import org.apache.hadoop.hive.ql.hooks.ReadEntity;
import org.apache.hadoop.hive.ql.io.ContentSummaryInputFormat;
import org.apache.hadoop.hive.ql.io.HiveInputFormat;
-import org.apache.hadoop.hive.ql.metadata.InputEstimator;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveStorageHandler;
+import org.apache.hadoop.hive.ql.metadata.InputEstimator;
import org.apache.hadoop.hive.ql.metadata.Partition;
import org.apache.hadoop.hive.ql.metadata.Table;
import org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner;
@@ -55,13 +55,25 @@ import org.apache.hadoop.hive.ql.parse.P
import org.apache.hadoop.hive.ql.parse.QB;
import org.apache.hadoop.hive.ql.parse.SemanticException;
import org.apache.hadoop.hive.ql.parse.SplitSample;
+import org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc;
import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc;
import org.apache.hadoop.hive.ql.plan.FetchWork;
import org.apache.hadoop.hive.ql.plan.ListSinkDesc;
import org.apache.hadoop.hive.ql.plan.OperatorDesc;
import org.apache.hadoop.hive.ql.plan.PartitionDesc;
import org.apache.hadoop.hive.ql.plan.PlanUtils;
+import org.apache.hadoop.hive.ql.plan.SelectDesc;
import org.apache.hadoop.hive.ql.plan.TableDesc;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToBinary;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToChar;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDecimal;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUtcTimestamp;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFToVarchar;
import org.apache.hadoop.mapred.InputFormat;
import org.apache.hadoop.mapred.JobConf;
@@ -73,9 +85,11 @@ public class SimpleFetchOptimizer implem
private final Log LOG =
LogFactory.getLog(SimpleFetchOptimizer.class.getName());
+ @Override
public ParseContext transform(ParseContext pctx) throws SemanticException {
Map<String, Operator<? extends OperatorDesc>> topOps = pctx.getTopOps();
- if (pctx.getQB().isSimpleSelectQuery() && topOps.size() == 1) {
+ if (pctx.getQB().getIsQuery() &&
!pctx.getQB().getParseInfo().isAnalyzeCommand()
+ && topOps.size() == 1) {
// no join, no groupby, no distinct, no lateral view, no subq,
// no CTAS or insert, not analyze command, and single sourced.
String alias = (String) pctx.getTopOps().keySet().toArray()[0];
@@ -144,7 +158,7 @@ public class SimpleFetchOptimizer implem
// for non-aggressive mode (minimal)
// 1. samping is not allowed
// 2. for partitioned table, all filters should be targeted to partition
column
- // 3. SelectOperator should be select star
+ // 3. SelectOperator should use only simple cast/column access
private FetchData checkTree(boolean aggressive, ParseContext pctx, String
alias,
TableScanOperator ts) throws HiveException {
SplitSample splitSample = pctx.getNameToSplitSample().get(alias);
@@ -156,7 +170,7 @@ public class SimpleFetchOptimizer implem
return null;
}
- Table table = qb.getMetaData().getAliasToTable().get(alias);
+ Table table = pctx.getTopToTable().get(ts);
if (table == null) {
return null;
}
@@ -181,34 +195,71 @@ public class SimpleFetchOptimizer implem
return null;
}
- private FetchData checkOperators(FetchData fetch, TableScanOperator ts,
boolean aggresive,
+ private FetchData checkOperators(FetchData fetch, TableScanOperator ts,
boolean aggressive,
boolean bypassFilter) {
if (ts.getChildOperators().size() != 1) {
return null;
}
Operator<?> op = ts.getChildOperators().get(0);
for (; ; op = op.getChildOperators().get(0)) {
- if (aggresive) {
- if (!(op instanceof LimitOperator || op instanceof FilterOperator
- || op instanceof SelectOperator)) {
+ if (op instanceof SelectOperator) {
+ if (!aggressive) {
+ if (!checkExpressions((SelectOperator) op)) {
+ break;
+ }
+ }
+ continue;
+ }
+
+ if (aggressive) {
+ if (!(op instanceof LimitOperator || op instanceof FilterOperator)) {
break;
}
- } else if (!(op instanceof LimitOperator || (op instanceof
FilterOperator && bypassFilter)
- || (op instanceof SelectOperator && ((SelectOperator)
op).getConf().isSelectStar()))) {
+ } else if (!(op instanceof LimitOperator || (op instanceof
FilterOperator && bypassFilter))) {
break;
}
+
if (op.getChildOperators() == null || op.getChildOperators().size() !=
1) {
return null;
}
}
+
if (op instanceof FileSinkOperator) {
fetch.scanOp = ts;
fetch.fileSink = op;
return fetch;
}
+
return null;
}
+ private boolean checkExpressions(SelectOperator op) {
+ SelectDesc desc = op.getConf();
+ for (ExprNodeDesc expr : desc.getColList()) {
+ if (!checkExpression(expr)) {
+ return false;
+ }
+ }
+ return true;
+ }
+
+ private boolean checkExpression(ExprNodeDesc expr) {
+ if (expr instanceof ExprNodeConstantDesc || expr instanceof
ExprNodeColumnDesc) {
+ return true;
+ }
+
+ if (expr instanceof ExprNodeGenericFuncDesc) {
+ GenericUDF udf = ((ExprNodeGenericFuncDesc) expr).getGenericUDF();
+ if (udf instanceof GenericUDFToBinary || udf instanceof GenericUDFToChar
+ || udf instanceof GenericUDFToDate || udf instanceof
GenericUDFToDecimal
+ || udf instanceof GenericUDFToUnixTimeStamp || udf instanceof
GenericUDFToUtcTimestamp
+ || udf instanceof GenericUDFToVarchar) {
+ return expr.getChildren().size() == 1 &&
checkExpression(expr.getChildren().get(0));
+ }
+ }
+ return false;
+ }
+
private class FetchData {
private final ReadEntity parent;
@@ -240,7 +291,7 @@ public class SimpleFetchOptimizer implem
this.splitSample = splitSample;
this.onlyPruningFilter = bypassFilter;
}
-
+
/*
* all filters were executed during partition pruning
*/
@@ -251,7 +302,7 @@ public class SimpleFetchOptimizer implem
private FetchWork convertToWork() throws HiveException {
inputs.clear();
if (!table.isPartitioned()) {
- inputs.add(new ReadEntity(table, parent));
+ inputs.add(new ReadEntity(table, parent, parent == null));
FetchWork work = new FetchWork(table.getPath(),
Utilities.getTableDesc(table));
PlanUtils.configureInputJobPropertiesForStorageHandler(work.getTblDesc());
work.setSplitSample(splitSample);
@@ -261,12 +312,12 @@ public class SimpleFetchOptimizer implem
List<PartitionDesc> partP = new ArrayList<PartitionDesc>();
for (Partition partition : partsList.getNotDeniedPartns()) {
- inputs.add(new ReadEntity(partition, parent));
+ inputs.add(new ReadEntity(partition, parent, parent == null));
listP.add(partition.getDataLocation());
partP.add(Utilities.getPartitionDesc(partition));
}
Table sourceTable = partsList.getSourceTable();
- inputs.add(new ReadEntity(sourceTable, parent));
+ inputs.add(new ReadEntity(sourceTable, parent, parent == null));
TableDesc table = Utilities.getTableDesc(sourceTable);
FetchWork work = new FetchWork(listP, partP, table);
if (!work.getPartDesc().isEmpty()) {
Modified:
hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
---
hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java
(original)
+++
hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java
Wed Sep 24 02:38:04 2014
@@ -165,7 +165,7 @@ public class TestSymlinkTextInputFormat
+ " failed with exit code= " + ecode);
}
- String cmd = "select key from " + tblName;
+ String cmd = "select key*1 from " + tblName;
drv.compile(cmd);
//create scratch dir
Modified: hive/trunk/ql/src/test/queries/positive/udf6.q
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/queries/positive/udf6.q?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/queries/positive/udf6.q (original)
+++ hive/trunk/ql/src/test/queries/positive/udf6.q Wed Sep 24 02:38:04 2014
@@ -1 +1 @@
-FROM src SELECT CONCAT('a', 'b'), IF(TRUE, 1 ,2)
+FROM src SELECT CONCAT('a', 'b'), IF(TRUE, 1 ,2) + key
Modified:
hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
Files
hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out
(original) and
hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out Wed
Sep 24 02:38:04 2014 differ
Modified:
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_part.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/annotate_stats_part.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/annotate_stats_part.q.out
(original)
+++ hive/trunk/ql/src/test/results/clientpositive/annotate_stats_part.q.out Wed
Sep 24 02:38:04 2014
@@ -322,33 +322,21 @@ POSTHOOK: query: -- basicStatState: COMP
explain select zip from loc_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: loc_orc
- Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE
Column stats: NONE
- Select Operator
- expressions: zip (type: bigint)
- outputColumnNames: _col0
- Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: loc_orc
+ Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE Column
stats: NONE
+ Select Operator
+ expressions: zip (type: bigint)
+ outputColumnNames: _col0
+ Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE
Column stats: NONE
+ ListSink
PREHOOK: query: -- basicStatState: COMPLETE colStatState: PARTIAL
explain select state from loc_orc
@@ -357,33 +345,21 @@ POSTHOOK: query: -- basicStatState: COMP
explain select state from loc_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: loc_orc
- Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE
Column stats: PARTIAL
- Select Operator
- expressions: state (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 8 Data size: 688 Basic stats: COMPLETE
Column stats: PARTIAL
- File Output Operator
- compressed: false
- Statistics: Num rows: 8 Data size: 688 Basic stats: COMPLETE
Column stats: PARTIAL
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: loc_orc
+ Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE Column
stats: PARTIAL
+ Select Operator
+ expressions: state (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 8 Data size: 688 Basic stats: COMPLETE
Column stats: PARTIAL
+ ListSink
PREHOOK: query: -- column statistics for __HIVE_DEFAULT_PARTITION__ is not
supported yet. Hence colStatState reports PARTIAL
-- basicStatState: COMPLETE colStatState: PARTIAL
@@ -394,33 +370,21 @@ POSTHOOK: query: -- column statistics fo
explain select state,locid from loc_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: loc_orc
- Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE
Column stats: PARTIAL
- Select Operator
- expressions: state (type: string), locid (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 8 Data size: 720 Basic stats: COMPLETE
Column stats: PARTIAL
- File Output Operator
- compressed: false
- Statistics: Num rows: 8 Data size: 720 Basic stats: COMPLETE
Column stats: PARTIAL
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: loc_orc
+ Statistics: Num rows: 8 Data size: 722 Basic stats: COMPLETE Column
stats: PARTIAL
+ Select Operator
+ expressions: state (type: string), locid (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 8 Data size: 720 Basic stats: COMPLETE
Column stats: PARTIAL
+ ListSink
PREHOOK: query: -- basicStatState: COMPLETE colStatState: COMPLETE
explain select state,locid from loc_orc where year='2001'
@@ -429,33 +393,21 @@ POSTHOOK: query: -- basicStatState: COMP
explain select state,locid from loc_orc where year='2001'
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: loc_orc
- Statistics: Num rows: 7 Data size: 399 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: state (type: string), locid (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 7 Data size: 630 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 7 Data size: 630 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: loc_orc
+ Statistics: Num rows: 7 Data size: 399 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: state (type: string), locid (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 7 Data size: 630 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- basicStatState: COMPLETE colStatState: NONE
explain select state,locid from loc_orc where year!='2001'
@@ -464,33 +416,21 @@ POSTHOOK: query: -- basicStatState: COMP
explain select state,locid from loc_orc where year!='2001'
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: loc_orc
- Statistics: Num rows: 1 Data size: 323 Basic stats: COMPLETE
Column stats: NONE
- Select Operator
- expressions: state (type: string), locid (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 323 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 323 Basic stats: COMPLETE
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: loc_orc
+ Statistics: Num rows: 1 Data size: 323 Basic stats: COMPLETE Column
stats: NONE
+ Select Operator
+ expressions: state (type: string), locid (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 323 Basic stats: COMPLETE
Column stats: NONE
+ ListSink
PREHOOK: query: -- basicStatState: COMPLETE colStatState: PARTIAL
explain select * from loc_orc
Modified:
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out
(original)
+++ hive/trunk/ql/src/test/results/clientpositive/annotate_stats_select.q.out
Wed Sep 24 02:38:04 2014
@@ -152,33 +152,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select bo1 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: bo1 (type: boolean)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: bo1 (type: boolean)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- col alias renaming
-- numRows: 2 rawDataSize: 8
@@ -189,33 +177,21 @@ POSTHOOK: query: -- col alias renaming
explain select i1 as int1 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: i1 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: i1 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 174
explain select s1 from alltypes_orc
@@ -224,33 +200,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select s1 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: s1 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: s1 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- column statistics for complex types unsupported and so
statistics will not be updated
-- numRows: 2 rawDataSize: 1514
@@ -261,33 +225,21 @@ POSTHOOK: query: -- column statistics fo
explain select m1 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: NONE
- Select Operator
- expressions: m1 (type: map<string,string>)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: NONE
+ Select Operator
+ expressions: m1 (type: map<string,string>)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: NONE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 246
explain select bo1, ti1, si1, i1, bi1, f1, d1,s1 from alltypes_orc
@@ -296,33 +248,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select bo1, ti1, si1, i1, bi1, f1, d1,s1 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: bo1 (type: boolean), ti1 (type: tinyint), si1
(type: smallint), i1 (type: int), bi1 (type: bigint), f1 (type: float), d1
(type: double), s1 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5,
_col6, _col7
- Statistics: Num rows: 2 Data size: 246 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 246 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: bo1 (type: boolean), ti1 (type: tinyint), si1 (type:
smallint), i1 (type: int), bi1 (type: bigint), f1 (type: float), d1 (type:
double), s1 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5,
_col6, _col7
+ Statistics: Num rows: 2 Data size: 246 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 0
explain select null from alltypes_orc
@@ -331,33 +271,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select null from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: null (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL Column
stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: null (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 8
explain select 11 from alltypes_orc
@@ -366,33 +294,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select 11 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 11 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: 11 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 16
explain select 11L from alltypes_orc
@@ -401,33 +317,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select 11L from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 11 (type: bigint)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: 11 (type: bigint)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 16
explain select 11.0 from alltypes_orc
@@ -436,33 +340,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select 11.0 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 11.0 (type: double)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: 11.0 (type: double)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 178
explain select "hello" from alltypes_orc
@@ -471,99 +363,63 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select "hello" from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 'hello' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: 'hello' (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: explain select cast("hello" as char(5)) from alltypes_orc
PREHOOK: type: QUERY
POSTHOOK: query: explain select cast("hello" as char(5)) from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: CAST( 'hello' AS CHAR(5) (type: char(5))
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: CAST( 'hello' AS CHAR(5) (type: char(5))
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: explain select cast("hello" as varchar(5)) from alltypes_orc
PREHOOK: type: QUERY
POSTHOOK: query: explain select cast("hello" as varchar(5)) from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: CAST( 'hello' AS varchar(5)) (type: varchar(5))
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: CAST( 'hello' AS varchar(5)) (type: varchar(5))
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 178 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 96
explain select unbase64("0xe23") from alltypes_orc
@@ -572,33 +428,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select unbase64("0xe23") from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: D317B6 (type: binary)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 96 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 96 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: D317B6 (type: binary)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 96 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 16
explain select cast("1" as TINYINT), cast("20" as SMALLINT) from alltypes_orc
@@ -607,33 +451,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select cast("1" as TINYINT), cast("20" as SMALLINT) from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 1 (type: tinyint), 20 (type: smallint)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: 1 (type: tinyint), 20 (type: smallint)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 80
explain select cast("1970-12-31 15:59:58.174" as TIMESTAMP) from alltypes_orc
@@ -642,33 +474,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select cast("1970-12-31 15:59:58.174" as TIMESTAMP) from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: 1970-12-31 15:59:58.174 (type: timestamp)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 80 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 80 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: 1970-12-31 15:59:58.174 (type: timestamp)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 80 Basic stats: COMPLETE Column
stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 112
explain select cast("1970-12-31 15:59:58.174" as DATE) from alltypes_orc
@@ -712,33 +532,21 @@ POSTHOOK: query: -- numRows: 2 rawDataSi
explain select cast("58.174" as DECIMAL) from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: CAST( '58.174' AS decimal(10,0)) (type:
decimal(10,0))
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 224 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 224 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: CAST( '58.174' AS decimal(10,0)) (type: decimal(10,0))
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 224 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 2 rawDataSize: 112
explain select array(1,2,3) from alltypes_orc
@@ -997,33 +805,21 @@ POSTHOOK: query: -- column statistics fo
explain select *,11 from alltypes_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: alltypes_orc
- Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE
Column stats: PARTIAL
- Select Operator
- expressions: bo1 (type: boolean), ti1 (type: tinyint), si1
(type: smallint), i1 (type: int), bi1 (type: bigint), f1 (type: float), d1
(type: double), de1 (type: decimal(10,0)), ts1 (type: timestamp), da1 (type:
timestamp), s1 (type: string), vc1 (type: varchar(5)), m1 (type:
map<string,string>), l1 (type: array<int>), st1 (type:
struct<c1:int,c2:string>), 11 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5,
_col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15
- Statistics: Num rows: 2 Data size: 428 Basic stats: COMPLETE
Column stats: PARTIAL
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 428 Basic stats: COMPLETE
Column stats: PARTIAL
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: alltypes_orc
+ Statistics: Num rows: 2 Data size: 1686 Basic stats: COMPLETE Column
stats: PARTIAL
+ Select Operator
+ expressions: bo1 (type: boolean), ti1 (type: tinyint), si1 (type:
smallint), i1 (type: int), bi1 (type: bigint), f1 (type: float), d1 (type:
double), de1 (type: decimal(10,0)), ts1 (type: timestamp), da1 (type:
timestamp), s1 (type: string), vc1 (type: varchar(5)), m1 (type:
map<string,string>), l1 (type: array<int>), st1 (type:
struct<c1:int,c2:string>), 11 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5,
_col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15
+ Statistics: Num rows: 2 Data size: 428 Basic stats: COMPLETE
Column stats: PARTIAL
+ ListSink
PREHOOK: query: -- subquery selects
-- inner select - numRows: 2 rawDataSize: 8
Modified:
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_table.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/annotate_stats_table.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/annotate_stats_table.q.out
(original)
+++ hive/trunk/ql/src/test/results/clientpositive/annotate_stats_table.q.out
Wed Sep 24 02:38:04 2014
@@ -171,33 +171,21 @@ POSTHOOK: query: -- all selected columns
explain select deptid from emp_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: emp_orc
- Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: deptid (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 48 Data size: 192 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 48 Data size: 192 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: emp_orc
+ Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: deptid (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 48 Data size: 192 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- column level complete statistics
analyze table emp_orc compute statistics for columns lastname,deptid
@@ -239,33 +227,21 @@ POSTHOOK: query: -- basicStatState: COMP
explain select lastname from emp_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: emp_orc
- Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: lastname (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 48 Data size: 4368 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 48 Data size: 4368 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: emp_orc
+ Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: lastname (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 48 Data size: 4368 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- basicStatState: COMPLETE colStatState: COMPLETE
explain select deptid from emp_orc
@@ -274,33 +250,21 @@ POSTHOOK: query: -- basicStatState: COMP
explain select deptid from emp_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: emp_orc
- Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: deptid (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 48 Data size: 192 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 48 Data size: 192 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: emp_orc
+ Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: deptid (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 48 Data size: 192 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- basicStatState: COMPLETE colStatState: COMPLETE
explain select lastname,deptid from emp_orc
@@ -309,31 +273,19 @@ POSTHOOK: query: -- basicStatState: COMP
explain select lastname,deptid from emp_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: emp_orc
- Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: lastname (type: string), deptid (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 48 Data size: 4560 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 48 Data size: 4560 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: emp_orc
+ Statistics: Num rows: 48 Data size: 364 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: lastname (type: string), deptid (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 48 Data size: 4560 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
Modified:
hive/trunk/ql/src/test/results/clientpositive/annotate_stats_union.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/annotate_stats_union.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/annotate_stats_union.q.out
(original)
+++ hive/trunk/ql/src/test/results/clientpositive/annotate_stats_union.q.out
Wed Sep 24 02:38:04 2014
@@ -67,33 +67,21 @@ POSTHOOK: query: -- numRows: 8 rawDataSi
explain select state from loc_orc
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: loc_orc
- Statistics: Num rows: 8 Data size: 796 Basic stats: COMPLETE
Column stats: COMPLETE
- Select Operator
- expressions: state (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 8 Data size: 688 Basic stats: COMPLETE
Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 8 Data size: 688 Basic stats: COMPLETE
Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: loc_orc
+ Statistics: Num rows: 8 Data size: 796 Basic stats: COMPLETE Column
stats: COMPLETE
+ Select Operator
+ expressions: state (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 8 Data size: 688 Basic stats: COMPLETE
Column stats: COMPLETE
+ ListSink
PREHOOK: query: -- numRows: 16 rawDataSize: 1376
explain select * from (select state from loc_orc union all select state from
loc_orc) tmp
Modified:
hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out
(original)
+++ hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out Wed
Sep 24 02:38:04 2014
@@ -83,33 +83,21 @@ PREHOOK: query: -- More complicated sele
EXPLAIN SELECT key FROM (SELECT key, val FROM T1) subq1
PREHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: t1
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column
stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL
Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: t1
+ Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column
stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column
stats: NONE
+ ListSink
PREHOOK: query: SELECT key FROM (SELECT key, val FROM T1) subq1
PREHOOK: type: QUERY
@@ -127,33 +115,21 @@ Columns:key
PREHOOK: query: EXPLAIN SELECT k FROM (SELECT key as k, val as v FROM T1) subq1
PREHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: t1
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column
stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL
Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: t1
+ Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column
stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column
stats: NONE
+ ListSink
PREHOOK: query: SELECT k FROM (SELECT key as k, val as v FROM T1) subq1
PREHOOK: type: QUERY
Modified: hive/trunk/ql/src/test/results/clientpositive/combine2.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/combine2.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/combine2.q.out (original)
+++ hive/trunk/ql/src/test/results/clientpositive/combine2.q.out Wed Sep 24
02:38:04 2014
@@ -94,33 +94,21 @@ POSTHOOK: query: explain
select key, value from combine2 where value is not null
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: combine2
- Statistics: Num rows: 12 Data size: 14 Basic stats: COMPLETE
Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 12 Data size: 14 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 12 Data size: 14 Basic stats: COMPLETE
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: combine2
+ Statistics: Num rows: 12 Data size: 14 Basic stats: COMPLETE Column
stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 12 Data size: 14 Basic stats: COMPLETE
Column stats: NONE
+ ListSink
PREHOOK: query: select key, value from combine2 where value is not null
PREHOOK: type: QUERY
Modified:
hive/trunk/ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
---
hive/trunk/ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out
(original)
+++
hive/trunk/ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out
Wed Sep 24 02:38:04 2014
@@ -74,48 +74,14 @@ TOK_QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: dynamic_part_table
- Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column
stats: NONE
- GatherStats: false
- Select Operator
- expressions: intcol (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- GlobalTableId: 0
-#### A masked pattern was here ####
- NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE
Column stats: NONE
-#### A masked pattern was here ####
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- columns _col0
- columns.types string
- escape.delim \
- hive.serialization.extend.nesting.levels true
- serialization.format 1
- serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- TotalFiles: 1
- GatherStats: false
- MultiFileSpray: false
- Path -> Alias:
-#### A masked pattern was here ####
- Path -> Partition:
-#### A masked pattern was here ####
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Partition Description:
Partition
- base file name: partcol2=1
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
@@ -159,14 +125,16 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dynamic_part_table
name: default.dynamic_part_table
- Truncated Path -> Alias:
- /dynamic_part_table/partcol1=1/partcol2=1 [dynamic_part_table]
-
- Stage: Stage-0
- Fetch Operator
- limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: dynamic_part_table
+ Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column
stats: NONE
+ GatherStats: false
+ Select Operator
+ expressions: intcol (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column
stats: NONE
+ ListSink
PREHOOK: query: explain extended select intcol from dynamic_part_table where
partcol1='1' and partcol2='1'
PREHOOK: type: QUERY
@@ -200,48 +168,14 @@ TOK_QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: dynamic_part_table
- Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column
stats: NONE
- GatherStats: false
- Select Operator
- expressions: intcol (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- GlobalTableId: 0
-#### A masked pattern was here ####
- NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE
Column stats: NONE
-#### A masked pattern was here ####
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- columns _col0
- columns.types string
- escape.delim \
- hive.serialization.extend.nesting.levels true
- serialization.format 1
- serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- TotalFiles: 1
- GatherStats: false
- MultiFileSpray: false
- Path -> Alias:
-#### A masked pattern was here ####
- Path -> Partition:
-#### A masked pattern was here ####
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Partition Description:
Partition
- base file name: partcol2=1
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
@@ -285,14 +219,16 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dynamic_part_table
name: default.dynamic_part_table
- Truncated Path -> Alias:
- /dynamic_part_table/partcol1=1/partcol2=1 [dynamic_part_table]
-
- Stage: Stage-0
- Fetch Operator
- limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: dynamic_part_table
+ Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column
stats: NONE
+ GatherStats: false
+ Select Operator
+ expressions: intcol (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column
stats: NONE
+ ListSink
PREHOOK: query: explain extended select intcol from dynamic_part_table where
(partcol1='1' and partcol2='1')or (partcol1='1' and
partcol2='__HIVE_DEFAULT_PARTITION__')
PREHOOK: type: QUERY
@@ -336,48 +272,14 @@ TOK_QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: dynamic_part_table
- Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column
stats: NONE
- GatherStats: false
- Select Operator
- expressions: intcol (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE
Column stats: NONE
- File Output Operator
- compressed: false
- GlobalTableId: 0
-#### A masked pattern was here ####
- NumFilesPerFileSink: 1
- Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE
Column stats: NONE
-#### A masked pattern was here ####
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- columns _col0
- columns.types string
- escape.delim \
- hive.serialization.extend.nesting.levels true
- serialization.format 1
- serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- TotalFiles: 1
- GatherStats: false
- MultiFileSpray: false
- Path -> Alias:
-#### A masked pattern was here ####
- Path -> Partition:
-#### A masked pattern was here ####
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Partition Description:
Partition
- base file name: partcol2=1
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
@@ -421,9 +323,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dynamic_part_table
name: default.dynamic_part_table
-#### A masked pattern was here ####
Partition
- base file name: partcol2=__HIVE_DEFAULT_PARTITION__
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
@@ -467,13 +367,14 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dynamic_part_table
name: default.dynamic_part_table
- Truncated Path -> Alias:
- /dynamic_part_table/partcol1=1/partcol2=1 [dynamic_part_table]
- /dynamic_part_table/partcol1=1/partcol2=__HIVE_DEFAULT_PARTITION__
[dynamic_part_table]
-
- Stage: Stage-0
- Fetch Operator
- limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: dynamic_part_table
+ Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column
stats: NONE
+ GatherStats: false
+ Select Operator
+ expressions: intcol (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column
stats: NONE
+ ListSink
Modified: hive/trunk/ql/src/test/results/clientpositive/explain_logical.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/explain_logical.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
--- hive/trunk/ql/src/test/results/clientpositive/explain_logical.q.out
(original)
+++ hive/trunk/ql/src/test/results/clientpositive/explain_logical.q.out Wed Sep
24 02:38:04 2014
@@ -453,13 +453,7 @@ v1:src
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column
stats: NONE
- File Output Operator (FS_3)
- compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column
stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ ListSink (OP_4)
PREHOOK: query: EXPLAIN LOGICAL SELECT * FROM V2
PREHOOK: type: QUERY
@@ -490,13 +484,7 @@ v2:srcpart
expressions: ds (type: string), key (type: string), value (type: string)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column
stats: NONE
- File Output Operator (FS_4)
- compressed: false
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE
Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ ListSink (OP_6)
PREHOOK: query: EXPLAIN LOGICAL SELECT * FROM V3
PREHOOK: type: QUERY
@@ -731,13 +719,7 @@ v5:srcpart
expressions: key (type: string), value (type: string), '10' (type:
string), hr (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats:
NONE
- File Output Operator (FS_4)
- compressed: false
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats:
NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ ListSink (OP_6)
PREHOOK: query: EXPLAIN LOGICAL SELECT s1.key, s1.cnt, s2.value FROM (SELECT
key, count(value) as cnt FROM src GROUP BY key) s1 JOIN src s2 ON (s1.key =
s2.key) ORDER BY s1.key
PREHOOK: type: QUERY
Modified:
hive/trunk/ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out
URL:
http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out?rev=1627194&r1=1627193&r2=1627194&view=diff
==============================================================================
Files
hive/trunk/ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out
(original) and
hive/trunk/ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out
Wed Sep 24 02:38:04 2014 differ
