hive git commit: HIVE-12545: Add sessionId and queryId logging support for methods like getCatalogs in HiveSessionImpl class (Aihua Xu, reviewed by Yongzhi Chen)

2016-01-29 Thread aihuaxu 
Repository: hive
Updated Branches:
  refs/heads/master 7b2f6703f -> 41fc1874b


HIVE-12545: Add sessionId and queryId logging support for methods like 
getCatalogs in HiveSessionImpl class (Aihua Xu, reviewed by Yongzhi Chen)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/41fc1874
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/41fc1874
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/41fc1874

Branch: refs/heads/master
Commit: 41fc1874bcabed3fae32ed5fcf6fa13d25811261
Parents: 7b2f670
Author: Aihua Xu 
Authored: Fri Jan 22 15:37:37 2016 -0500
Committer: Aihua Xu 
Committed: Fri Jan 29 10:35:25 2016 -0500

--
 .../src/java/org/apache/hive/service/cli/operation/Operation.java | 3 +++
 1 file changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/41fc1874/service/src/java/org/apache/hive/service/cli/operation/Operation.java
--
diff --git 
a/service/src/java/org/apache/hive/service/cli/operation/Operation.java 
b/service/src/java/org/apache/hive/service/cli/operation/Operation.java
index 113eddf..d6ac16d 100644
--- a/service/src/java/org/apache/hive/service/cli/operation/Operation.java
+++ b/service/src/java/org/apache/hive/service/cli/operation/Operation.java
@@ -32,6 +32,7 @@ import 
org.apache.hadoop.hive.common.metrics.common.MetricsConstant;
 import org.apache.hadoop.hive.common.metrics.common.MetricsFactory;
 import org.apache.hadoop.hive.common.metrics.common.MetricsScope;
 import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.ql.QueryPlan;
 import org.apache.hadoop.hive.ql.processors.CommandProcessorResponse;
 import org.apache.hadoop.hive.ql.session.OperationLog;
 import org.apache.hadoop.hive.ql.session.SessionState;
@@ -80,6 +81,8 @@ public abstract class Operation {
 
   protected Operation(HiveSession parentSession, OperationType opType, boolean 
runInBackground) {
 this(parentSession, null, opType, runInBackground);
+// Generate a queryId for the operation if no queryId is provided
+confOverlay.put(HiveConf.ConfVars.HIVEQUERYID.varname, 
QueryPlan.makeQueryId());
  }
 
   protected Operation(HiveSession parentSession, Map 
confOverlay, OperationType opType, boolean runInBackground) {

hive git commit: HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now (Sergey Shelukhin, reviewed by Xuefu Zhang) ADDENDUM2

2016-01-29 Thread sershe 
Repository: hive
Updated Branches:
  refs/heads/master cc32d881a -> 208ab3523


HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no 
limit and no partition filter by default for now (Sergey Shelukhin, reviewed by 
Xuefu Zhang) ADDENDUM2


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/208ab352
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/208ab352
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/208ab352

Branch: refs/heads/master
Commit: 208ab352311a6cbbcd1f7fcd40964da2dbc6703d
Parents: cc32d88
Author: Sergey Shelukhin 
Authored: Fri Jan 29 18:14:14 2016 -0800
Committer: Sergey Shelukhin 
Committed: Fri Jan 29 18:14:34 2016 -0800

--
 ql/src/test/queries/clientpositive/perf/query12.q | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/208ab352/ql/src/test/queries/clientpositive/perf/query12.q
--
diff --git a/ql/src/test/queries/clientpositive/perf/query12.q 
b/ql/src/test/queries/clientpositive/perf/query12.q
index 29976fc..034fa7b 100644
--- a/ql/src/test/queries/clientpositive/perf/query12.q
+++ b/ql/src/test/queries/clientpositive/perf/query12.q
@@ -1,2 +1,3 @@
 set hive.cbo.enable=false;
 explain select i_item_desc ,i_category ,i_class ,i_current_price ,i_item_id 
,sum(ws_ext_sales_price) as itemrevenue 
,sum(ws_ext_sales_price)*100/sum(sum(ws_ext_sales_price)) over (partition by 
i_class) as revenueratio from web_sales ,item ,date_dim where 
web_sales.ws_item_sk = item.i_item_sk and item.i_category in ('Jewelry', 
'Sports', 'Books') and web_sales.ws_sold_date_sk = date_dim.d_date_sk and 
date_dim.d_date between '2001-01-12' and '2001-02-11' group by i_item_id 
,i_item_desc ,i_category ,i_class ,i_current_price order by i_category ,i_class 
,i_item_id ,i_item_desc ,revenueratio limit 100;
+set hive.cbo.enable=true;

hive git commit: HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now (Sergey Shelukhin, reviewed by Xuefu Zhang) ADDENDUM

2016-01-29 Thread sershe 
Repository: hive
Updated Branches:
  refs/heads/master 1ab7fe145 -> cc32d881a


HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no 
limit and no partition filter by default for now (Sergey Shelukhin, reviewed by 
Xuefu Zhang) ADDENDUM


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/cc32d881
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/cc32d881
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/cc32d881

Branch: refs/heads/master
Commit: cc32d881a90f5b51e3f10b42e86261c66ace4520
Parents: 1ab7fe1
Author: Sergey Shelukhin 
Authored: Fri Jan 29 13:39:17 2016 -0800
Committer: Sergey Shelukhin 
Committed: Fri Jan 29 13:39:17 2016 -0800

--
 ql/src/test/results/clientpositive/empty_join.q.out  | 8 
 .../results/clientpositive/join_cond_pushdown_unqual5.q.out  | 4 ++--
 .../test/results/clientpositive/spark/smb_mapjoin_18.q.out   | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/cc32d881/ql/src/test/results/clientpositive/empty_join.q.out
--
diff --git a/ql/src/test/results/clientpositive/empty_join.q.out 
b/ql/src/test/results/clientpositive/empty_join.q.out
index 2abba0e..99e8f94 100644
--- a/ql/src/test/results/clientpositive/empty_join.q.out
+++ b/ql/src/test/results/clientpositive/empty_join.q.out
@@ -59,14 +59,14 @@ STAGE PLANS:
   Stage: Stage-5
 Map Reduce Local Work
   Alias -> Map Local Tables:
-$hdt$_1:t2
+$hdt$_1:t2 
   Fetch Operator
 limit: -1
-$hdt$_2:t3
+$hdt$_2:t3 
   Fetch Operator
 limit: -1
   Alias -> Map Local Operator Tree:
-$hdt$_1:t2
+$hdt$_1:t2 
   TableScan
 alias: t2
 Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column 
stats: NONE
@@ -82,7 +82,7 @@ STAGE PLANS:
 0 _col0 (type: int)
 1 _col0 (type: int)
 2 _col0 (type: int)
-$hdt$_2:t3
+$hdt$_2:t3 
   TableScan
 alias: t3
 Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column 
stats: NONE

http://git-wip-us.apache.org/repos/asf/hive/blob/cc32d881/ql/src/test/results/clientpositive/join_cond_pushdown_unqual5.q.out
--
diff --git 
a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual5.q.out 
b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual5.q.out
index 7d4fa71..aea9822 100644
--- a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual5.q.out
+++ b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual5.q.out
@@ -127,11 +127,11 @@ STAGE PLANS:
   Stage: Stage-4
 Map Reduce Local Work
   Alias -> Map Local Tables:
-$hdt$_1:r
+$hdt$_1:r 
   Fetch Operator
 limit: -1
   Alias -> Map Local Operator Tree:
-$hdt$_1:r
+$hdt$_1:r 
   TableScan
 alias: r
 Statistics: Num rows: 2 Data size: 47 Basic stats: COMPLETE Column 
stats: NONE

http://git-wip-us.apache.org/repos/asf/hive/blob/cc32d881/ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out
--
diff --git a/ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out 
b/ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out
index 8822447..bfdd529 100644
--- a/ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out
+++ b/ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out
@@ -255,7 +255,7 @@ STAGE PLANS:
 Map-reduce partition columns: _col0 (type: int)
 Statistics: Num rows: 250 Data size: 2656 Basic stats: 
COMPLETE Column stats: NONE
 value expressions: _col1 (type: string)
-Reducer 2
+Reducer 2 
 Reduce Operator Tree:
   Select Operator
 expressions: KEY.reducesinkkey0 (type: int), VALUE._col0 
(type: string)

hive git commit: HIVE-12550 : Cache and display last N completed queries in HS2 WebUI (Szehon, reviewed by Aihua Xu and Mohit Sabharwal)

2016-01-29 Thread szehon 
Repository: hive
Updated Branches:
  refs/heads/master 41fc1874b -> 3db7227fc


HIVE-12550 : Cache and display last N completed queries in HS2 WebUI (Szehon, 
reviewed by Aihua Xu and Mohit Sabharwal)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/3db7227f
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/3db7227f
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/3db7227f

Branch: refs/heads/master
Commit: 3db7227fc7e40ef65d59a8e5d26b9c83bc6fe08c
Parents: 41fc187
Author: Szehon Ho 
Authored: Fri Jan 22 15:37:37 2016 -0500
Committer: Szehon Ho 
Committed: Fri Jan 29 10:36:34 2016 -0800

--
 .../org/apache/hadoop/hive/conf/HiveConf.java   |  2 +
 pom.xml |  2 +-
 .../java/org/apache/hadoop/hive/ql/Driver.java  |  9 
 .../hive/service/cli/operation/Operation.java   | 12 -
 .../service/cli/operation/OperationManager.java | 49 +++-
 .../service/cli/operation/SQLOperation.java | 22 -
 .../service/cli/operation/SQLOperationInfo.java | 48 +++
 .../hive-webapps/hiveserver2/hiveserver2.jsp| 41 +++-
 8 files changed, 180 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/3db7227f/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
--
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index bfd88f8..a78f78a 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -1910,6 +1910,8 @@ public class HiveConf extends Configuration {
 "HTTP/_h...@example.com", "The HiveServer2 WebUI SPNEGO service 
principal.\n" +
 "The special string _HOST will be replaced automatically with \n" +
 "the value of hive.server2.webui.host or the correct host name."),
+
HIVE_SERVER2_WEBUI_MAX_HISTORIC_QUERIES("hive.server2.webui.max.historic.queries",
 25,
+"The maximum number of past queries to show in HiverSever2 WebUI."),
 
 // Tez session settings
 HIVE_SERVER2_TEZ_DEFAULT_QUEUES("hive.server2.tez.default.queues", "",

http://git-wip-us.apache.org/repos/asf/hive/blob/3db7227f/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 2d2a3de..802d3d4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -125,7 +125,7 @@
 1.4
 10.10.2.0
 3.1.0
-14.0.1
+15.0
 2.4.4
 2.6.0
 
${basedir}/${hive.path.to.root}/testutils/hadoop

http://git-wip-us.apache.org/repos/asf/hive/blob/3db7227f/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
--
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 
b/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
index 75187cf..4c89812 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
@@ -158,6 +158,9 @@ public class Driver implements CommandProcessor {
   // HS2 operation handle guid string
   private String operationId;
 
+  // For WebUI.  Kept alive after queryPlan is freed.
+  private String savedQueryString;
+
   private boolean checkConcurrency() {
 boolean supportConcurrency = 
conf.getBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY);
 if (!supportConcurrency) {
@@ -384,6 +387,7 @@ public class Driver implements CommandProcessor {
 } catch (Exception e) {
   LOG.warn("WARNING! Query command could not be redacted." + e);
 }
+this.savedQueryString = queryStr;
 
 //holder for parent command type/string when executing reentrant queries
 QueryState queryState = new QueryState();
@@ -1943,6 +1947,11 @@ public class Driver implements CommandProcessor {
 return errorMessage;
   }
 
+
+  public String getQueryString() {
+return savedQueryString == null ? "Unknown" : savedQueryString;
+  }
+
   /**
* Set the HS2 operation handle's guid string
* @param opId base64 encoded guid string

http://git-wip-us.apache.org/repos/asf/hive/blob/3db7227f/service/src/java/org/apache/hive/service/cli/operation/Operation.java
--
diff --git 
a/service/src/java/org/apache/hive/service/cli/operation/Operation.java 
b/service/src/java/org/apache/hive/service/cli/operation/Operation.java
index d6ac16d..0c263cf 100644
--- a/service/src/java/org/apache/hive/service/cli/operation/Operation.java
+++ b/service/src/java/org/apache/hive/service/cli/operation/Operation.java
@@ -75,6 +75,7 @@ public abstract class Operation {
 
   private long

[33/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by 
Laljo John Pullokkaran)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/be9735e1
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/be9735e1
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/be9735e1

Branch: refs/heads/master
Commit: be9735e1022cb3388496562d09202eb3f2f2bc87
Parents: 3db7227
Author: Pengcheng Xiong 
Authored: Fri Jan 29 10:51:03 2016 -0800
Committer: Pengcheng Xiong 
Committed: Fri Jan 29 10:51:03 2016 -0800

--
 .../hive/common/jsonexplain/tez/Attr.java   |   39 -
 .../hadoop/hive/common/jsonexplain/tez/Op.java  |  306 +-
 .../hive/common/jsonexplain/tez/Stage.java  |   42 +-
 .../common/jsonexplain/tez/TezJsonParser.java   |   23 +-
 .../jsonexplain/tez/TezJsonParserUtils.java |   53 +
 .../hive/common/jsonexplain/tez/Vertex.java |  115 +-
 .../test/resources/testconfiguration.properties |1 +
 .../apache/hadoop/hive/ql/exec/ExplainTask.java |2 +-
 .../hive/ql/plan/AbstractOperatorDesc.java  |7 +-
 .../apache/hadoop/hive/ql/plan/BaseWork.java|1 +
 .../hadoop/hive/ql/plan/CreateTableDesc.java|   14 +-
 .../hive/ql/plan/CreateTableLikeDesc.java   |6 +-
 .../hadoop/hive/ql/plan/FileSinkDesc.java   |2 +-
 .../apache/hadoop/hive/ql/plan/FilterDesc.java  |   13 +-
 .../apache/hadoop/hive/ql/plan/GroupByDesc.java |   14 +-
 .../hadoop/hive/ql/plan/HashTableSinkDesc.java  |   11 +-
 .../hadoop/hive/ql/plan/JoinCondDesc.java   |   45 +-
 .../apache/hadoop/hive/ql/plan/JoinDesc.java|   24 +-
 .../hive/ql/plan/LateralViewJoinDesc.java   |7 +-
 .../hadoop/hive/ql/plan/LoadTableDesc.java  |2 +-
 .../apache/hadoop/hive/ql/plan/MapJoinDesc.java |   14 +-
 .../apache/hadoop/hive/ql/plan/PlanUtils.java   |   18 +-
 .../hadoop/hive/ql/plan/ReduceSinkDesc.java |   13 +-
 .../apache/hadoop/hive/ql/plan/ReduceWork.java  |1 +
 .../apache/hadoop/hive/ql/plan/SelectDesc.java  |7 +-
 .../apache/hadoop/hive/ql/plan/Statistics.java  |   17 +-
 .../apache/hadoop/hive/ql/plan/TableDesc.java   |6 +-
 .../hadoop/hive/ql/plan/TableScanDesc.java  |   28 +-
 .../hive/ql/plan/ptf/PTFQueryInputDef.java  |6 +-
 .../plan/ptf/PartitionedTableFunctionDef.java   |4 +-
 .../test/queries/clientpositive/explainuser_4.q |  102 +
 .../clientpositive/llap/constprog_dpp.q.out |  103 +-
 .../results/clientpositive/perf/query12.q.out   |  173 +-
 .../results/clientpositive/perf/query13.q.out   |  275 +-
 .../results/clientpositive/perf/query15.q.out   |  210 +-
 .../results/clientpositive/perf/query17.q.out   |  371 +-
 .../results/clientpositive/perf/query18.q.out   |  330 +-
 .../results/clientpositive/perf/query19.q.out   |  288 +-
 .../results/clientpositive/perf/query20.q.out   |  188 +-
 .../results/clientpositive/perf/query21.q.out   |  206 +-
 .../results/clientpositive/perf/query22.q.out   |  203 +-
 .../results/clientpositive/perf/query25.q.out   |  371 +-
 .../results/clientpositive/perf/query26.q.out   |  240 +-
 .../results/clientpositive/perf/query27.q.out   |  246 +-
 .../results/clientpositive/perf/query28.q.out   |  344 +-
 .../results/clientpositive/perf/query29.q.out   |  371 +-
 .../results/clientpositive/perf/query3.q.out|  159 +-
 .../results/clientpositive/perf/query31.q.out   |  922 +-
 .../results/clientpositive/perf/query32.q.out   |  239 +-
 .../results/clientpositive/perf/query34.q.out   |  245 +-
 .../results/clientpositive/perf/query39.q.out   |  424 +-
 .../results/clientpositive/perf/query40.q.out   |  243 +-
 .../results/clientpositive/perf/query42.q.out   |  162 +-
 .../results/clientpositive/perf/query43.q.out   |  158 +-
 .../results/clientpositive/perf/query45.q.out   |  293 +-
 .../results/clientpositive/perf/query46.q.out   |  339 +-
 .../results/clientpositive/perf/query48.q.out   |  222 +-
 .../results/clientpositive/perf/query50.q.out   |  241 +-
 .../results/clientpositive/perf/query51.q.out   |  280 +-
 .../results/clientpositive/perf/query52.q.out   |  163 +-
 .../results/clientpositive/perf/query54.q.out   |  441 +-
 .../results/clientpositive/perf/query55.q.out   |  158 +-
 .../results/clientpositive/perf/query58.q.out   |  730 +-
 .../results/clientpositive/perf/query64.q.out   | 1725 +--
 .../results/clientpositive/perf/query65.q.out   |  334 +-
 .../results/clientpositive/perf/query66.q.out   |  514 +-
 .../results/clientpositive/perf/query67.q.out   |  235 +-
 .../results/clientpositive/perf/query68.q.out   |  339 +-
 .../results/clientpositive/perf/query7.q.out|  240 +-
 .../results/clientpositive/perf/query70.q.out   |  376 +-
 .../results/clientpositive/perf/query71.q.out   |  344 +-
 .../results/clientpositive/perf/query72.q.out   |  505 +-
 .../results/clientpositive/perf/query73.q.out   |  245 +-

[10/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query88.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query88.q.out 
b/ql/src/test/results/clientpositive/perf/query88.q.out
index c1d868f..89a7f52 100644
--- a/ql/src/test/results/clientpositive/perf/query88.q.out
+++ b/ql/src/test/results/clientpositive/perf/query88.q.out
@@ -233,925 +233,483 @@ Reducer 8 <- Reducer 36 (SIMPLE_EDGE), Reducer 7 
(SIMPLE_EDGE)
 Reducer 9 <- Reducer 44 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 12
- File Output Operator [FS_237]
-compressed:false
-Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Merge Join Operator [MERGEJOIN_372]
-|  condition map:[{"":"Inner Join 0 to 1"}]
-|  keys:{}
-|  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
-|  Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-|<-Reducer 11 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_233]
-| sort order:
-| Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-| value expressions:_col0 (type: bigint), _col1 (type: 
bigint), _col2 (type: bigint), _col3 (type: bigint), _col4 (type: bigint), 
_col5 (type: bigint), _col6 (type: bigint)
-| Merge Join Operator [MERGEJOIN_371]
-| |  condition map:[{"":"Inner Join 0 to 1"}]
-| |  keys:{}
-| |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-| |  Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-| |<-Reducer 10 [SIMPLE_EDGE]
-| |  Reduce Output Operator [RS_230]
-| | sort order:
-| | Statistics:Num rows: 1 Data size: 8 Basic stats: 
COMPLETE Column stats: NONE
-| | value expressions:_col0 (type: bigint), _col1 (type: 
bigint), _col2 (type: bigint), _col3 (type: bigint), _col4 (type: bigint), 
_col5 (type: bigint)
-| | Merge Join Operator [MERGEJOIN_370]
-| | |  condition map:[{"":"Inner Join 0 to 1"}]
-| | |  keys:{}
-| | |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-| | |  Statistics:Num rows: 1 Data size: 8 Basic stats: 
COMPLETE Column stats: NONE
-| | |<-Reducer 52 [SIMPLE_EDGE]
-| | |  Reduce Output Operator [RS_228]
-| | | sort order:
-| | | Statistics:Num rows: 1 Data size: 8 Basic stats: 
COMPLETE Column stats: NONE
-| | | value expressions:_col0 (type: bigint)
-| | | Group By Operator [GBY_154]
-| | | |  aggregations:["count(VALUE._col0)"]
-| | | |  outputColumnNames:["_col0"]
-| | | |  Statistics:Num rows: 1 Data size: 8 Basic 
stats: COMPLETE Column stats: NONE
-| | | |<-Reducer 51 [SIMPLE_EDGE]
-| | |Reduce Output Operator [RS_153]
-| | |   sort order:
-| | |   Statistics:Num rows: 1 Data size: 8 Basic 
stats: COMPLETE Column stats: NONE
-| | |   value expressions:_col0 (type: bigint)
-| | |   Group By Operator [GBY_152]
-| | |  aggregations:["count()"]
-| | |  outputColumnNames:["_col0"]
-| | |  Statistics:Num rows: 1 Data size: 8 
Basic stats: COMPLETE Column stats: NONE
-| | |  Merge Join Operator [MERGEJOIN_359]
-| | |  |  condition map:[{"":"Inner Join 0 to 
1"}]
-| | |  |  keys:{"0":"_col2 (type: 
int)","1":"_col0 (type: int)"}
-| | |  |  Statistics:Num rows: 17424 Data 
size: 8206704 Basic stats: COMPLETE Column stats: NONE
-| | |  |<-Map 55 [SIMPLE_EDGE]
-| | |  |  Reduce Output Operator [RS_149]
-| | |  | key expressions:_col0 (type: int)
-| | |  | Map-reduce partition 
columns:_col0 (type: int)
-| | |

[24/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query46.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query46.q.out 
b/ql/src/test/results/clientpositive/perf/query46.q.out
index 16f1606..62d7e21 100644
--- a/ql/src/test/results/clientpositive/perf/query46.q.out
+++ b/ql/src/test/results/clientpositive/perf/query46.q.out
@@ -15,227 +15,120 @@ Reducer 8 <- Map 15 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 9
- File Output Operator [FS_50]
-compressed:false
-Statistics:Num rows: 100 Data size: 86000 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_49]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 86000 Basic stats: COMPLETE 
Column stats: NONE
-   Select Operator [SEL_48]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-   |  Statistics:Num rows: 9683 Data size: 83249958789 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 8 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_47]
- key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: string), _col3 (type: string), _col4 (type: int)
- sort order:+
- Statistics:Num rows: 9683 Data size: 83249958789 
Basic stats: COMPLETE Column stats: NONE
- value expressions:_col5 (type: decimal(17,2)), _col6 
(type: decimal(17,2))
- Select Operator [SEL_46]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-Statistics:Num rows: 9683 Data size: 83249958789 
Basic stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_45]
-   predicate:(_col10 <> _col2) (type: boolean)
-   Statistics:Num rows: 9683 Data size: 
83249958789 Basic stats: COMPLETE Column stats: NONE
-   Merge Join Operator [MERGEJOIN_90]
-   |  condition map:[{"":"Inner Join 0 to 1"}]
-   |  keys:{"0":"_col6 (type: int)","1":"_col0 (type: 
int)"}
-   |  
outputColumnNames:["_col0","_col2","_col3","_col4","_col7","_col8","_col10"]
-   |  Statistics:Num rows: 9683 Data size: 
83249958789 Basic stats: COMPLETE Column stats: NONE
-   |<-Map 15 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_43]
-   | key expressions:_col0 (type: int)
-   | Map-reduce partition columns:_col0 (type: int)
-   | sort order:+
-   | Statistics:Num rows: 4000 Data size: 
40595195284 Basic stats: COMPLETE Column stats: NONE
-   | value expressions:_col1 (type: string)
-   | Select Operator [SEL_38]
-   |outputColumnNames:["_col0","_col1"]
-   |Statistics:Num rows: 4000 Data size: 
40595195284 Basic stats: COMPLETE Column stats: NONE
-   |Filter Operator [FIL_84]
-   |   predicate:ca_address_sk is not null 
(type: boolean)
-   |   Statistics:Num rows: 4000 Data 
size: 40595195284 Basic stats: COMPLETE Column stats: NONE
-   |   TableScan [TS_36]
-   |  alias:customer_address
-   |  Statistics:Num rows: 4000 Data 
size: 40595195284 Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 7 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_42]
- key expressions:_col6 (type: int)
- Map-reduce partition columns:_col6 (type: int)
- sort order:+
- Statistics:Num rows: 8801 Data size: 
75681779077 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: int), _col2 
(type: string), _col3 (type: decimal(17,2)), _col4 (type: decimal(17,2)), _col7 
(type: string), _col8 (type: string)
- Merge Join Operator [MERGEJOIN_89]
- |

[03/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/explainuser_3.q.out 
b/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
index e2db163..3f948a5 100644
--- a/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
+++ b/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
@@ -26,29 +26,19 @@ Vertex dependency in root stage
 Reducer 2 <- Map 1 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 2 vectorized
- File Output Operator [FS_8]
-compressed:false
-Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [OP_7]
-|  outputColumnNames:["_col0","_col1"]
-|  Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE 
Column stats: NONE
-|<-Map 1 [SIMPLE_EDGE] vectorized
-   Reduce Output Operator [RS_6]
-  key expressions:_col0 (type: int), _col1 (type: string)
-  sort order:++
-  Statistics:Num rows: 10 Data size: 1704 Basic stats: 
COMPLETE Column stats: NONE
-  Select Operator [OP_5]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 10 Data size: 1704 Basic stats: 
COMPLETE Column stats: NONE
- TableScan [TS_0]
-ACID table:true
-alias:acid_vectorized
-Statistics:Num rows: 10 Data size: 1704 Basic stats: 
COMPLETE Column stats: NONE
+  Fetch Operator
+limit:-1
+Stage-1
+  Reducer 2 vectorized
+  File Output Operator [FS_8]
+Select Operator [OP_7] (rows=10 width=170)
+  Output:["_col0","_col1"]
+<-Map 1 [SIMPLE_EDGE] vectorized
+  SHUFFLE [RS_6]
+Select Operator [OP_5] (rows=10 width=170)
+  Output:["_col0","_col1"]
+  TableScan [TS_0] (rows=10 width=170)
+default@acid_vectorized,acid_vectorized, ACID 
table,Tbl:COMPLETE,Col:NONE,Output:["a","b"]
 
 PREHOOK: query: explain select key, value
 FROM srcpart LATERAL VIEW explode(array(1,2,3)) myTable AS myCol
@@ -59,56 +49,41 @@ POSTHOOK: type: QUERY
 Plan not optimized by CBO.
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Map 1
- File Output Operator [FS_7]
-compressed:false
-Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [SEL_6]
-   outputColumnNames:["_col0","_col1"]
-   Statistics:Num rows: 4000 Data size: 42496 Basic stats: 
COMPLETE Column stats: NONE
-   Lateral View Join Operator [LVJ_5]
-  outputColumnNames:["_col0","_col1","_col7"]
-  Statistics:Num rows: 4000 Data size: 42496 Basic stats: 
COMPLETE Column stats: NONE
-  Select Operator [SEL_2]
- outputColumnNames:["key","value"]
- Statistics:Num rows: 2000 Data size: 21248 Basic stats: 
COMPLETE Column stats: NONE
- Lateral View Forward [LVF_1]
-Statistics:Num rows: 2000 Data size: 21248 Basic 
stats: COMPLETE Column stats: NONE
-TableScan [TS_0]
-   alias:srcpart
-   Statistics:Num rows: 2000 Data size: 21248 Basic 
stats: COMPLETE Column stats: NONE
- File Output Operator [FS_7]
-compressed:false
-Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [SEL_6]
-   outputColumnNames:["_col0","_col1"]
-   Statistics:Num rows: 4000 Data size: 42496 Basic stats: 
COMPLETE Column stats: NONE
-   Lateral View Join Operator [LVJ_5]
-  outputColumnNames:["_col0","_col1","_col7"]
-  Statistics:Num rows: 4000 Data size: 42496 Basic stats: 
COMPLETE Column stats: NONE
-  UDTF Operator [UDTF_4]
- function name:explode
-

[26/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query32.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query32.q.out 
b/ql/src/test/results/clientpositive/perf/query32.q.out
index ecc5834..dc085dd 100644
--- a/ql/src/test/results/clientpositive/perf/query32.q.out
+++ b/ql/src/test/results/clientpositive/perf/query32.q.out
@@ -46,160 +46,87 @@ Reducer 8 <- Map 10 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 4
- File Output Operator [FS_37]
-compressed:false
-Statistics:Num rows: 1 Data size: 112 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Group By Operator [GBY_35]
-|  aggregations:["sum(VALUE._col0)"]
-|  outputColumnNames:["_col0"]
-|  Statistics:Num rows: 1 Data size: 112 Basic stats: COMPLETE 
Column stats: NONE
-|<-Reducer 3 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_34]
-  sort order:
-  Statistics:Num rows: 1 Data size: 112 Basic stats: COMPLETE 
Column stats: NONE
-  value expressions:_col0 (type: decimal(17,2))
-  Group By Operator [GBY_33]
- aggregations:["sum(_col1)"]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 1 Data size: 112 Basic stats: 
COMPLETE Column stats: NONE
- Select Operator [SEL_32]
-outputColumnNames:["_col1"]
-Statistics:Num rows: 169400 Data size: 243305505 Basic 
stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_31]
-   predicate:(_col1 > CAST( _col5 AS decimal(20,15))) 
(type: boolean)
-   Statistics:Num rows: 169400 Data size: 243305505 
Basic stats: COMPLETE Column stats: NONE
-   Merge Join Operator [MERGEJOIN_59]
-   |  condition map:[{"":"Inner Join 0 to 
1"},{"":"Inner Join 1 to 2"}]
-   |  keys:{"0":"_col0 (type: int)","1":"_col0 (type: 
int)","2":"_col0 (type: int)"}
-   |  outputColumnNames:["_col1","_col5"]
-   |  Statistics:Num rows: 508200 Data size: 729916517 
Basic stats: COMPLETE Column stats: NONE
-   |<-Map 6 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_28]
-   | key expressions:_col0 (type: int)
-   | Map-reduce partition columns:_col0 (type: int)
-   | sort order:+
-   | Statistics:Num rows: 231000 Data size: 
331780228 Basic stats: COMPLETE Column stats: NONE
-   | Select Operator [SEL_12]
-   |outputColumnNames:["_col0"]
-   |Statistics:Num rows: 231000 Data size: 
331780228 Basic stats: COMPLETE Column stats: NONE
-   |Filter Operator [FIL_54]
-   |   predicate:(i_item_sk is not null and 
(i_manufact_id = 436)) (type: boolean)
-   |   Statistics:Num rows: 231000 Data size: 
331780228 Basic stats: COMPLETE Column stats: NONE
-   |   TableScan [TS_10]
-   |  alias:i
-   |  Statistics:Num rows: 462000 Data 
size: 663560457 Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 2 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_27]
-   | key expressions:_col0 (type: int)
-   | Map-reduce partition columns:_col0 (type: int)
-   | sort order:+
-   | Statistics:Num rows: 40176 Data size: 
44957392 Basic stats: COMPLETE Column stats: NONE
-   | value expressions:_col1 (type: decimal(7,2))
-   | Select Operator [SEL_9]
-   |outputColumnNames:["_col0","_col1"]
-   |Statistics:Num rows: 40176 Data size: 
44957392 Basic stats: COMPLETE Column stats: NONE
-   |Merge Join Operator [MERGEJOIN_57]
-   ||  condition map:[{"":"Inner Join 0 to 1"}]
-   ||  keys:{"0":"_col0 (type: 
int)","1":"_col0 (type: int)"}

[05/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/explainuser_1.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/explainuser_1.q.out 
b/ql/src/test/results/clientpositive/tez/explainuser_1.q.out
index 4c7e91b..120894d 100644
--- a/ql/src/test/results/clientpositive/tez/explainuser_1.q.out
+++ b/ql/src/test/results/clientpositive/tez/explainuser_1.q.out
@@ -3,12 +3,8 @@ PREHOOK: type: CREATETABLE
 POSTHOOK: query: explain create table src_orc_merge_test_part(key int, value 
string) partitioned by (ds string, ts string) stored as orc
 POSTHOOK: type: CREATETABLE
 Stage-0
-   Create Table Operator:
-  columns:["key int","value string"]
-  input format:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
-  name:default.src_orc_merge_test_part
-  output format:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
-  partition columns:["ds string","ts string"]
+  Create Table Operator:
+name:default.src_orc_merge_test_part
 
 PREHOOK: query: create table src_orc_merge_test_part(key int, value string) 
partitioned by (ds string, ts string) stored as orc
 PREHOOK: type: CREATETABLE
@@ -50,25 +46,20 @@ POSTHOOK: type: QUERY
 Plan optimized by CBO.
 
 Stage-3
-   Stats-Aggr Operator
-  Stage-0
- Move Operator
-partition:{"ds":"2012-01-03","ts":"2012-01-03+14:46:31"}
-table:{"input 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
-Stage-2
-   Dependency Collection{}
-  Stage-1
- Map 1
- File Output Operator [FS_3]
-compressed:false
-Statistics:Num rows: 500 Data size: 5312 Basic stats: 
COMPLETE Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
-Select Operator [SEL_1]
-   outputColumnNames:["_col0","_col1"]
-   Statistics:Num rows: 500 Data size: 5312 Basic 
stats: COMPLETE Column stats: NONE
-   TableScan [TS_0]
-  alias:src
-  Statistics:Num rows: 500 Data size: 5312 Basic 
stats: COMPLETE Column stats: NONE
+  Stats-Aggr Operator
+Stage-0
+  Move Operator
+table:{"name:":"default.src_orc_merge_test_part"}
+Stage-2
+  Dependency Collection{}
+Stage-1
+  Map 1
+  File Output Operator [FS_3]
+table:{"name:":"default.src_orc_merge_test_part"}
+Select Operator [SEL_1] (rows=500 width=10)
+  Output:["_col0","_col1"]
+  TableScan [TS_0] (rows=500 width=10)
+
default@src,src,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
 
 PREHOOK: query: insert overwrite table src_orc_merge_test_part 
partition(ds='2012-01-03', ts='2012-01-03+14:46:31') select * from src
 PREHOOK: type: QUERY
@@ -90,42 +81,30 @@ Vertex dependency in root stage
 Reducer 2 <- Map 1 (SIMPLE_EDGE)
 
 Stage-3
-   Stats-Aggr Operator
-  Stage-0
- Move Operator
-partition:{"ds":"2012-01-03","ts":"2012-01-03+14:46:31"}
-table:{"input 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
-Stage-2
-   Dependency Collection{}
-  Stage-1
- Reducer 2
- File Output Operator [FS_7]
-compressed:false
-Statistics:Num rows: 100 Data size: 1000 Basic stats: 
COMPLETE Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
-Select Operator [SEL_6]
-   outputColumnNames:["_col0","_col1"]
-   Statistics:Num rows: 100 Data size: 1000 Basic 
stats: COMPLETE Column stats: NONE
-   Limit [LIM_5]
-  Number of rows:100
-  Statistics:Num rows: 100 Data size: 1000 Basic 
stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_4]
-

[29/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query25.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query25.q.out 
b/ql/src/test/results/clientpositive/perf/query25.q.out
index a4973cd..0157845 100644
--- a/ql/src/test/results/clientpositive/perf/query25.q.out
+++ b/ql/src/test/results/clientpositive/perf/query25.q.out
@@ -16,249 +16,130 @@ Reducer 8 <- Map 17 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 10
- File Output Operator [FS_53]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_52]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_51]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-   |  Statistics:Num rows: 254100 Data size: 364958259 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 9 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_50]
- key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: string), _col3 (type: string)
- sort order:
- Statistics:Num rows: 254100 Data size: 364958259 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col4 (type: decimal(17,2)), _col5 
(type: decimal(17,2)), _col6 (type: decimal(17,2))
- Select Operator [SEL_49]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-Statistics:Num rows: 254100 Data size: 364958259 Basic 
stats: COMPLETE Column stats: NONE
-Group By Operator [GBY_48]
-|  
aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"]
-|  keys:KEY._col0 (type: string), KEY._col1 (type: 
string), KEY._col2 (type: string), KEY._col3 (type: string)
-|  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-|  Statistics:Num rows: 254100 Data size: 364958259 
Basic stats: COMPLETE Column stats: NONE
-|<-Reducer 8 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_47]
-  key expressions:_col0 (type: string), _col1 
(type: string), _col2 (type: string), _col3 (type: string)
-  Map-reduce partition columns:_col0 (type: 
string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
-  sort order:
-  Statistics:Num rows: 508200 Data size: 729916518 
Basic stats: COMPLETE Column stats: NONE
-  value expressions:_col4 (type: decimal(17,2)), 
_col5 (type: decimal(17,2)), _col6 (type: decimal(17,2))
-  Group By Operator [GBY_46]
- 
aggregations:["sum(_col5)","sum(_col10)","sum(_col14)"]
- keys:_col25 (type: string), _col26 (type: 
string), _col28 (type: string), _col29 (type: string)
- 
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Statistics:Num rows: 508200 Data size: 
729916518 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_45]
-
outputColumnNames:["_col25","_col26","_col28","_col29","_col5","_col10","_col14"]
-Statistics:Num rows: 508200 Data size: 
729916518 Basic stats: COMPLETE Column stats: NONE
-Merge Join Operator [MERGEJOIN_103]
-|  condition map:[{"":"Inner Join 0 to 1"}]
-|  keys:{"0":"_col1 (type: 
int)","1":"_col0 (type: int)"}
-|  
outputColumnNames:["_col5","_col10","_col14","_col25","_col26","_col28","_col29"]
-|  Statistics:Num rows: 508200 Data size: 
729916518 Basic stats: COMPLETE Column stats: NONE
-|<-Map 17 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_43]
-| key expressions:_col0 (type: int)
-

[18/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query67.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query67.q.out 
b/ql/src/test/results/clientpositive/perf/query67.q.out
index dda5347..f158829 100644
--- a/ql/src/test/results/clientpositive/perf/query67.q.out
+++ b/ql/src/test/results/clientpositive/perf/query67.q.out
@@ -95,158 +95,85 @@ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 7
- File Output Operator [FS_37]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_36]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_35]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
-   |  Statistics:Num rows: 762300 Data size: 1094874777 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 6 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_34]
- key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: string), _col3 (type: string), _col4 (type: int), _col5 
(type: int), _col6 (type: int), _col7 (type: string), _col8 (type: 
decimal(28,2)), _col9 (type: int)
- sort order:++
- Statistics:Num rows: 762300 Data size: 1094874777 Basic 
stats: COMPLETE Column stats: NONE
- Select Operator [SEL_30]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
-Statistics:Num rows: 762300 Data size: 1094874777 
Basic stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_47]
-   predicate:(rank_window_0 <= 100) (type: boolean)
-   Statistics:Num rows: 762300 Data size: 1094874777 
Basic stats: COMPLETE Column stats: NONE
-   PTF Operator [PTF_29]
-  Function definitions:[{"Input 
definition":{"type:":"WINDOWING"}},{"name:":"windowingtablefunction","order 
by:":"_col16(DESC)","partition by:":"_col0"}]
-  Statistics:Num rows: 2286900 Data size: 
3284624331 Basic stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_28]
-  |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col16"]
-  |  Statistics:Num rows: 2286900 Data size: 
3284624331 Basic stats: COMPLETE Column stats: NONE
-  |<-Reducer 5 [SIMPLE_EDGE]
- Reduce Output Operator [RS_27]
-key expressions:_col0 (type: string), 
_col16 (type: decimal(28,2))
-Map-reduce partition columns:_col0 (type: 
string)
-sort order:+-
-Statistics:Num rows: 2286900 Data size: 
3284624331 Basic stats: COMPLETE Column stats: NONE
-value expressions:_col1 (type: string), 
_col2 (type: string), _col3 (type: string), _col4 (type: int), _col5 (type: 
int), _col6 (type: int), _col7 (type: string)
-Select Operator [SEL_26]
-   
outputColumnNames:["_col0","_col1","_col16","_col2","_col3","_col4","_col5","_col6","_col7"]
-   Statistics:Num rows: 2286900 Data size: 
3284624331 Basic stats: COMPLETE Column stats: NONE
-   Group By Operator [GBY_25]
-   |  aggregations:["sum(VALUE._col0)"]
-   |  keys:KEY._col0 (type: string), 
KEY._col1 (type: string), KEY._col2 (type: string), KEY._col3 (type: string), 
KEY._col4 (type: int), KEY._col5 (type: int), KEY._col6 (type: int), KEY._col7 
(type: string), KEY._col8 (type: string)
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col9"]
-   |  Statistics:Num rows: 2286900 Data 
size: 3284624331 Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 4 [SIMPLE_EDGE]
-

[16/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query72.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query72.q.out 
b/ql/src/test/results/clientpositive/perf/query72.q.out
index 5d010c6..99e00a7 100644
--- a/ql/src/test/results/clientpositive/perf/query72.q.out
+++ b/ql/src/test/results/clientpositive/perf/query72.q.out
@@ -19,337 +19,176 @@ Reducer 8 <- Map 20 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
 Reducer 9 <- Map 21 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 13
- File Output Operator [FS_75]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_74]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_73]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-   |  Statistics:Num rows: 165056 Data size: 237066834 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 12 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_72]
- key expressions:_col5 (type: bigint), _col0 (type: 
string), _col1 (type: string), _col2 (type: int)
- sort order:-+++
- Statistics:Num rows: 165056 Data size: 237066834 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col3 (type: bigint), _col4 (type: 
bigint)
- Group By Operator [GBY_70]
- |  
aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"]
- |  keys:KEY._col0 (type: string), KEY._col1 (type: 
string), KEY._col2 (type: int)
- |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
- |  Statistics:Num rows: 165056 Data size: 237066834 Basic 
stats: COMPLETE Column stats: NONE
- |<-Reducer 11 [SIMPLE_EDGE]
-Reduce Output Operator [RS_69]
-   key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: int)
-   Map-reduce partition columns:_col0 (type: string), 
_col1 (type: string), _col2 (type: int)
-   sort order:+++
-   Statistics:Num rows: 330112 Data size: 474133669 
Basic stats: COMPLETE Column stats: NONE
-   value expressions:_col3 (type: bigint), _col4 
(type: bigint), _col5 (type: bigint)
-   Group By Operator [GBY_68]
-  
aggregations:["count(_col3)","count(_col4)","count()"]
-  keys:_col0 (type: string), _col1 (type: string), 
_col2 (type: int)
-  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-  Statistics:Num rows: 330112 Data size: 474133669 
Basic stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_66]
- 
outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
- Statistics:Num rows: 330112 Data size: 
474133669 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_142]
- |  condition map:[{"":"Left Outer Join0 to 
1"}]
- |  keys:{"0":"_col4 (type: int), _col6 (type: 
int)","1":"_col0 (type: int), _col1 (type: int)"}
- |  
outputColumnNames:["_col13","_col15","_col22","_col28"]
- |  Statistics:Num rows: 330112 Data size: 
474133669 Basic stats: COMPLETE Column stats: NONE
- |<-Map 23 [SIMPLE_EDGE]
- |  Reduce Output Operator [RS_64]
- | key expressions:_col0 (type: int), 
_col1 (type: int)
- | Map-reduce partition columns:_col0 
(type: int), _col1 (type: int)
- | sort order:++
- | Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: NONE
- | Select Operator [SEL_59]
- |outputColumnNames:["_col0","_col1"]
- |Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column

[25/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query40.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query40.q.out 
b/ql/src/test/results/clientpositive/perf/query40.q.out
index d404c81..5e2ad72 100644
--- a/ql/src/test/results/clientpositive/perf/query40.q.out
+++ b/ql/src/test/results/clientpositive/perf/query40.q.out
@@ -13,165 +13,86 @@ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 7
- File Output Operator [FS_36]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_35]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_34]
-   |  outputColumnNames:["_col0","_col1","_col2","_col3"]
-   |  Statistics:Num rows: 139755 Data size: 200727046 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 6 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_33]
- key expressions:_col0 (type: string), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 139755 Data size: 200727046 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col2 (type: decimal(23,2)), _col3 
(type: decimal(23,2))
- Group By Operator [GBY_31]
- |  aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
- |  keys:KEY._col0 (type: string), KEY._col1 (type: string)
- |  outputColumnNames:["_col0","_col1","_col2","_col3"]
- |  Statistics:Num rows: 139755 Data size: 200727046 Basic 
stats: COMPLETE Column stats: NONE
- |<-Reducer 5 [SIMPLE_EDGE]
-Reduce Output Operator [RS_30]
-   key expressions:_col0 (type: string), _col1 (type: 
string)
-   Map-reduce partition columns:_col0 (type: string), 
_col1 (type: string)
-   sort order:++
-   Statistics:Num rows: 279510 Data size: 401454092 
Basic stats: COMPLETE Column stats: NONE
-   value expressions:_col2 (type: decimal(23,2)), 
_col3 (type: decimal(23,2))
-   Group By Operator [GBY_29]
-  aggregations:["sum(_col2)","sum(_col3)"]
-  keys:_col0 (type: string), _col1 (type: string)
-  
outputColumnNames:["_col0","_col1","_col2","_col3"]
-  Statistics:Num rows: 279510 Data size: 401454092 
Basic stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_27]
- 
outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 279510 Data size: 
401454092 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_58]
- |  condition map:[{"":"Inner Join 0 to 1"}]
- |  keys:{"0":"_col0 (type: int)","1":"_col0 
(type: int)"}
- |  
outputColumnNames:["_col4","_col7","_col9","_col11","_col14"]
- |  Statistics:Num rows: 279510 Data size: 
401454092 Basic stats: COMPLETE Column stats: NONE
- |<-Map 11 [SIMPLE_EDGE]
- |  Reduce Output Operator [RS_25]
- | key expressions:_col0 (type: int)
- | Map-reduce partition columns:_col0 
(type: int)
- | sort order:+
- | Statistics:Num rows: 36524 Data size: 
40870356 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col1 (type: string)
- | Select Operator [SEL_14]
- |outputColumnNames:["_col0","_col1"]
- |Statistics:Num rows: 36524 Data 
size: 40870356 Basic stats: COMPLETE Column stats: NONE
- |Filter Operator [FIL_54]
- |   predicate:(d_date BETWEEN 
'1998-03-09' AND '1998-05-08' and d_date_sk is not null) (type: boolean)
-

[04/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/explainuser_2.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/explainuser_2.q.out 
b/ql/src/test/results/clientpositive/tez/explainuser_2.q.out
index 2d2f5f2..d8eb6c8 100644
--- a/ql/src/test/results/clientpositive/tez/explainuser_2.q.out
+++ b/ql/src/test/results/clientpositive/tez/explainuser_2.q.out
@@ -185,81 +185,47 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
 Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 3
- File Output Operator [FS_16]
-compressed:false
-Statistics:Num rows: 605 Data size: 6427 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [SEL_15]
-   outputColumnNames:["_col0","_col1","_col2"]
-   Statistics:Num rows: 605 Data size: 6427 Basic stats: COMPLETE 
Column stats: NONE
-   Merge Join Operator [MERGEJOIN_26]
-   |  condition map:[{"":"Inner Join 0 to 1"}]
-   |  keys:{"0":"_col3 (type: string)","1":"_col0 (type: string)"}
-   |  outputColumnNames:["_col0","_col3","_col6"]
-   |  Statistics:Num rows: 605 Data size: 6427 Basic stats: 
COMPLETE Column stats: NONE
-   |<-Map 5 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_13]
-   | key expressions:_col0 (type: string)
-   | Map-reduce partition columns:_col0 (type: string)
-   | sort order:+
-   | Statistics:Num rows: 500 Data size: 5312 Basic stats: 
COMPLETE Column stats: NONE
-   | value expressions:_col1 (type: string)
-   | Select Operator [SEL_8]
-   |outputColumnNames:["_col0","_col1"]
-   |Statistics:Num rows: 500 Data size: 5312 Basic stats: 
COMPLETE Column stats: NONE
-   |Filter Operator [FIL_24]
-   |   predicate:key is not null (type: boolean)
-   |   Statistics:Num rows: 500 Data size: 5312 Basic 
stats: COMPLETE Column stats: NONE
-   |   TableScan [TS_6]
-   |  alias:y
-   |  Statistics:Num rows: 500 Data size: 5312 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 2 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_12]
- key expressions:_col3 (type: string)
- Map-reduce partition columns:_col3 (type: string)
- sort order:+
- Statistics:Num rows: 550 Data size: 5843 Basic stats: 
COMPLETE Column stats: NONE
- value expressions:_col0 (type: string)
- Merge Join Operator [MERGEJOIN_25]
- |  condition map:[{"":"Inner Join 0 to 1"}]
- |  keys:{"0":"_col0 (type: string)","1":"_col1 (type: 
string)"}
- |  outputColumnNames:["_col0","_col3"]
- |  Statistics:Num rows: 550 Data size: 5843 Basic stats: 
COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- |  Reduce Output Operator [RS_9]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 500 Data size: 5312 Basic 
stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_2]
- |outputColumnNames:["_col0"]
- |Statistics:Num rows: 500 Data size: 5312 Basic 
stats: COMPLETE Column stats: NONE
- |Filter Operator [FIL_22]
- |   predicate:value is not null (type: boolean)
- |   Statistics:Num rows: 500 Data size: 5312 
Basic stats: COMPLETE Column stats: NONE
- |   TableScan [TS_0]
- |  alias:z
- |  Statistics:Num rows: 500 Data size: 5312 
Basic stats: COMPLETE Column stats: NONE
- |<-Map 4 [SIMPLE_EDGE]
-Reduce Output Operator [RS_10]
-   key expressions:_col1 (type: string)
-   Map-reduce partition columns:_col1 (type: string)
-   sort order:+
-   Statistics:Num rows: 25 Data size: 191 Basic stats: 
COMPLETE Column stats: NONE
-

[15/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query75.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query75.q.out 
b/ql/src/test/results/clientpositive/perf/query75.q.out
index 0ca6ac1..3791621 100644
--- a/ql/src/test/results/clientpositive/perf/query75.q.out
+++ b/ql/src/test/results/clientpositive/perf/query75.q.out
@@ -29,721 +29,375 @@ Reducer 7 <- Reducer 31 (SIMPLE_EDGE), Reducer 6 
(SIMPLE_EDGE)
 Reducer 8 <- Reducer 7 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 8
- File Output Operator [FS_156]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_155]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_154]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
-   |  Statistics:Num rows: 169103 Data size: 242878993 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 7 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_153]
- key expressions:_col8 (type: bigint)
- sort order:+
- Statistics:Num rows: 169103 Data size: 242878993 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: int), _col1 (type: int), 
_col2 (type: int), _col3 (type: int), _col4 (type: int), _col5 (type: int), 
_col6 (type: bigint), _col7 (type: bigint), _col9 (type: double)
- Select Operator [SEL_152]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
-Statistics:Num rows: 169103 Data size: 242878993 Basic 
stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_151]
-   predicate:((CAST( _col5 AS decimal(17,2)) / CAST( 
_col12 AS decimal(17,2))) < 0.9) (type: boolean)
-   Statistics:Num rows: 169103 Data size: 242878993 
Basic stats: COMPLETE Column stats: NONE
-   Merge Join Operator [MERGEJOIN_259]
-   |  condition map:[{"":"Inner Join 0 to 1"}]
-   |  keys:{"0":"_col1 (type: int), _col2 (type: int), 
_col3 (type: int), _col4 (type: int)","1":"_col1 (type: int), _col2 (type: 
int), _col3 (type: int), _col4 (type: int)"}
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col12","_col13"]
-   |  Statistics:Num rows: 507310 Data size: 728638416 
Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 31 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_149]
-   | key expressions:_col1 (type: int), _col2 
(type: int), _col3 (type: int), _col4 (type: int)
-   | Map-reduce partition columns:_col1 (type: 
int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
-   | sort order:
-   | Statistics:Num rows: 461191 Data size: 
662398546 Basic stats: COMPLETE Column stats: NONE
-   | value expressions:_col0 (type: int), _col5 
(type: bigint), _col6 (type: double)
-   | Group By Operator [GBY_146]
-   | |  
aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
-   | |  keys:KEY._col0 (type: int), KEY._col1 
(type: int), KEY._col2 (type: int), KEY._col3 (type: int), KEY._col4 (type: int)
-   | |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-   | |  Statistics:Num rows: 461191 Data size: 
662398546 Basic stats: COMPLETE Column stats: NONE
-   | |<-Union 30 [SIMPLE_EDGE]
-   ||<-Reducer 29 [CONTAINS]
-   ||  Reduce Output Operator [RS_145]
-   || key expressions:_col0 (type: int), 
_col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: int)
-   || Map-reduce partition columns:_col0 
(type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 
(type: int)
-   || sort order:+
-

[07/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query95.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query95.q.out 
b/ql/src/test/results/clientpositive/perf/query95.q.out
index 28c516b..2c682e7 100644
--- a/ql/src/test/results/clientpositive/perf/query95.q.out
+++ b/ql/src/test/results/clientpositive/perf/query95.q.out
@@ -15,261 +15,140 @@ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 Reducer 8 <- Map 7 (SIMPLE_EDGE), Map 9 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 6
- File Output Operator [FS_63]
-compressed:false
-Statistics:Num rows: 1 Data size: 344 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Group By Operator [GBY_61]
-|  aggregations:["count(DISTINCT 
KEY._col0:0._col0)","sum(VALUE._col1)","sum(VALUE._col2)"]
-|  outputColumnNames:["_col0","_col1","_col2"]
-|  Statistics:Num rows: 1 Data size: 344 Basic stats: COMPLETE 
Column stats: NONE
-|<-Reducer 5 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_60]
-  key expressions:_col0 (type: int)
-  sort order:+
-  Statistics:Num rows: 2662 Data size: 27016104217 Basic 
stats: COMPLETE Column stats: NONE
-  value expressions:_col2 (type: decimal(17,2)), _col3 (type: 
decimal(17,2))
-  Group By Operator [GBY_59]
- aggregations:["count(DISTINCT 
_col3)","sum(_col4)","sum(_col5)"]
- keys:_col3 (type: int)
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 2662 Data size: 27016104217 
Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_122]
- |  condition map:[{"":"Inner Join 0 to 1"}]
- |  keys:{"0":"_col0 (type: int)","1":"_col0 (type: int)"}
- |  outputColumnNames:["_col3","_col4","_col5"]
- |  Statistics:Num rows: 2662 Data size: 27016104217 
Basic stats: COMPLETE Column stats: NONE
- |<-Map 17 [SIMPLE_EDGE]
- |  Reduce Output Operator [RS_56]
- | key expressions:_col0 (type: int)
- | Map-reduce partition columns:_col0 (type: int)
- | sort order:+
- | Statistics:Num rows: 36524 Data size: 40870356 
Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_40]
- |outputColumnNames:["_col0"]
- |Statistics:Num rows: 36524 Data size: 40870356 
Basic stats: COMPLETE Column stats: NONE
- |Filter Operator [FIL_115]
- |   predicate:(d_date_sk is not null and d_date 
BETWEEN '2002-05-01' AND '2002-06-30') (type: boolean)
- |   Statistics:Num rows: 36524 Data size: 
40870356 Basic stats: COMPLETE Column stats: NONE
- |   TableScan [TS_38]
- |  alias:d
- |  Statistics:Num rows: 73049 Data size: 
81741831 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 4 [SIMPLE_EDGE]
-Reduce Output Operator [RS_55]
-   key expressions:_col0 (type: int)
-   Map-reduce partition columns:_col0 (type: int)
-   sort order:+
-   Statistics:Num rows: 2420 Data size: 
24560094211 Basic stats: COMPLETE Column stats: NONE
-   value expressions:_col3 (type: int), _col4 (type: 
decimal(7,2)), _col5 (type: decimal(7,2))
-   Merge Join Operator [MERGEJOIN_121]
-   |  condition map:[{"":"Inner Join 0 to 1"}]
-   |  keys:{"0":"_col2 (type: int)","1":"_col0 (type: 
int)"}
-   |  
outputColumnNames:["_col0","_col3","_col4","_col5"]
-   |  Statistics:Num rows: 2420 Data size: 
24560094211 Basic stats: COMPLETE Column stats: NONE
-   |<-Map 16 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_53]
-   | key expressions:_col0 (type: int)
-   | Map-reduce partition columns:_col0 (type: int)
-   | sort order:+
-   | Statistics:Num rows: 42 Data size: 77704

[30/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query19.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query19.q.out 
b/ql/src/test/results/clientpositive/perf/query19.q.out
index e6d6438..b0fda23 100644
--- a/ql/src/test/results/clientpositive/perf/query19.q.out
+++ b/ql/src/test/results/clientpositive/perf/query19.q.out
@@ -14,194 +14,102 @@ Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
 Reducer 8 <- Reducer 7 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 8
- File Output Operator [FS_43]
-compressed:false
-Statistics:Num rows: 100 Data size: 86000 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_42]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 86000 Basic stats: COMPLETE 
Column stats: NONE
-   Select Operator [SEL_41]
-   |  outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-   |  Statistics:Num rows: 53240002 Data size: 45787477895 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 7 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_40]
- key expressions:_col4 (type: decimal(17,2)), _col1 (type: 
string), _col0 (type: int), _col2 (type: int), _col3 (type: string)
- sort order:-
- Statistics:Num rows: 53240002 Data size: 45787477895 
Basic stats: COMPLETE Column stats: NONE
- Group By Operator [GBY_37]
- |  aggregations:["sum(VALUE._col0)"]
- |  keys:KEY._col0 (type: int), KEY._col1 (type: string), 
KEY._col2 (type: int), KEY._col3 (type: string)
- |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
- |  Statistics:Num rows: 53240002 Data size: 45787477895 
Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 6 [SIMPLE_EDGE]
-Reduce Output Operator [RS_36]
-   key expressions:_col0 (type: int), _col1 (type: 
string), _col2 (type: int), _col3 (type: string)
-   Map-reduce partition columns:_col0 (type: int), 
_col1 (type: string), _col2 (type: int), _col3 (type: string)
-   sort order:
-   Statistics:Num rows: 106480005 Data size: 
91574956652 Basic stats: COMPLETE Column stats: NONE
-   value expressions:_col4 (type: decimal(17,2))
-   Group By Operator [GBY_35]
-  aggregations:["sum(_col7)"]
-  keys:_col9 (type: int), _col10 (type: string), 
_col11 (type: int), _col12 (type: string)
-  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-  Statistics:Num rows: 106480005 Data size: 
91574956652 Basic stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_34]
- 
outputColumnNames:["_col9","_col10","_col11","_col12","_col7"]
- Statistics:Num rows: 106480005 Data size: 
91574956652 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_33]
-predicate:(substr(_col17, 1, 5) <> 
substr(_col19, 1, 5)) (type: boolean)
-Statistics:Num rows: 106480005 Data size: 
91574956652 Basic stats: COMPLETE Column stats: NONE
-Merge Join Operator [MERGEJOIN_74]
-|  condition map:[{"":"Inner Join 0 to 1"}]
-|  keys:{"0":"_col6 (type: 
int)","1":"_col0 (type: int)"}
-|  
outputColumnNames:["_col7","_col9","_col10","_col11","_col12","_col17","_col19"]
-|  Statistics:Num rows: 106480005 Data 
size: 91574956652 Basic stats: COMPLETE Column stats: NONE
-|<-Map 13 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_31]
-| key expressions:_col0 (type: int)
-| Map-reduce partition columns:_col0 
(type: int)
-| sort order:+
-| Statistics:Num rows: 1704 Data size: 
3256276 Basic stats: COMPLETE Column stats: NONE
-| value expressions:_col1 (type: 
string)
-

[14/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query76.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query76.q.out 
b/ql/src/test/results/clientpositive/perf/query76.q.out
index 0eba466..c90578b 100644
--- a/ql/src/test/results/clientpositive/perf/query76.q.out
+++ b/ql/src/test/results/clientpositive/perf/query76.q.out
@@ -15,270 +15,141 @@ Reducer 5 <- Union 4 (SIMPLE_EDGE)
 Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 6
- File Output Operator [FS_59]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_58]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_57]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-   |  Statistics:Num rows: 838530 Data size: 1204362280 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 5 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_56]
- key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: int), _col3 (type: int), _col4 (type: string)
- sort order:+
- Statistics:Num rows: 838530 Data size: 1204362280 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col5 (type: bigint), _col6 (type: 
decimal(17,2))
- Group By Operator [GBY_54]
- |  aggregations:["count(VALUE._col0)","sum(VALUE._col1)"]
- |  keys:KEY._col0 (type: string), KEY._col1 (type: 
string), KEY._col2 (type: int), KEY._col3 (type: int), KEY._col4 (type: string)
- |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- |  Statistics:Num rows: 838530 Data size: 1204362280 
Basic stats: COMPLETE Column stats: NONE
- |<-Union 4 [SIMPLE_EDGE]
-|<-Reducer 11 [CONTAINS]
-|  Reduce Output Operator [RS_53]
-| key expressions:_col0 (type: string), _col1 
(type: string), _col2 (type: int), _col3 (type: int), _col4 (type: string)
-| Map-reduce partition columns:_col0 (type: 
string), _col1 (type: string), _col2 (type: int), _col3 (type: int), _col4 
(type: string)
-| sort order:+
-| Statistics:Num rows: 1677060 Data size: 
2408724561 Basic stats: COMPLETE Column stats: NONE
-| value expressions:_col5 (type: bigint), _col6 
(type: decimal(17,2))
-| Group By Operator [GBY_52]
-|aggregations:["count()","sum(_col5)"]
-|keys:_col0 (type: string), _col1 (type: 
string), _col2 (type: int), _col3 (type: int), _col4 (type: string)
-|
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-|Statistics:Num rows: 1677060 Data size: 
2408724561 Basic stats: COMPLETE Column stats: NONE
-|Select Operator [SEL_31]
-|   
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-|   Statistics:Num rows: 559020 Data size: 
802908187 Basic stats: COMPLETE Column stats: NONE
-|   Merge Join Operator [MERGEJOIN_87]
-|   |  condition map:[{"":"Inner Join 0 to 1"}]
-|   |  keys:{"0":"_col0 (type: 
int)","1":"_col0 (type: int)"}
-|   |  
outputColumnNames:["_col3","_col5","_col7","_col8"]
-|   |  Statistics:Num rows: 559020 Data size: 
802908187 Basic stats: COMPLETE Column stats: NONE
-|   |<-Map 13 [SIMPLE_EDGE]
-|   |  Reduce Output Operator [RS_29]
-|   | key expressions:_col0 (type: int)
-|   | Map-reduce partition columns:_col0 
(type: int)
-|   | sort order:+
-|   | Statistics:Num rows: 73049 Data 
size: 81741831 Basic stats: COMPLETE Column stats: NONE
-|   | value expressions:_col1 (type:

[21/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query64.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query64.q.out 
b/ql/src/test/results/clientpositive/perf/query64.q.out
index d3b710a..2f52a47 100644
--- a/ql/src/test/results/clientpositive/perf/query64.q.out
+++ b/ql/src/test/results/clientpositive/perf/query64.q.out
@@ -47,1146 +47,587 @@ Reducer 8 <- Map 27 (SIMPLE_EDGE), Reducer 7 
(SIMPLE_EDGE)
 Reducer 9 <- Map 28 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 20
- File Output Operator [FS_255]
-compressed:false
-Statistics:Num rows: 122532649 Data size: 105380558466 Basic 
stats: COMPLETE Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [SEL_254]
-|  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20"]
-|  Statistics:Num rows: 122532649 Data size: 105380558466 Basic 
stats: COMPLETE Column stats: NONE
-|<-Reducer 19 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_253]
-  key expressions:_col0 (type: string), _col1 (type: string), 
_col20 (type: bigint)
-  sort order:+++
-  Statistics:Num rows: 122532649 Data size: 105380558466 Basic 
stats: COMPLETE Column stats: NONE
-  value expressions:_col2 (type: string), _col3 (type: 
string), _col4 (type: string), _col5 (type: string), _col6 (type: string), 
_col7 (type: string), _col8 (type: string), _col9 (type: string), _col10 (type: 
string), _col12 (type: bigint), _col13 (type: decimal(17,2)), _col14 (type: 
decimal(17,2)), _col15 (type: decimal(17,2)), _col16 (type: decimal(17,2)), 
_col17 (type: decimal(17,2)), _col18 (type: decimal(17,2))
-  Select Operator [SEL_252]
- 
outputColumnNames:["_col0","_col1","_col10","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col2","_col20","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
- Statistics:Num rows: 122532649 Data size: 105380558466 
Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_251]
-predicate:(_col34 <= _col15) (type: boolean)
-Statistics:Num rows: 122532649 Data size: 105380558466 
Basic stats: COMPLETE Column stats: NONE
-Merge Join Operator [MERGEJOIN_716]
-|  condition map:[{"":"Inner Join 0 to 1"}]
-|  keys:{"0":"_col1 (type: int), _col2 (type: string), 
_col3 (type: string)","1":"_col1 (type: int), _col2 (type: string), _col3 
(type: string)"}
-|  
outputColumnNames:["_col0","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col34","_col35","_col36","_col37"]
-|  Statistics:Num rows: 367597947 Data size: 
316141675400 Basic stats: COMPLETE Column stats: NONE
-|<-Reducer 18 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_248]
-| key expressions:_col1 (type: int), _col2 (type: 
string), _col3 (type: string)
-| Map-reduce partition columns:_col1 (type: int), 
_col2 (type: string), _col3 (type: string)
-| sort order:+++
-| Statistics:Num rows: 334179945 Data size: 
287401516862 Basic stats: COMPLETE Column stats: NONE
-| value expressions:_col0 (type: string), _col4 
(type: string), _col5 (type: string), _col6 (type: string), _col7 (type: 
string), _col8 (type: string), _col9 (type: string), _col10 (type: string), 
_col11 (type: string), _col15 (type: bigint), _col16 (type: decimal(17,2)), 
_col17 (type: decimal(17,2)), _col18 (type: decimal(17,2))
-| Select Operator [SEL_123]
-|
outputColumnNames:["_col0","_col1","_col10","_col11","_col15","_col16","_col17","_col18","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
-|Statistics:Num rows: 334179945 Data size: 
287401516862 Basic stats: COMPLETE Column stats: NONE
-|Group By Operator [GBY_122]
-||  
aggregations:["count(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"]
-||  keys:KEY._col0 (type:

[08/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query92.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query92.q.out 
b/ql/src/test/results/clientpositive/perf/query92.q.out
index 865290c..8ec1db2 100644
--- a/ql/src/test/results/clientpositive/perf/query92.q.out
+++ b/ql/src/test/results/clientpositive/perf/query92.q.out
@@ -13,148 +13,81 @@ Reducer 8 <- Map 10 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 5
- File Output Operator [FS_37]
-compressed:false
-Statistics:Num rows: 1 Data size: 24 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Group By Operator [GBY_35]
-|  
aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"]
-|  outputColumnNames:["_col0","_col1","_col2"]
-|  Statistics:Num rows: 1 Data size: 24 Basic stats: COMPLETE 
Column stats: NONE
-|<-Reducer 4 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_34]
-  sort order:
-  Statistics:Num rows: 1 Data size: 24 Basic stats: COMPLETE 
Column stats: NONE
-  value expressions:_col0 (type: bigint), _col1 (type: 
bigint), _col2 (type: bigint)
-  Group By Operator [GBY_33]
- aggregations:["sum(_col0)","sum(_col1)","sum(_col2)"]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 24 Basic stats: 
COMPLETE Column stats: NONE
- Select Operator [SEL_31]
-outputColumnNames:["_col0","_col1","_col2"]
-Statistics:Num rows: 4909 Data size: 5493875 Basic 
stats: COMPLETE Column stats: NONE
-Merge Join Operator [MERGEJOIN_48]
-|  condition map:[{"":"Outer Join 0 to 1"}]
-|  keys:{"0":"_col0 (type: int), _col1 (type: 
int)","1":"_col0 (type: int), _col1 (type: int)"}
-|  outputColumnNames:["_col0","_col2"]
-|  Statistics:Num rows: 4909 Data size: 5493875 Basic 
stats: COMPLETE Column stats: NONE
-|<-Reducer 3 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_28]
-| key expressions:_col0 (type: int), _col1 (type: 
int)
-| Map-reduce partition columns:_col0 (type: int), 
_col1 (type: int)
-| sort order:++
-| Statistics:Num rows: 4463 Data size: 4994432 
Basic stats: COMPLETE Column stats: NONE
-| Select Operator [SEL_13]
-|outputColumnNames:["_col0","_col1"]
-|Statistics:Num rows: 4463 Data size: 4994432 
Basic stats: COMPLETE Column stats: NONE
-|Group By Operator [GBY_12]
-||  keys:KEY._col0 (type: int), KEY._col1 
(type: int)
-||  outputColumnNames:["_col0","_col1"]
-||  Statistics:Num rows: 4463 Data size: 
4994432 Basic stats: COMPLETE Column stats: NONE
-||<-Reducer 2 [SIMPLE_EDGE]
-|   Reduce Output Operator [RS_11]
-|  key expressions:_col0 (type: int), 
_col1 (type: int)
-|  Map-reduce partition columns:_col0 
(type: int), _col1 (type: int)
-|  sort order:++
-|  Statistics:Num rows: 8927 Data size: 
9989984 Basic stats: COMPLETE Column stats: NONE
-|  Group By Operator [GBY_10]
-| keys:_col1 (type: int), _col2 (type: 
int)
-| outputColumnNames:["_col0","_col1"]
-| Statistics:Num rows: 8927 Data size: 
9989984 Basic stats: COMPLETE Column stats: NONE
-| Merge Join Operator [MERGEJOIN_46]
-| |  condition map:[{"":"Inner Join 0 
to 1"}]
-| |  keys:{"0":"_col0 (type: 
int)","1":"_col0 (type: int)"}
-| |  
outputColumnNames:["_col1","_col2"]
-| |  Statistics:Num rows: 8927 Data 
size: 9989984 Basic stats: COMPLETE Column stats: NONE
-

[06/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/constprog_dpp.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/constprog_dpp.q.out 
b/ql/src/test/results/clientpositive/tez/constprog_dpp.q.out
index acebd05..1614cb0 100644
--- a/ql/src/test/results/clientpositive/tez/constprog_dpp.q.out
+++ b/ql/src/test/results/clientpositive/tez/constprog_dpp.q.out
@@ -47,70 +47,41 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
 Reducer 5 <- Union 4 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 2
- File Output Operator [FS_17]
-compressed:false
-Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Merge Join Operator [MERGEJOIN_21]
-|  condition map:[{"":"Left Outer Join0 to 1"}]
-|  keys:{"0":"_col0 (type: int)","1":"_col0 (type: int)"}
-|  outputColumnNames:["_col0"]
-|  Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column 
stats: NONE
-|<-Map 1 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_13]
-| key expressions:_col0 (type: int)
-| Map-reduce partition columns:_col0 (type: int)
-| sort order:+
-| Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
-| Select Operator [SEL_1]
-|outputColumnNames:["_col0"]
-|Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
-|TableScan [TS_0]
-|   alias:a
-|   Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: NONE
-|<-Reducer 5 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_14]
-  key expressions:_col0 (type: int)
-  Map-reduce partition columns:_col0 (type: int)
-  sort order:+
-  Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: COMPLETE
-  Limit [LIM_11]
- Number of rows:1
- Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: COMPLETE
- Select Operator [SEL_10]
- |  outputColumnNames:["_col0"]
- |  Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: COMPLETE
- |<-Union 4 [SIMPLE_EDGE]
-|<-Map 3 [CONTAINS]
-|  Reduce Output Operator [RS_9]
-| sort order:
-| Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: COMPLETE
-| value expressions:_col0 (type: int)
-| Limit [LIM_8]
-|Number of rows:1
-|Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: COMPLETE
-|Select Operator [SEL_3]
-|   outputColumnNames:["_col0"]
-|   Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: NONE
-|   TableScan [TS_2]
-|  alias:tb2
-|  Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: NONE
-|<-Map 6 [CONTAINS]
-   Reduce Output Operator [RS_9]
-  sort order:
-  Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: COMPLETE
-  value expressions:_col0 (type: int)
-  Limit [LIM_8]
- Number of rows:1
- Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: COMPLETE
- Select Operator [SEL_5]
-outputColumnNames:["_col0"]
-Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: COMPLETE
-TableScan [TS_4]
-   alias:tb2
-   Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: COMPLETE
+  Fetch Operator
+limit:-1
+Stage-1
+  Reducer 2
+  File Output Operator [FS_17]
+Merge Join Operator [MERGEJOIN_21] (rows=1 width=0)
+

[20/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query65.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query65.q.out 
b/ql/src/test/results/clientpositive/perf/query65.q.out
index c016fed..9799fa9 100644
--- a/ql/src/test/results/clientpositive/perf/query65.q.out
+++ b/ql/src/test/results/clientpositive/perf/query65.q.out
@@ -90,226 +90,116 @@ Reducer 8 <- Map 11 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 5
- File Output Operator [FS_55]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_54]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_53]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-   |  Statistics:Num rows: 204974 Data size: 294399674 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 4 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_52]
- key expressions:_col0 (type: string), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 204974 Data size: 294399674 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col2 (type: decimal(17,2)), _col3 
(type: decimal(7,2)), _col4 (type: decimal(7,2)), _col5 (type: string)
- Select Operator [SEL_51]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-Statistics:Num rows: 204974 Data size: 294399674 Basic 
stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_50]
-   predicate:(_col11 <= CAST( (0.1 * 
UDFToDouble(_col8)) AS decimal(30,15))) (type: boolean)
-   Statistics:Num rows: 204974 Data size: 294399674 
Basic stats: COMPLETE Column stats: NONE
-   Merge Join Operator [MERGEJOIN_76]
-   |  condition map:[{"":"Inner Join 0 to 1"}]
-   |  keys:{"0":"_col7 (type: int), _col0 (type: int), 
_col2 (type: int)","1":"_col0 (type: int), _col0 (type: int), _col1 (type: 
int)"}
-   |  
outputColumnNames:["_col1","_col3","_col4","_col5","_col6","_col8","_col11"]
-   |  Statistics:Num rows: 614922 Data size: 883199024 
Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 14 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_48]
-   | key expressions:_col0 (type: int), _col0 
(type: int), _col1 (type: int)
-   | Map-reduce partition columns:_col0 (type: 
int), _col0 (type: int), _col1 (type: int)
-   | sort order:+++
-   | Statistics:Num rows: 20088 Data size: 
22478696 Basic stats: COMPLETE Column stats: NONE
-   | value expressions:_col2 (type: decimal(17,2))
-   | Select Operator [SEL_38]
-   |outputColumnNames:["_col0","_col1","_col2"]
-   |Statistics:Num rows: 20088 Data size: 
22478696 Basic stats: COMPLETE Column stats: NONE
-   |Group By Operator [GBY_37]
-   ||  aggregations:["sum(VALUE._col0)"]
-   ||  keys:KEY._col0 (type: int), KEY._col1 
(type: int)
-   ||  
outputColumnNames:["_col0","_col1","_col2"]
-   ||  Statistics:Num rows: 20088 Data size: 
22478696 Basic stats: COMPLETE Column stats: NONE
-   ||<-Reducer 13 [SIMPLE_EDGE]
-   |   Reduce Output Operator [RS_36]
-   |  key expressions:_col0 (type: int), 
_col1 (type: int)
-   |  Map-reduce partition columns:_col0 
(type: int), _col1 (type: int)
-   |  sort order:++
-   |  Statistics:Num rows: 40176 Data 
size: 44957392 Basic stats: COMPLETE Column stats: NONE
-   |  value expressions:_col2 (type: 
decimal(17,2))
-   |  Group By Operator [GBY_35]
-

[17/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query70.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query70.q.out 
b/ql/src/test/results/clientpositive/perf/query70.q.out
index 00b1c8f..9b58fdb 100644
--- a/ql/src/test/results/clientpositive/perf/query70.q.out
+++ b/ql/src/test/results/clientpositive/perf/query70.q.out
@@ -17,251 +17,133 @@ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 Reducer 9 <- Map 8 (SIMPLE_EDGE), Reducer 14 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 6
- File Output Operator [FS_62]
-compressed:false
-Statistics:Num rows: 100 Data size: 111900 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_61]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 111900 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_60]
-   |  outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-   |  Statistics:Num rows: 66289 Data size: 74179138 Basic stats: 
COMPLETE Column stats: NONE
-   |<-Reducer 5 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_59]
- key expressions:_col3 (type: string), CASE WHEN ((_col3 = 
0)) THEN (_col1) END (type: string), _col4 (type: int)
- sort order:-++
- Statistics:Num rows: 66289 Data size: 74179138 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: decimal(17,2)), _col1 
(type: string), _col2 (type: string)
- Select Operator [SEL_57]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-Statistics:Num rows: 66289 Data size: 74179138 Basic 
stats: COMPLETE Column stats: NONE
-PTF Operator [PTF_56]
-   Function definitions:[{"Input 
definition":{"type:":"WINDOWING"}},{"name:":"windowingtablefunction","order 
by:":"_col4","partition by:":"_col5, CASE WHEN ((_col5 = 2)) THEN (_col0) END"}]
-   Statistics:Num rows: 66289 Data size: 74179138 
Basic stats: COMPLETE Column stats: NONE
-   Select Operator [SEL_55]
-   |  
outputColumnNames:["_col0","_col1","_col4","_col5"]
-   |  Statistics:Num rows: 66289 Data size: 74179138 
Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 4 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_54]
- key expressions:_col5 (type: string), CASE 
WHEN ((_col5 = 2)) THEN (_col0) END (type: string), _col4 (type: decimal(17,2))
- Map-reduce partition columns:_col5 (type: 
string), CASE WHEN ((_col5 = 2)) THEN (_col0) END (type: string)
- sort order:+++
- Statistics:Num rows: 66289 Data size: 
74179138 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: string), _col1 
(type: string)
- Select Operator [SEL_53]
-
outputColumnNames:["_col0","_col1","_col4","_col5"]
-Statistics:Num rows: 66289 Data size: 
74179138 Basic stats: COMPLETE Column stats: NONE
-Group By Operator [GBY_52]
-|  aggregations:["sum(VALUE._col0)"]
-|  keys:KEY._col0 (type: string), 
KEY._col1 (type: string), KEY._col2 (type: string)
-|  
outputColumnNames:["_col0","_col1","_col2","_col3"]
-|  Statistics:Num rows: 66289 Data size: 
74179138 Basic stats: COMPLETE Column stats: NONE
-|<-Reducer 3 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_51]
-  key expressions:_col0 (type: 
string), _col1 (type: string), _col2 (type: string)
-  Map-reduce partition columns:_col0 
(type: string), _col1 (type: string), _col2 (type: string)
-  sort order:+++
-  Statistics:Num rows: 132579 Data 
size: 148359396 Basic stats: COMPLETE Column stats: NONE
-  value expressions:_col3 (type: 
decimal(17,2))
-  Group By Operator

[11/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query87.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query87.q.out 
b/ql/src/test/results/clientpositive/perf/query87.q.out
index 8e7b634..734b334 100644
--- a/ql/src/test/results/clientpositive/perf/query87.q.out
+++ b/ql/src/test/results/clientpositive/perf/query87.q.out
@@ -19,316 +19,167 @@ Reducer 6 <- Reducer 19 (SIMPLE_EDGE), Reducer 5 
(SIMPLE_EDGE)
 Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 7
- File Output Operator [FS_74]
-compressed:false
-Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Group By Operator [GBY_72]
-|  aggregations:["count(VALUE._col0)"]
-|  outputColumnNames:["_col0"]
-|  Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-|<-Reducer 6 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_71]
-  sort order:
-  Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-  value expressions:_col0 (type: bigint)
-  Group By Operator [GBY_70]
- aggregations:["count()"]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
- Select Operator [SEL_69]
-Statistics:Num rows: 2420 Data size: 20812489029 
Basic stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_68]
-   predicate:_col6 is null (type: boolean)
-   Statistics:Num rows: 2420 Data size: 
20812489029 Basic stats: COMPLETE Column stats: NONE
-   Merge Join Operator [MERGEJOIN_112]
-   |  condition map:[{"":"Left Outer Join0 to 1"}]
-   |  keys:{"0":"_col0 (type: string), _col1 (type: 
string), _col2 (type: string)","1":"_col0 (type: string), _col1 (type: string), 
_col2 (type: string)"}
-   |  outputColumnNames:["_col6"]
-   |  Statistics:Num rows: 4841 Data size: 
41624978920 Basic stats: COMPLETE Column stats: NONE
-   |<-Reducer 19 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_66]
-   | key expressions:_col0 (type: string), _col1 
(type: string), _col2 (type: string)
-   | Map-reduce partition columns:_col0 (type: 
string), _col1 (type: string), _col2 (type: string)
-   | sort order:+++
-   | Statistics:Num rows: 4400 Data size: 
37840889108 Basic stats: COMPLETE Column stats: NONE
-   | Select Operator [SEL_64]
-   |outputColumnNames:["_col0","_col1","_col2"]
-   |Statistics:Num rows: 4400 Data size: 
37840889108 Basic stats: COMPLETE Column stats: NONE
-   |Group By Operator [GBY_63]
-   ||  keys:KEY._col0 (type: string), 
KEY._col1 (type: string), KEY._col2 (type: string)
-   ||  
outputColumnNames:["_col0","_col1","_col2"]
-   ||  Statistics:Num rows: 4400 Data 
size: 37840889108 Basic stats: COMPLETE Column stats: NONE
-   ||<-Reducer 18 [SIMPLE_EDGE]
-   |   Reduce Output Operator [RS_62]
-   |  key expressions:_col0 (type: 
string), _col1 (type: string), _col2 (type: string)
-   |  Map-reduce partition columns:_col0 
(type: string), _col1 (type: string), _col2 (type: string)
-   |  sort order:+++
-   |  Statistics:Num rows: 8801 Data 
size: 75681779077 Basic stats: COMPLETE Column stats: NONE
-   |  Group By Operator [GBY_61]
-   | keys:_col3 (type: string), _col6 
(type: string), _col7 (type: string)
-   | 
outputColumnNames:["_col0","_col1","_col2"]
-   | Statistics:Num rows: 8801 
Data size: 75681779077 Basic stats: COMPLETE Column stats: NONE
-   | Merge Join

[01/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
Repository: hive
Updated Branches:
  refs/heads/master 3db7227fc -> be9735e10


http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/vector_struct_in.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/vector_struct_in.q.out 
b/ql/src/test/results/clientpositive/tez/vector_struct_in.q.out
index 078c07c..1d75d04 100644
--- a/ql/src/test/results/clientpositive/tez/vector_struct_in.q.out
+++ b/ql/src/test/results/clientpositive/tez/vector_struct_in.q.out
@@ -51,14 +51,14 @@ struct('ten','1')
 )
 POSTHOOK: type: QUERY
 Stage-0
-   Fetch Operator
-  limit:-1
-  Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1"]
- Filter Operator [FIL_4]
-predicate:(struct(id,lineid)) IN (const struct('two','3'), const 
struct('three','1'), const struct('one','1'), const struct('five','2'), const 
struct('six','1'), const struct('eight','1'), const struct('seven','1'), const 
struct('nine','1'), const struct('ten','1')) (type: boolean)
-TableScan [TS_0]
-   alias:test_1
+  Fetch Operator
+limit:-1
+Select Operator [SEL_2]
+  Output:["_col0","_col1"]
+  Filter Operator [FIL_4]
+predicate:(struct(id,lineid)) IN (const struct('two','3'), const 
struct('three','1'), const struct('one','1'), const struct('five','2'), const 
struct('six','1'), const struct('eight','1'), const struct('seven','1'), const 
struct('nine','1'), const struct('ten','1'))
+TableScan [TS_0]
+  Output:["id","lineid"]
 
 PREHOOK: query: select * from test_1 where struct(`id`, `lineid`)
 IN (
@@ -121,12 +121,12 @@ struct('ten','1')
 ) as b from test_1
 POSTHOOK: type: QUERY
 Stage-0
-   Fetch Operator
-  limit:-1
-  Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1","_col2"]
- TableScan [TS_0]
-alias:test_1
+  Fetch Operator
+limit:-1
+Select Operator [SEL_1]
+  Output:["_col0","_col1","_col2"]
+  TableScan [TS_0]
+Output:["id","lineid"]
 
 PREHOOK: query: select `id`, `lineid`, struct(`id`, `lineid`)
 IN (
@@ -209,14 +209,14 @@ struct(10,1)
 )
 POSTHOOK: type: QUERY
 Stage-0
-   Fetch Operator
-  limit:-1
-  Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1"]
- Filter Operator [FIL_4]
-predicate:(struct(id,lineid)) IN (const struct(2,3), const 
struct(3,1), const struct(1,1), const struct(5,2), const struct(6,1), const 
struct(8,1), const struct(7,1), const struct(9,1), const struct(10,1)) (type: 
boolean)
-TableScan [TS_0]
-   alias:test_2
+  Fetch Operator
+limit:-1
+Select Operator [SEL_2]
+  Output:["_col0","_col1"]
+  Filter Operator [FIL_4]
+predicate:(struct(id,lineid)) IN (const struct(2,3), const 
struct(3,1), const struct(1,1), const struct(5,2), const struct(6,1), const 
struct(8,1), const struct(7,1), const struct(9,1), const struct(10,1))
+TableScan [TS_0]
+  Output:["id","lineid"]
 
 PREHOOK: query: select * from test_2 where struct(`id`, `lineid`)
 IN (
@@ -279,12 +279,12 @@ struct(10,1)
 ) as b from test_2
 POSTHOOK: type: QUERY
 Stage-0
-   Fetch Operator
-  limit:-1
-  Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1","_col2"]
- TableScan [TS_0]
-alias:test_2
+  Fetch Operator
+limit:-1
+Select Operator [SEL_1]
+  Output:["_col0","_col1","_col2"]
+  TableScan [TS_0]
+Output:["id","lineid"]
 
 PREHOOK: query: select `id`, `lineid`, struct(`id`, `lineid`)
 IN (
@@ -367,14 +367,14 @@ struct('ten',1)
 )
 POSTHOOK: type: QUERY
 Stage-0
-   Fetch Operator
-  limit:-1
-  Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1"]
- Filter Operator [FIL_4]
-predicate:(struct(id,lineid)) IN (const struct('two',3), const 
struct('three',1), const struct('one',1), const struct('five',2), const 
struct('six',1), const struct('eight',1), const struct('seven',1), const 
struct('nine',1), const struct('ten',1)) (type: boolean)
-TableScan [TS_0]
-   alias:test_3
+  Fetch Operator
+limit:-1
+Select Operator [SEL_2]
+  Output:["_col0","_col1"]
+  Filter Operator [FIL_4]
+predicate:(struct(id,lineid)) IN (const struct('two',3), const 
struct('three',1), const struct('one',1), const struct('five',2), const 
struct('six',1), const struct('eight',1), const struct('seven',1), const 
struct('nine',1), const struct('ten',1))
+TableScan [TS_0]
+  Output:["id","lineid"]
 
 PREHOOK: query: select * from test_3 where struct(`id`, `lineid`)
 IN (
@@ -437,12 +437,12 @@ struct('ten',1)
 ) as b from test_3
 POSTHOOK: type: QUERY
 Stage-0
-   Fetch Operator
-  limit:-1
-  Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1","_col2"]
- TableScan [TS_0]
-

[27/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query31.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query31.q.out 
b/ql/src/test/results/clientpositive/perf/query31.q.out
index 5c2186d..a16eead 100644
--- a/ql/src/test/results/clientpositive/perf/query31.q.out
+++ b/ql/src/test/results/clientpositive/perf/query31.q.out
@@ -28,614 +28,316 @@ Reducer 6 <- Reducer 37 (SIMPLE_EDGE), Reducer 5 
(SIMPLE_EDGE)
 Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 7
- File Output Operator [FS_141]
-compressed:false
-Statistics:Num rows: 1183 Data size: 12007156967 Basic stats: 
COMPLETE Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [SEL_140]
-|  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-|  Statistics:Num rows: 1183 Data size: 12007156967 Basic 
stats: COMPLETE Column stats: NONE
-|<-Reducer 6 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_139]
-  key expressions:_col2 (type: decimal(37,20))
-  sort order:+
-  Statistics:Num rows: 1183 Data size: 12007156967 Basic 
stats: COMPLETE Column stats: NONE
-  value expressions:_col0 (type: string), _col3 (type: 
decimal(37,20)), _col4 (type: decimal(37,20)), _col5 (type: decimal(37,20))
-  Select Operator [SEL_138]
- 
outputColumnNames:["_col0","_col2","_col3","_col4","_col5"]
- Statistics:Num rows: 1183 Data size: 12007156967 
Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_137]
-predicate:(CASE WHEN ((_col19 > 0)) THEN ((_col23 / 
_col19)) ELSE (null) END > CASE WHEN ((_col7 > 0)) THEN ((_col11 / _col7)) ELSE 
(null) END) (type: boolean)
-Statistics:Num rows: 1183 Data size: 12007156967 
Basic stats: COMPLETE Column stats: NONE
-Merge Join Operator [MERGEJOIN_281]
-|  condition map:[{"":"Inner Join 0 to 1"}]
-|  keys:{"0":"_col12 (type: string)","1":"_col0 (type: 
string)"}
-|  
outputColumnNames:["_col0","_col3","_col7","_col11","_col15","_col19","_col23"]
-|  Statistics:Num rows: 35493334 Data size: 
36021471918 Basic stats: COMPLETE Column stats: NONE
-|<-Reducer 37 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_135]
-| key expressions:_col0 (type: string)
-| Map-reduce partition columns:_col0 (type: string)
-| sort order:+
-| Statistics:Num rows: 2200 Data size: 
22327357890 Basic stats: COMPLETE Column stats: NONE
-| value expressions:_col3 (type: decimal(17,2))
-| Select Operator [SEL_133]
-|outputColumnNames:["_col0","_col3"]
-|Statistics:Num rows: 2200 Data size: 
22327357890 Basic stats: COMPLETE Column stats: NONE
-|Group By Operator [GBY_132]
-||  aggregations:["sum(VALUE._col0)"]
-||  keys:KEY._col0 (type: string), 3 (type: 
int), 1998 (type: int)
-||  
outputColumnNames:["_col0","_col1","_col2","_col3"]
-||  Statistics:Num rows: 2200 Data size: 
22327357890 Basic stats: COMPLETE Column stats: NONE
-||<-Reducer 36 [SIMPLE_EDGE]
-|   Reduce Output Operator [RS_131]
-|  key expressions:_col0 (type: string), 3 
(type: int), 1998 (type: int)
-|  Map-reduce partition columns:_col0 
(type: string), 3 (type: int), 1998 (type: int)
-|  sort order:+++
-|  Statistics:Num rows: 4400 Data 
size: 44654715780 Basic stats: COMPLETE Column stats: NONE
-|  value expressions:_col3 (type: 
decimal(17,2))
-|  Group By Operator [GBY_130]
-| aggregations:["sum(_col3)"]
-| keys:_col0 (type: string), 3 (type: 
int), 1998 (type: int)
-| 
outputColumnNames:["_col0","_col1","_col2","_col3"]

[32/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/llap/constprog_dpp.q.out
--
diff --git a/ql/src/test/results/clientpositive/llap/constprog_dpp.q.out 
b/ql/src/test/results/clientpositive/llap/constprog_dpp.q.out
index a1addb7..ab0b36c 100644
--- a/ql/src/test/results/clientpositive/llap/constprog_dpp.q.out
+++ b/ql/src/test/results/clientpositive/llap/constprog_dpp.q.out
@@ -47,70 +47,41 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
 Reducer 5 <- Union 4 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 2 llap
- File Output Operator [FS_17]
-compressed:false
-Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Merge Join Operator [MERGEJOIN_21]
-|  condition map:[{"":"Left Outer Join0 to 1"}]
-|  keys:{"0":"_col0 (type: int)","1":"_col0 (type: int)"}
-|  outputColumnNames:["_col0"]
-|  Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column 
stats: NONE
-|<-Map 1 [SIMPLE_EDGE] llap
-|  Reduce Output Operator [RS_13]
-| key expressions:_col0 (type: int)
-| Map-reduce partition columns:_col0 (type: int)
-| sort order:+
-| Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
-| Select Operator [SEL_1]
-|outputColumnNames:["_col0"]
-|Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
-|TableScan [TS_0]
-|   alias:a
-|   Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: NONE
-|<-Reducer 5 [SIMPLE_EDGE] llap
-   Reduce Output Operator [RS_14]
-  key expressions:_col0 (type: int)
-  Map-reduce partition columns:_col0 (type: int)
-  sort order:+
-  Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: COMPLETE
-  Limit [LIM_11]
- Number of rows:1
- Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: COMPLETE
- Select Operator [SEL_10]
- |  outputColumnNames:["_col0"]
- |  Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: COMPLETE
- |<-Union 4 [SIMPLE_EDGE]
-|<-Map 3 [CONTAINS] llap
-|  Reduce Output Operator [RS_9]
-| sort order:
-| Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: COMPLETE
-| value expressions:_col0 (type: int)
-| Limit [LIM_8]
-|Number of rows:1
-|Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: COMPLETE
-|Select Operator [SEL_3]
-|   outputColumnNames:["_col0"]
-|   Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: NONE
-|   TableScan [TS_2]
-|  alias:tb2
-|  Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: NONE
-|<-Map 6 [CONTAINS] llap
-   Reduce Output Operator [RS_9]
-  sort order:
-  Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: COMPLETE
-  value expressions:_col0 (type: int)
-  Limit [LIM_8]
- Number of rows:1
- Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: COMPLETE
- Select Operator [SEL_5]
-outputColumnNames:["_col0"]
-Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: COMPLETE
-TableScan [TS_4]
-   alias:tb2
-   Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: COMPLETE
+  Fetch Operator
+limit:-1
+Stage-1
+  Reducer 2 llap
+  File Output Operator [FS_17]
+Merge Join Operator

[09/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query89.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query89.q.out 
b/ql/src/test/results/clientpositive/perf/query89.q.out
index 445ab6c..4d0fb37 100644
--- a/ql/src/test/results/clientpositive/perf/query89.q.out
+++ b/ql/src/test/results/clientpositive/perf/query89.q.out
@@ -63,162 +63,87 @@ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 7
- File Output Operator [FS_36]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_35]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_34]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
-   |  Statistics:Num rows: 51243 Data size: 73599199 Basic stats: 
COMPLETE Column stats: NONE
-   |<-Reducer 6 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_33]
- key expressions:(_col6 - _col7) (type: decimal(22,6)), 
_col3 (type: string)
- sort order:++
- Statistics:Num rows: 51243 Data size: 73599199 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: string), _col4 (type: string), _col5 (type: int), _col6 
(type: decimal(17,2)), _col7 (type: decimal(21,6))
- Select Operator [SEL_30]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
-Statistics:Num rows: 51243 Data size: 73599199 Basic 
stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_46]
-   predicate:(CASE WHEN ((avg_window_0 <> 0)) THEN 
((abs((_col6 - avg_window_0)) / avg_window_0)) ELSE (null) END > 0.1) (type: 
boolean)
-   Statistics:Num rows: 51243 Data size: 73599199 
Basic stats: COMPLETE Column stats: NONE
-   Select Operator [SEL_29]
-  
outputColumnNames:["avg_window_0","_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-  Statistics:Num rows: 153730 Data size: 220799036 
Basic stats: COMPLETE Column stats: NONE
-  PTF Operator [PTF_28]
- Function definitions:[{"Input 
definition":{"type:":"WINDOWING"}},{"name:":"windowingtablefunction","order 
by:":"_col0, _col2, _col3, _col4","partition by:":"_col0, _col2, _col3, _col4"}]
- Statistics:Num rows: 153730 Data size: 
220799036 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_27]
- |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- |  Statistics:Num rows: 153730 Data size: 
220799036 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 5 [SIMPLE_EDGE]
-Reduce Output Operator [RS_26]
-   key expressions:_col0 (type: string), 
_col2 (type: string), _col3 (type: string), _col4 (type: string)
-   Map-reduce partition columns:_col0 
(type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string)
-   sort order:
-   Statistics:Num rows: 153730 Data size: 
220799036 Basic stats: COMPLETE Column stats: NONE
-   value expressions:_col1 (type: string), 
_col5 (type: int), _col6 (type: decimal(17,2))
-   Select Operator [SEL_25]
-  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
-  Statistics:Num rows: 153730 Data 
size: 220799036 Basic stats: COMPLETE Column stats: NONE
-  Group By Operator [GBY_24]
-  |  aggregations:["sum(VALUE._col0)"]
-  |  keys:KEY._col0 (type: string), 
KEY._col1 (type: string), KEY._col2 (type: string), KEY._col3 (type: int), 
KEY._col4 (type: string), KEY._col5 (type:

[19/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query66.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query66.q.out 
b/ql/src/test/results/clientpositive/perf/query66.q.out
index 9913a1d..4127631 100644
--- a/ql/src/test/results/clientpositive/perf/query66.q.out
+++ b/ql/src/test/results/clientpositive/perf/query66.q.out
@@ -453,344 +453,178 @@ Reducer 8 <- Union 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 9
- File Output Operator [FS_76]
-compressed:false
-Statistics:Num rows: 100 Data size: 47100 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_75]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 47100 Basic stats: COMPLETE 
Column stats: NONE
-   Select Operator [SEL_74]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41","_col42","_col43"]
-   |  Statistics:Num rows: 26136 Data size: 12310056 Basic stats: 
COMPLETE Column stats: NONE
-   |<-Reducer 8 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_73]
- key expressions:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 26136 Data size: 12310056 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col1 (type: int), _col2 (type: 
string), _col3 (type: string), _col4 (type: string), _col5 (type: string), 
_col6 (type: string), _col7 (type: int), _col8 (type: decimal(38,2)), _col9 
(type: decimal(38,2)), _col10 (type: decimal(38,2)), _col11 (type: 
decimal(38,2)), _col12 (type: decimal(38,2)), _col13 (type: decimal(38,2)), 
_col14 (type: decimal(38,2)), _col15 (type: decimal(38,2)), _col16 (type: 
decimal(38,2)), _col17 (type: decimal(38,2)), _col18 (type: decimal(38,2)), 
_col19 (type: decimal(38,2)), _col20 (type: decimal(38,12)), _col21 (type: 
decimal(38,12)), _col22 (type: decimal(38,12)), _col23 (type: decimal(38,12)), 
_col24 (type: decimal(38,12)), _col25 (type: decimal(38,12)), _col26 (type: 
decimal(38,12)), _col27 (type: decimal(38,12)), _col28 (type: decimal(38,12)), 
_col29 (type: decimal(38,12)), _col30 (type: decimal(38,12)), _col31 (type: 
decimal(38,12)), _col32 (type: decimal(38,2)), _col33 (type: decimal(38,2)), 
_col34 (type: decima
 l(38,2)), _col35 (type: decimal(38,2)), _col36 (type: decimal(38,2)), _col37 
(type: decimal(38,2)), _col38 (type: decimal(38,2)), _col39 (type: 
decimal(38,2)), _col40 (type: decimal(38,2)), _col41 (type: decimal(38,2)), 
_col42 (type: decimal(38,2)), _col43 (type: decimal(38,2))
- Group By Operator [GBY_71]
- |  
aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","sum(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)","sum(VALUE._col12)","sum(VALUE._col13)","sum(VALUE._col14)","sum(VALUE._col15)","sum(VALUE._col16)","sum(VALUE._col17)","sum(VALUE._col18)","sum(VALUE._col19)","sum(VALUE._col20)","sum(VALUE._col21)","sum(VALUE._col22)","sum(VALUE._col23)","sum(VALUE._col24)","sum(VALUE._col25)","sum(VALUE._col26)","sum(VALUE._col27)","sum(VALUE._col28)","sum(VALUE._col29)","sum(VALUE._col30)","sum(VALUE._col31)","sum(VALUE._col32)","sum(VALUE._col33)","sum(VALUE._col34)","sum(VALUE._col35)"]
- |  keys:KEY._col0 (type: string), KEY._col1 (type: int), 
KEY._col2 (type: string), KEY._col3 (type: string), KEY._col4 (type: string), 
KEY._col5 (type: string), KEY._col6 (type: string), KEY._col7 (type: int)
- |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41","_col42","_col43"]
- |  Statistics:Num rows: 26136 Data size: 12310056 Basic 
stats: COMPLETE Column stats: NONE
- |<-Union 7 [SIMPLE_EDGE]
-

[13/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query80.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query80.q.out 
b/ql/src/test/results/clientpositive/perf/query80.q.out
index 061bac4..e1bbb33 100644
--- a/ql/src/test/results/clientpositive/perf/query80.q.out
+++ b/ql/src/test/results/clientpositive/perf/query80.q.out
@@ -27,576 +27,296 @@ Reducer 7 <- Reducer 6 (SIMPLE_EDGE), Union 8 (CONTAINS)
 Reducer 9 <- Union 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 10
- File Output Operator [FS_128]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_127]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_126]
-   |  outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-   |  Statistics:Num rows: 419265 Data size: 602181139 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 9 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_125]
- key expressions:_col0 (type: string), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 419265 Data size: 602181139 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col2 (type: decimal(27,2)), _col3 
(type: decimal(32,2)), _col4 (type: decimal(33,2))
- Select Operator [SEL_124]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-Statistics:Num rows: 419265 Data size: 602181139 Basic 
stats: COMPLETE Column stats: NONE
-Group By Operator [GBY_123]
-|  
aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"]
-|  keys:KEY._col0 (type: string), KEY._col1 (type: 
string), KEY._col2 (type: string)
-|  
outputColumnNames:["_col0","_col1","_col3","_col4","_col5"]
-|  Statistics:Num rows: 419265 Data size: 602181139 
Basic stats: COMPLETE Column stats: NONE
-|<-Union 8 [SIMPLE_EDGE]
-   |<-Reducer 22 [CONTAINS]
-   |  Reduce Output Operator [RS_122]
-   | key expressions:_col0 (type: string), _col1 
(type: string), _col2 (type: string)
-   | Map-reduce partition columns:_col0 (type: 
string), _col1 (type: string), _col2 (type: string)
-   | sort order:+++
-   | Statistics:Num rows: 838530 Data size: 
1204362279 Basic stats: COMPLETE Column stats: NONE
-   | value expressions:_col3 (type: 
decimal(27,2)), _col4 (type: decimal(32,2)), _col5 (type: decimal(33,2))
-   | Group By Operator [GBY_121]
-   |
aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"]
-   |keys:_col0 (type: string), _col1 (type: 
string), '0' (type: string)
-   |
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-   |Statistics:Num rows: 838530 Data size: 
1204362279 Basic stats: COMPLETE Column stats: NONE
-   |Select Operator [SEL_77]
-   |   
outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
-   |   Statistics:Num rows: 93170 Data size: 
133818031 Basic stats: COMPLETE Column stats: NONE
-   |   Group By Operator [GBY_76]
-   |   |  
aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"]
-   |   |  keys:KEY._col0 (type: string)
-   |   |  
outputColumnNames:["_col0","_col1","_col2","_col3"]
-   |   |  Statistics:Num rows: 93170 Data 
size: 133818031 Basic stats: COMPLETE Column stats: NONE
-   |   |<-Reducer 21 [SIMPLE_EDGE]
-   |  Reduce Output Operator [RS_75]
-   | key expressions:_col0 (type: 
string)
-   | Map-reduce partition 
columns:_col0 (type: string)
-   | sort order:+
-

[31/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query17.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query17.q.out 
b/ql/src/test/results/clientpositive/perf/query17.q.out
index 1e468f5..f8805e7 100644
--- a/ql/src/test/results/clientpositive/perf/query17.q.out
+++ b/ql/src/test/results/clientpositive/perf/query17.q.out
@@ -16,249 +16,130 @@ Reducer 8 <- Map 17 (SIMPLE_EDGE), Reducer 7 (SIMPLE_EDGE)
 Reducer 9 <- Reducer 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 10
- File Output Operator [FS_54]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_53]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_52]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
-   |  Statistics:Num rows: 254100 Data size: 364958259 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 9 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_51]
- key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: string)
- sort order:+++
- Statistics:Num rows: 254100 Data size: 364958259 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col3 (type: bigint), _col4 (type: 
double), _col5 (type: double), _col6 (type: double), _col7 (type: bigint), 
_col8 (type: double), _col9 (type: double), _col10 (type: double), _col11 
(type: bigint), _col12 (type: double), _col13 (type: double)
- Select Operator [SEL_49]
-
outputColumnNames:["_col0","_col1","_col10","_col11","_col12","_col13","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
-Statistics:Num rows: 254100 Data size: 364958259 Basic 
stats: COMPLETE Column stats: NONE
-Group By Operator [GBY_48]
-|  
aggregations:["count(VALUE._col0)","avg(VALUE._col1)","stddev_samp(VALUE._col2)","count(VALUE._col3)","avg(VALUE._col4)","stddev_samp(VALUE._col5)","count(VALUE._col6)","avg(VALUE._col7)","stddev_samp(VALUE._col8)"]
-|  keys:KEY._col0 (type: string), KEY._col1 (type: 
string), KEY._col2 (type: string)
-|  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
-|  Statistics:Num rows: 254100 Data size: 364958259 
Basic stats: COMPLETE Column stats: NONE
-|<-Reducer 8 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_47]
-  key expressions:_col0 (type: string), _col1 
(type: string), _col2 (type: string)
-  Map-reduce partition columns:_col0 (type: 
string), _col1 (type: string), _col2 (type: string)
-  sort order:+++
-  Statistics:Num rows: 508200 Data size: 729916518 
Basic stats: COMPLETE Column stats: NONE
-  value expressions:_col3 (type: bigint), _col4 
(type: struct), _col5 (type: 
struct), _col6 (type: bigint), _col7 
(type: struct), _col8 (type: 
struct), _col9 (type: bigint), _col10 
(type: struct), _col11 (type: 
struct)
-  Group By Operator [GBY_46]
- 
aggregations:["count(_col5)","avg(_col5)","stddev_samp(_col5)","count(_col10)","avg(_col10)","stddev_samp(_col10)","count(_col14)","avg(_col14)","stddev_samp(_col14)"]
- keys:_col22 (type: string), _col24 (type: 
string), _col25 (type: string)
- 
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- Statistics:Num rows: 508200 Data size: 
729916518 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_45]
-
outputColumnNames:["_col22","_col24","_col25","_col5","_col10","_col14"]
-

[22/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query55.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query55.q.out 
b/ql/src/test/results/clientpositive/perf/query55.q.out
index bcec2cc..91d0c72 100644
--- a/ql/src/test/results/clientpositive/perf/query55.q.out
+++ b/ql/src/test/results/clientpositive/perf/query55.q.out
@@ -11,108 +11,58 @@ Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
 Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 5
- File Output Operator [FS_24]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_23]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_22]
-   |  outputColumnNames:["_col0","_col1","_col2"]
-   |  Statistics:Num rows: 127050 Data size: 182479129 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 4 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_21]
- key expressions:_col2 (type: decimal(17,2)), _col0 (type: 
int)
- sort order:-+
- Statistics:Num rows: 127050 Data size: 182479129 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col1 (type: string)
- Group By Operator [GBY_18]
- |  aggregations:["sum(VALUE._col0)"]
- |  keys:KEY._col0 (type: int), KEY._col1 (type: string)
- |  outputColumnNames:["_col0","_col1","_col2"]
- |  Statistics:Num rows: 127050 Data size: 182479129 Basic 
stats: COMPLETE Column stats: NONE
- |<-Reducer 3 [SIMPLE_EDGE]
-Reduce Output Operator [RS_17]
-   key expressions:_col0 (type: int), _col1 (type: 
string)
-   Map-reduce partition columns:_col0 (type: int), 
_col1 (type: string)
-   sort order:++
-   Statistics:Num rows: 254100 Data size: 364958258 
Basic stats: COMPLETE Column stats: NONE
-   value expressions:_col2 (type: decimal(17,2))
-   Group By Operator [GBY_16]
-  aggregations:["sum(_col5)"]
-  keys:_col7 (type: int), _col8 (type: string)
-  outputColumnNames:["_col0","_col1","_col2"]
-  Statistics:Num rows: 254100 Data size: 364958258 
Basic stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_15]
- outputColumnNames:["_col7","_col8","_col5"]
- Statistics:Num rows: 254100 Data size: 
364958258 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_34]
- |  condition map:[{"":"Inner Join 0 to 1"}]
- |  keys:{"0":"_col4 (type: int)","1":"_col0 
(type: int)"}
- |  outputColumnNames:["_col5","_col7","_col8"]
- |  Statistics:Num rows: 254100 Data size: 
364958258 Basic stats: COMPLETE Column stats: NONE
- |<-Map 7 [SIMPLE_EDGE]
- |  Reduce Output Operator [RS_13]
- | key expressions:_col0 (type: int)
- | Map-reduce partition columns:_col0 
(type: int)
- | sort order:+
- | Statistics:Num rows: 231000 Data size: 
331780228 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col1 (type: int), 
_col2 (type: string)
- | Select Operator [SEL_8]
- |
outputColumnNames:["_col0","_col1","_col2"]
- |Statistics:Num rows: 231000 Data 
size: 331780228 Basic stats: COMPLETE Column stats: NONE
- |Filter Operator [FIL_32]
- |   predicate:((i_manager_id = 36) 
and i_item_sk is not null) (type: boolean)
- |   Statistics:Num rows: 231000 Data 
size: 331780228 Basic stats: COMPLETE Column stats: NONE
- |

[23/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query51.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query51.q.out 
b/ql/src/test/results/clientpositive/perf/query51.q.out
index f57af2c..f568ead 100644
--- a/ql/src/test/results/clientpositive/perf/query51.q.out
+++ b/ql/src/test/results/clientpositive/perf/query51.q.out
@@ -96,188 +96,100 @@ Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
 Reducer 9 <- Map 11 (SIMPLE_EDGE), Map 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 6
- File Output Operator [FS_53]
-compressed:false
-Statistics:Num rows: 100 Data size: 111900 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_52]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 111900 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_51]
-   |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-   |  Statistics:Num rows: 7365 Data size: 8241815 Basic stats: 
COMPLETE Column stats: NONE
-   |<-Reducer 5 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_50]
- key expressions:_col0 (type: int), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 7365 Data size: 8241815 Basic stats: 
COMPLETE Column stats: NONE
- value expressions:_col2 (type: decimal(27,2)), _col3 
(type: decimal(27,2)), _col4 (type: decimal(27,2)), _col5 (type: decimal(27,2))
- Select Operator [SEL_46]
-
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-Statistics:Num rows: 7365 Data size: 8241815 Basic 
stats: COMPLETE Column stats: NONE
-Filter Operator [FIL_58]
-   predicate:(max_window_0 > max_window_1) (type: 
boolean)
-   Statistics:Num rows: 7365 Data size: 8241815 Basic 
stats: COMPLETE Column stats: NONE
-   PTF Operator [PTF_45]
-  Function definitions:[{"Input 
definition":{"type:":"WINDOWING"}},{"name:":"windowingtablefunction","order 
by:":"CASE WHEN (_col1 is not null) THEN (_col1) ELSE (_col4) END","partition 
by:":"CASE WHEN (_col0 is not null) THEN (_col0) ELSE (_col3) END"}]
-  Statistics:Num rows: 22096 Data size: 24726566 
Basic stats: COMPLETE Column stats: NONE
-  Select Operator [SEL_44]
-  |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-  |  Statistics:Num rows: 22096 Data size: 
24726566 Basic stats: COMPLETE Column stats: NONE
-  |<-Reducer 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_43]
-key expressions:CASE WHEN (_col0 is not 
null) THEN (_col0) ELSE (_col3) END (type: int), CASE WHEN (_col1 is not null) 
THEN (_col1) ELSE (_col4) END (type: string)
-Map-reduce partition columns:CASE WHEN 
(_col0 is not null) THEN (_col0) ELSE (_col3) END (type: int)
-sort order:++
-Statistics:Num rows: 22096 Data size: 
24726566 Basic stats: COMPLETE Column stats: NONE
-value expressions:_col0 (type: int), _col1 
(type: string), _col2 (type: decimal(27,2)), _col3 (type: int), _col4 (type: 
string), _col5 (type: decimal(27,2))
-Merge Join Operator [MERGEJOIN_67]
-|  condition map:[{"":"Outer Join 0 to 1"}]
-|  keys:{"0":"_col0 (type: int), _col1 
(type: string)","1":"_col0 (type: int), _col1 (type: string)"}
-|  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
-|  Statistics:Num rows: 22096 Data size: 
24726566 Basic stats: COMPLETE Column stats: NONE
-|<-Reducer 10 [SIMPLE_EDGE]
-|  Reduce Output Operator [RS_41]
-| key expressions:_col0 (type: int), 
_col1 (type: string)
-| Map-reduce partition columns:_col0 
(type: int), _col1 (type: string)
-| sort order:++
-| Statistics:Num rows:

[28/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query28.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query28.q.out 
b/ql/src/test/results/clientpositive/perf/query28.q.out
index cac9ac3..aed4808 100644
--- a/ql/src/test/results/clientpositive/perf/query28.q.out
+++ b/ql/src/test/results/clientpositive/perf/query28.q.out
@@ -121,237 +121,115 @@ Reducer 7 <- Reducer 17 (SIMPLE_EDGE), Reducer 6 
(SIMPLE_EDGE)
 Reducer 9 <- Map 8 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 7
- File Output Operator [FS_64]
-compressed:false
-Statistics:Num rows: 1 Data size: 215 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_63]
-   Number of rows:100
-   Statistics:Num rows: 1 Data size: 215 Basic stats: COMPLETE 
Column stats: NONE
-   Select Operator [SEL_62]
-  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17"]
-  Statistics:Num rows: 1 Data size: 215 Basic stats: COMPLETE 
Column stats: NONE
-  Merge Join Operator [MERGEJOIN_75]
-  |  condition map:[{"":"Inner Join 0 to 1"}]
-  |  keys:{}
-  |  
outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17"]
-  |  Statistics:Num rows: 1 Data size: 215 Basic stats: 
COMPLETE Column stats: NONE
-  |<-Reducer 17 [SIMPLE_EDGE]
-  |  Reduce Output Operator [RS_60]
-  | sort order:
-  | Statistics:Num rows: 1 Data size: 136 Basic stats: 
COMPLETE Column stats: NONE
-  | value expressions:_col0 (type: decimal(11,6)), _col1 
(type: bigint), _col2 (type: bigint)
-  | Group By Operator [GBY_40]
-  | |  
aggregations:["avg(VALUE._col0)","count(VALUE._col1)","count(DISTINCT 
KEY._col0:0._col0)"]
-  | |  outputColumnNames:["_col0","_col1","_col2"]
-  | |  Statistics:Num rows: 1 Data size: 136 Basic stats: 
COMPLETE Column stats: NONE
-  | |<-Map 16 [SIMPLE_EDGE]
-  |Reduce Output Operator [RS_39]
-  |   key expressions:_col0 (type: decimal(7,2))
-  |   sort order:+
-  |   Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: NONE
-  |   value expressions:_col1 (type: 
struct), _col2 (type: bigint)
-  |   Group By Operator [GBY_38]
-  |  
aggregations:["avg(ss_list_price)","count(ss_list_price)","count(DISTINCT 
ss_list_price)"]
-  |  keys:ss_list_price (type: decimal(7,2))
-  |  
outputColumnNames:["_col0","_col1","_col2","_col3"]
-  |  Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: NONE
-  |  Select Operator [SEL_37]
-  | outputColumnNames:["ss_list_price"]
-  | Statistics:Num rows: 1 Data size: 0 Basic 
stats: PARTIAL Column stats: NONE
-  | Filter Operator [FIL_70]
-  |predicate:(ss_quantity BETWEEN 6 AND 10 
and (ss_list_price BETWEEN 91 AND 101 or ss_coupon_amt BETWEEN 1430 AND 2430 or 
ss_wholesale_cost BETWEEN 32 AND 52)) (type: boolean)
-  |Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: NONE
-  |TableScan [TS_35]
-  |   alias:store_sales
-  |   Statistics:Num rows: 1 Data size: 0 
Basic stats: PARTIAL Column stats: NONE
-  |<-Reducer 6 [SIMPLE_EDGE]
- Reduce Output Operator [RS_59]
-sort order:
-Statistics:Num rows: 1 Data size: 196 Basic stats: 
COMPLETE Column stats: NONE
-value expressions:_col0 (type: decimal(11,6)), _col1 
(type: bigint), _col2 (type: bigint), _col3 (type: decimal(11,6)), _col4 (type: 
bigint), _col5 (type: bigint), _col6 (type: decimal(11,6)), _col7 (type: 
bigint), _col8

[02/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/vector_auto_smb_mapjoin_14.q.out
--
diff --git 
a/ql/src/test/results/clientpositive/tez/vector_auto_smb_mapjoin_14.q.out 
b/ql/src/test/results/clientpositive/tez/vector_auto_smb_mapjoin_14.q.out
index 51c4ac8..7175be0 100644
--- a/ql/src/test/results/clientpositive/tez/vector_auto_smb_mapjoin_14.q.out
+++ b/ql/src/test/results/clientpositive/tez/vector_auto_smb_mapjoin_14.q.out
@@ -60,50 +60,31 @@ Vertex dependency in root stage
 Reducer 2 <- Map 1 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
- Reducer 2 vectorized
- File Output Operator [FS_14]
-compressed:false
-Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column 
stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Group By Operator [OP_21]
-|  aggregations:["count(VALUE._col0)"]
-|  outputColumnNames:["_col0"]
-|  Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-|<-Map 1 [SIMPLE_EDGE]
-   Reduce Output Operator [RS_11]
-  sort order:
-  Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
-  value expressions:_col0 (type: bigint)
-  Group By Operator [GBY_10]
- aggregations:["count()"]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE 
Column stats: NONE
- Merge Join Operator [MERGEJOIN_19]
- |  condition map:[{"":"Inner Join 0 to 1"}]
- |  keys:{"0":"_col0 (type: int)","1":"_col0 (type: int)"}
- |  Statistics:Num rows: 11 Data size: 1023 Basic stats: 
COMPLETE Column stats: NONE
- |  
- |<-Select Operator [SEL_5]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 10 Data size: 930 Basic stats: 
COMPLETE Column stats: NONE
- | Filter Operator [FIL_18]
- |predicate:key is not null (type: boolean)
- |Statistics:Num rows: 10 Data size: 930 Basic 
stats: COMPLETE Column stats: NONE
- |TableScan [TS_3]
- |   alias:b
- |   Statistics:Num rows: 10 Data size: 930 Basic 
stats: COMPLETE Column stats: NONE
- |<-Select Operator [SEL_2]
-   outputColumnNames:["_col0"]
-   Statistics:Num rows: 10 Data size: 930 Basic stats: 
COMPLETE Column stats: NONE
-   Filter Operator [FIL_17]
-  predicate:key is not null (type: boolean)
-  Statistics:Num rows: 10 Data size: 930 Basic 
stats: COMPLETE Column stats: NONE
-  TableScan [TS_0]
- alias:a
- Statistics:Num rows: 10 Data size: 930 Basic 
stats: COMPLETE Column stats: NONE
+  Fetch Operator
+limit:-1
+Stage-1
+  Reducer 2 vectorized
+  File Output Operator [FS_14]
+Group By Operator [OP_21] (rows=1 width=8)
+  Output:["_col0"],aggregations:["count(VALUE._col0)"]
+<-Map 1 [SIMPLE_EDGE]
+  SHUFFLE [RS_11]
+Group By Operator [GBY_10] (rows=1 width=8)
+  Output:["_col0"],aggregations:["count()"]
+  Merge Join Operator [MERGEJOIN_19] (rows=11 width=93)
+Conds:SEL_2._col0=SEL_5._col0(Inner)
+  <-Select Operator [SEL_5] (rows=10 width=93)
+  Output:["_col0"]
+  Filter Operator [FIL_18] (rows=10 width=93)
+predicate:key is not null
+TableScan [TS_3] (rows=10 width=93)
+  default@tbl2,b,Tbl:COMPLETE,Col:NONE,Output:["key"]
+  <-Select Operator [SEL_2] (rows=10 width=93)
+  Output:["_col0"]
+  Filter Operator [FIL_17] (rows=10 width=93)
+predicate:key is not null
+TableScan [TS_0] (rows=10 width=93)
+  default@tbl1,a,Tbl:COMPLETE,Col:NONE,Output:["key"]
 
 PREHOOK: query: select count(*) from (
   select a.key as key, a.value as val1, b.value as val2 from tbl1 a join tbl2 
b on a.key = b.key
@@ -149,67 +130,39 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE)
 Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:-1
-  Stage-1
-

[12/33] hive git commit: HIVE-12709: further improve user level explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)

2016-01-29 Thread pxiong 
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/perf/query82.q.out
--
diff --git a/ql/src/test/results/clientpositive/perf/query82.q.out 
b/ql/src/test/results/clientpositive/perf/query82.q.out
index 0bf4243..2461644 100644
--- a/ql/src/test/results/clientpositive/perf/query82.q.out
+++ b/ql/src/test/results/clientpositive/perf/query82.q.out
@@ -11,117 +11,65 @@ Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
 Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
 
 Stage-0
-   Fetch Operator
-  limit:100
-  Stage-1
- Reducer 5
- File Output Operator [FS_27]
-compressed:false
-Statistics:Num rows: 100 Data size: 143600 Basic stats: COMPLETE 
Column stats: NONE
-table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Limit [LIM_26]
-   Number of rows:100
-   Statistics:Num rows: 100 Data size: 143600 Basic stats: 
COMPLETE Column stats: NONE
-   Select Operator [SEL_25]
-   |  outputColumnNames:["_col0","_col1","_col2"]
-   |  Statistics:Num rows: 139755 Data size: 200727046 Basic 
stats: COMPLETE Column stats: NONE
-   |<-Reducer 4 [SIMPLE_EDGE]
-  Reduce Output Operator [RS_24]
- key expressions:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 139755 Data size: 200727046 Basic 
stats: COMPLETE Column stats: NONE
- value expressions:_col1 (type: string), _col2 (type: 
decimal(7,2))
- Group By Operator [GBY_22]
- |  keys:KEY._col0 (type: string), KEY._col1 (type: 
string), KEY._col2 (type: decimal(7,2))
- |  outputColumnNames:["_col0","_col1","_col2"]
- |  Statistics:Num rows: 139755 Data size: 200727046 Basic 
stats: COMPLETE Column stats: NONE
- |<-Reducer 3 [SIMPLE_EDGE]
-Reduce Output Operator [RS_21]
-   key expressions:_col0 (type: string), _col1 (type: 
string), _col2 (type: decimal(7,2))
-   Map-reduce partition columns:_col0 (type: string), 
_col1 (type: string), _col2 (type: decimal(7,2))
-   sort order:+++
-   Statistics:Num rows: 279510 Data size: 401454092 
Basic stats: COMPLETE Column stats: NONE
-   Group By Operator [GBY_20]
-  keys:_col1 (type: string), _col2 (type: string), 
_col3 (type: decimal(7,2))
-  outputColumnNames:["_col0","_col1","_col2"]
-  Statistics:Num rows: 279510 Data size: 401454092 
Basic stats: COMPLETE Column stats: NONE
-  Merge Join Operator [MERGEJOIN_43]
-  |  condition map:[{"":"Inner Join 0 to 1"}]
-  |  keys:{"0":"_col5 (type: int)","1":"_col0 
(type: int)"}
-  |  outputColumnNames:["_col1","_col2","_col3"]
-  |  Statistics:Num rows: 279510 Data size: 
401454092 Basic stats: COMPLETE Column stats: NONE
-  |<-Map 7 [SIMPLE_EDGE]
-  |  Reduce Output Operator [RS_17]
-  | key expressions:_col0 (type: int)
-  | Map-reduce partition columns:_col0 (type: 
int)
-  | sort order:+
-  | Statistics:Num rows: 36524 Data size: 
40870356 Basic stats: COMPLETE Column stats: NONE
-  | Select Operator [SEL_8]
-  |outputColumnNames:["_col0"]
-  |Statistics:Num rows: 36524 Data size: 
40870356 Basic stats: COMPLETE Column stats: NONE
-  |Filter Operator [FIL_40]
-  |   predicate:(d_date BETWEEN 
'2002-05-30' AND '2002-07-30' and d_date_sk is not null) (type: boolean)
-  |   Statistics:Num rows: 36524 Data 
size: 40870356 Basic stats: COMPLETE Column stats: NONE
-  |   TableScan [TS_6]
-  |  alias:date_dim
-  |  Statistics:Num rows: 73049 Data 
size: 81741831 Basic stats: COMPLETE Column stats: NONE
-  |<-Reducer 2 [SIMPLE_EDGE]
- Reduce Output Operator [RS_16]
-key expressions:_col5 (type: int)
-Map-reduce

[2/2] hive git commit: HIVE-12902 Refactor TxnHandler to be an interface (gates reviewed by Eugene Koifman)

2016-01-29 Thread gates 
HIVE-12902 Refactor TxnHandler to be an interface (gates reviewed by Eugene 
Koifman)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/1b2583ba
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/1b2583ba
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/1b2583ba

Branch: refs/heads/master
Commit: 1b2583ba87b80e04d861d4fa91f938ba0998fa4d
Parents: be9735e
Author: Alan Gates 
Authored: Fri Jan 29 11:46:43 2016 -0800
Committer: Alan Gates 
Committed: Fri Jan 29 11:46:43 2016 -0800

--
 .../org/apache/hadoop/hive/conf/HiveConf.java   |  44 +--
 .../hive/ql/txn/compactor/TestCompactor.java|  25 +-
 .../hadoop/hive/metastore/HiveMetaStore.java| 126 ++-
 .../hive/metastore/HiveMetaStoreClient.java |  69 ++--
 .../metastore/txn/CompactionTxnHandler.java |  46 +--
 .../hadoop/hive/metastore/txn/TxnHandler.java   |  26 +-
 .../hadoop/hive/metastore/txn/TxnStore.java | 349 +++
 .../hadoop/hive/metastore/txn/TxnUtils.java |  97 ++
 .../metastore/txn/TestCompactionTxnHandler.java |  31 +-
 .../hive/metastore/txn/TestTxnHandler.java  | 195 ++-
 .../metastore/txn/TestTxnHandlerNegative.java   |   2 +-
 .../ql/txn/AcidCompactionHistoryService.java|  16 +-
 .../hive/ql/txn/AcidHouseKeeperService.java |   9 +-
 .../hive/ql/txn/compactor/CompactorThread.java  |  11 +-
 .../hadoop/hive/ql/txn/compactor/Initiator.java |  12 +-
 .../hadoop/hive/ql/txn/compactor/Worker.java|   6 +-
 .../apache/hadoop/hive/ql/TestTxnCommands2.java |  22 +-
 .../hive/ql/lockmgr/TestDbTxnManager.java   |  15 +-
 .../hive/ql/txn/compactor/CompactorTest.java|  38 +-
 .../hive/ql/txn/compactor/TestCleaner.java  |  40 ++-
 .../hive/ql/txn/compactor/TestInitiator.java|  28 +-
 .../hive/ql/txn/compactor/TestWorker.java   |  32 +-
 22 files changed, 914 insertions(+), 325 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/1b2583ba/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
--
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index a78f78a..f4ca4a0 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -18,25 +18,7 @@
 
 package org.apache.hadoop.hive.conf;
 
-import java.io.ByteArrayOutputStream;
-import java.io.File;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.PrintStream;
-import java.net.URL;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Map;
-import java.util.Map.Entry;
-import java.util.Properties;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-import java.util.regex.Matcher;
-import java.util.regex.Pattern;
-import javax.security.auth.login.LoginException;
+import com.google.common.base.Joiner;
 import org.apache.commons.lang.StringUtils;
 import org.apache.hadoop.conf.Configuration;
 import 
org.apache.hadoop.hive.common.classification.InterfaceAudience.LimitedPrivate;
@@ -56,7 +38,25 @@ import org.apache.hive.common.HiveCompat;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import com.google.common.base.Joiner;
+import javax.security.auth.login.LoginException;
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintStream;
+import java.net.URL;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
 
 /**
  * Hive Configuration.
@@ -655,6 +655,10 @@ public class HiveConf extends Configuration {
 METASTORE_RAW_STORE_IMPL("hive.metastore.rawstore.impl", 
"org.apache.hadoop.hive.metastore.ObjectStore",
 "Name of the class that implements 
org.apache.hadoop.hive.metastore.rawstore interface. \n" +
 "This class is used to store and retrieval of raw metadata objects 
such as table, database"),
+METASTORE_TXN_STORE_IMPL("hive.metastore.txn.store.impl",
+"org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler",
+"Name of class that implements 
org.apache.hadoop.hive.metastore.txn.TxnStore.  This " +
+"class is used to store and retrieve transactions and locks"),
 METASTORE_CONNECTION_DRIVER("javax.jdo.option.ConnectionDriverName",

[1/2] hive git commit: HIVE-12902 Refactor TxnHandler to be an interface (gates reviewed by Eugene Koifman)

2016-01-29 Thread gates 
Repository: hive
Updated Branches:
  refs/heads/master be9735e10 -> 1b2583ba8


http://git-wip-us.apache.org/repos/asf/hive/blob/1b2583ba/ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestCleaner.java
--
diff --git 
a/ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestCleaner.java 
b/ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestCleaner.java
index 899f5a1..a247065 100644
--- a/ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestCleaner.java
+++ b/ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestCleaner.java
@@ -17,15 +17,27 @@
  */
 package org.apache.hadoop.hive.ql.txn.compactor;
 
-import org.apache.hadoop.hive.metastore.txn.TxnHandler;
-import org.junit.Assert;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hive.conf.HiveConf;
-import org.apache.hadoop.hive.metastore.api.*;
+import org.apache.hadoop.hive.metastore.api.CompactionRequest;
+import org.apache.hadoop.hive.metastore.api.CompactionType;
+import org.apache.hadoop.hive.metastore.api.LockComponent;
+import org.apache.hadoop.hive.metastore.api.LockLevel;
+import org.apache.hadoop.hive.metastore.api.LockRequest;
+import org.apache.hadoop.hive.metastore.api.LockResponse;
+import org.apache.hadoop.hive.metastore.api.LockType;
+import org.apache.hadoop.hive.metastore.api.Partition;
+import org.apache.hadoop.hive.metastore.api.ShowCompactRequest;
+import org.apache.hadoop.hive.metastore.api.ShowCompactResponse;
+import org.apache.hadoop.hive.metastore.api.ShowCompactResponseElement;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.api.UnlockRequest;
 import org.apache.hadoop.hive.metastore.txn.CompactionInfo;
+import org.apache.hadoop.hive.metastore.txn.TxnStore;
+import org.junit.Assert;
 import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 import java.util.ArrayList;
 import java.util.Collections;
@@ -73,7 +85,7 @@ public class TestCleaner extends CompactorTest {
 // Check there are no compactions requests left.
 ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
 Assert.assertEquals(1, rsp.getCompactsSize());
-
Assert.assertTrue(TxnHandler.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
+
Assert.assertTrue(TxnStore.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
 
 // Check that the files are removed
 List paths = getDirectories(conf, t, null);
@@ -105,7 +117,7 @@ public class TestCleaner extends CompactorTest {
 // Check there are no compactions requests left.
 ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
 Assert.assertEquals(1, rsp.getCompactsSize());
-
Assert.assertTrue(TxnHandler.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
+
Assert.assertTrue(TxnStore.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
 
 // Check that the files are removed
 List paths = getDirectories(conf, t, p);
@@ -135,7 +147,7 @@ public class TestCleaner extends CompactorTest {
 // Check there are no compactions requests left.
 ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
 Assert.assertEquals(1, rsp.getCompactsSize());
-
Assert.assertTrue(TxnHandler.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
+
Assert.assertTrue(TxnStore.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
 
 // Check that the files are removed
 List paths = getDirectories(conf, t, null);
@@ -174,7 +186,7 @@ public class TestCleaner extends CompactorTest {
 // Check there are no compactions requests left.
 ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
 Assert.assertEquals(1, rsp.getCompactsSize());
-
Assert.assertTrue(TxnHandler.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
+
Assert.assertTrue(TxnStore.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
 
 // Check that the files are removed
 List paths = getDirectories(conf, t, p);
@@ -329,7 +341,7 @@ public class TestCleaner extends CompactorTest {
 rsp = txnHandler.showCompact(new ShowCompactRequest());
 compacts = rsp.getCompacts();
 Assert.assertEquals(1, compacts.size());
-
Assert.assertTrue(TxnHandler.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
+
Assert.assertTrue(TxnStore.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
   }
 
   @Test
@@ -403,7 +415,7 @@ public class TestCleaner extends CompactorTest {
 rsp = txnHandler.showCompact(new ShowCompactRequest());
 compacts = rsp.getCompacts();
 Assert.assertEquals(1, compacts.size());
-
Assert.assertTrue(TxnHandler.SUCCEEDED_RESPONSE.equals(rsp.getCompacts().get(0).getState()));
+

[4/4] hive git commit: HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now (Sergey Shelukhin, reviewed by Xuefu Zhang)

2016-01-29 Thread sershe 
HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no 
limit and no partition filter by default for now (Sergey Shelukhin, reviewed by 
Xuefu Zhang)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/27800976
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/27800976
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/27800976

Branch: refs/heads/branch-2.0
Commit: 27800976df202668835436fb69a663311050af4f
Parents: 2961b0a
Author: Sergey Shelukhin 
Authored: Fri Jan 29 12:17:08 2016 -0800
Committer: Sergey Shelukhin 
Committed: Fri Jan 29 12:17:08 2016 -0800

--
 .../org/apache/hadoop/hive/conf/HiveConf.java   | 65 +++--
 .../calcite/translator/HiveOpConverter.java | 42 -
 .../hive/ql/optimizer/ppr/PartitionPruner.java  | 12 ++-
 .../hadoop/hive/ql/parse/CalcitePlanner.java| 19 ++--
 .../hadoop/hive/ql/parse/SemanticAnalyzer.java  | 20 ++--
 .../hive/ql/plan/ExprNodeGenericFuncDesc.java   | 19 ++--
 .../queries/clientnegative/invalid_distinct1.q  |  1 +
 .../queries/clientnegative/invalid_distinct3.q  |  1 +
 .../clientnegative/alter_view_failure6.q.out|  2 +-
 .../clientnegative/compare_double_bigint.q.out  |  2 +-
 .../clientnegative/compare_string_bigint.q.out  |  2 +-
 ql/src/test/results/clientnegative/input4.q.out |  2 +-
 .../clientnegative/input_part0_neg.q.out|  2 +-
 .../clientnegative/invalid_distinct1.q.out  |  2 +-
 .../clientnegative/invalid_distinct3.q.out  |  2 +-
 .../results/clientnegative/strict_join.q.out|  2 +-
 .../results/clientnegative/strict_orderby.q.out |  2 +-
 .../results/clientnegative/strict_pruning.q.out |  2 +-
 .../subquery_notexists_implicit_gby.q.out   |  2 +-
 .../clientpositive/create_genericudf.q.out  |  8 +-
 ql/src/test/results/clientpositive/cte_2.q.out  |  2 +-
 .../results/clientpositive/empty_join.q.out | 83 ++---
 .../clientpositive/groupby_duplicate_key.q.out  |  4 +-
 .../test/results/clientpositive/insert1.q.out   |  2 +-
 .../join_cond_pushdown_unqual5.q.out| 96 +++-
 .../clientpositive/literal_decimal.q.out|  6 +-
 .../clientpositive/load_dyn_part14.q.out| 18 ++--
 .../results/clientpositive/offset_limit.q.out   |  8 +-
 .../offset_limit_ppd_optimizer.q.out| 38 
 .../clientpositive/rand_partitionpruner2.q.out  |  2 +-
 .../results/clientpositive/smb_mapjoin_18.q.out | 29 --
 .../results/clientpositive/spark/insert1.q.out  |  2 +-
 .../clientpositive/spark/load_dyn_part14.q.out  |  6 +-
 .../clientpositive/spark/smb_mapjoin_18.q.out   | 32 +--
 .../clientpositive/spark/smb_mapjoin_8.q.out| 22 ++---
 .../clientpositive/spark/union_date_trim.q.out  |  2 +-
 .../clientpositive/stats_empty_partition.q.out  |  2 +-
 .../results/clientpositive/tez/empty_join.q.out | 75 +--
 .../tez/vector_when_case_null.q.out |  8 +-
 .../clientpositive/udf_concat_insert2.q.out |  2 +-
 .../clientpositive/union_date_trim.q.out|  2 +-
 .../clientpositive/union_remove_26.q.out| 54 +--
 .../clientpositive/vector_when_case_null.q.out  |  8 +-
 43 files changed, 410 insertions(+), 302 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/27800976/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
--
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index a11d1bc..2fb283d 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -18,6 +18,7 @@
 
 package org.apache.hadoop.hive.conf;
 
+import com.google.common.base.Joiner;
 import java.io.ByteArrayOutputStream;
 import java.io.File;
 import java.io.IOException;
@@ -56,8 +57,6 @@ import org.apache.hive.common.HiveCompat;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import com.google.common.base.Joiner;
-
 /**
  * Hive Configuration.
  */
@@ -780,14 +779,21 @@ public class HiveConf extends Configuration {
 "hive.txn.valid.txns,hive.script.operator.env.blacklist",
 "Comma separated list of keys from the configuration file not to 
convert to environment " +
 "variables when envoking the script operator"),
-HIVEMAPREDMODE("hive.mapred.mode", "strict",
-"The mode in which the Hive operations are being performed. \n" +
-"In strict mode, some risky queries are not allowed to run. They 
include:\n" +
-"  Cartesian Product.\n" +
-"  No partition being picked up for a query.\n" +
+HIVE_STRICT_CHECKS_LARGE_QUERY("hive.strict.checks.large.query", false,
+

[2/4] hive git commit: HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now (Sergey Shelukhin, reviewed by Xuefu Zhang)

2016-01-29 Thread sershe 
HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no 
limit and no partition filter by default for now (Sergey Shelukhin, reviewed by 
Xuefu Zhang)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/1ab7fe14
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/1ab7fe14
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/1ab7fe14

Branch: refs/heads/master
Commit: 1ab7fe145c8361b5d553d7b36a4c6d44a4d99b52
Parents: 1b2583b
Author: Sergey Shelukhin 
Authored: Fri Jan 29 12:14:18 2016 -0800
Committer: Sergey Shelukhin 
Committed: Fri Jan 29 12:14:18 2016 -0800

--
 .../org/apache/hadoop/hive/conf/HiveConf.java   | 62 +++--
 .../calcite/translator/HiveOpConverter.java | 42 -
 .../hive/ql/optimizer/ppr/PartitionPruner.java  | 12 ++-
 .../hadoop/hive/ql/parse/CalcitePlanner.java| 19 ++--
 .../hadoop/hive/ql/parse/SemanticAnalyzer.java  | 20 ++--
 .../hive/ql/plan/ExprNodeGenericFuncDesc.java   | 19 ++--
 .../queries/clientnegative/invalid_distinct1.q  |  1 +
 .../queries/clientnegative/invalid_distinct3.q  |  1 +
 .../test/queries/clientpositive/perf/query12.q  |  1 +
 .../clientnegative/alter_view_failure6.q.out|  2 +-
 .../clientnegative/compare_double_bigint.q.out  |  2 +-
 .../clientnegative/compare_string_bigint.q.out  |  2 +-
 ql/src/test/results/clientnegative/input4.q.out |  2 +-
 .../clientnegative/input_part0_neg.q.out|  2 +-
 .../clientnegative/invalid_distinct1.q.out  |  2 +-
 .../clientnegative/invalid_distinct3.q.out  |  2 +-
 .../results/clientnegative/strict_join.q.out|  2 +-
 .../results/clientnegative/strict_orderby.q.out |  2 +-
 .../results/clientnegative/strict_pruning.q.out |  2 +-
 .../subquery_notexists_implicit_gby.q.out   |  2 +-
 .../clientpositive/create_genericudf.q.out  |  8 +-
 ql/src/test/results/clientpositive/cte_2.q.out  |  2 +-
 .../results/clientpositive/empty_join.q.out | 83 ++---
 .../clientpositive/groupby_duplicate_key.q.out  |  4 +-
 .../test/results/clientpositive/insert1.q.out   |  2 +-
 .../join_cond_pushdown_unqual5.q.out| 96 +++-
 .../clientpositive/literal_decimal.q.out|  6 +-
 .../clientpositive/load_dyn_part14.q.out| 18 ++--
 .../results/clientpositive/offset_limit.q.out   |  8 +-
 .../offset_limit_ppd_optimizer.q.out| 38 
 .../results/clientpositive/perf/query12.q.out   |  2 -
 .../clientpositive/rand_partitionpruner2.q.out  |  2 +-
 .../results/clientpositive/smb_mapjoin_18.q.out | 29 --
 .../results/clientpositive/spark/insert1.q.out  |  2 +-
 .../clientpositive/spark/load_dyn_part14.q.out  |  6 +-
 .../clientpositive/spark/smb_mapjoin_18.q.out   | 32 +--
 .../clientpositive/spark/smb_mapjoin_8.q.out| 22 ++---
 .../clientpositive/spark/union_date_trim.q.out  |  2 +-
 .../clientpositive/stats_empty_partition.q.out  |  2 +-
 .../results/clientpositive/tez/empty_join.q.out | 46 ++
 .../tez/vector_when_case_null.q.out |  8 +-
 .../clientpositive/udf_concat_insert2.q.out |  2 +-
 .../clientpositive/union_date_trim.q.out|  2 +-
 .../clientpositive/union_remove_26.q.out| 54 +--
 .../clientpositive/vector_when_case_null.q.out  |  8 +-
 45 files changed, 393 insertions(+), 290 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/1ab7fe14/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
--
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index f4ca4a0..ff376a8 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -791,14 +791,21 @@ public class HiveConf extends Configuration {
 "hive.txn.valid.txns,hive.script.operator.env.blacklist",
 "Comma separated list of keys from the configuration file not to 
convert to environment " +
 "variables when envoking the script operator"),
-HIVEMAPREDMODE("hive.mapred.mode", "strict",
-"The mode in which the Hive operations are being performed. \n" +
-"In strict mode, some risky queries are not allowed to run. They 
include:\n" +
-"  Cartesian Product.\n" +
-"  No partition being picked up for a query.\n" +
+HIVE_STRICT_CHECKS_LARGE_QUERY("hive.strict.checks.large.query", false,
+"Enabling strict large query checks disallows the following:\n" +
+"  Orderby without limit.\n" +
+"  No partition being picked up for a query against partitioned 
table.\n" +
+"Note that these checks currently do not consider data size, only the

[1/4] hive git commit: HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now (Sergey Shelukhin, reviewed by Xuefu Zhang)

2016-01-29 Thread sershe 
Repository: hive
Updated Branches:
  refs/heads/branch-2.0 2961b0ad0 -> 27800976d
  refs/heads/master 1b2583ba8 -> 1ab7fe145


http://git-wip-us.apache.org/repos/asf/hive/blob/1ab7fe14/ql/src/test/results/clientpositive/stats_empty_partition.q.out
--
diff --git a/ql/src/test/results/clientpositive/stats_empty_partition.q.out 
b/ql/src/test/results/clientpositive/stats_empty_partition.q.out
index 0ad031c..d7f2e73 100644
--- a/ql/src/test/results/clientpositive/stats_empty_partition.q.out
+++ b/ql/src/test/results/clientpositive/stats_empty_partition.q.out
@@ -20,7 +20,7 @@ POSTHOOK: query: insert overwrite table tmptable partition 
(part = '1') select *
 POSTHOOK: type: QUERY
 POSTHOOK: Input: default@src
 POSTHOOK: Output: default@tmptable@part=1
-POSTHOOK: Lineage: tmptable PARTITION(part=1).key SIMPLE 
[(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: tmptable PARTITION(part=1).key SIMPLE []
 POSTHOOK: Lineage: tmptable PARTITION(part=1).value SIMPLE 
[(src)src.FieldSchema(name:value, type:string, comment:default), ]
 PREHOOK: query: describe formatted tmptable partition (part = '1')
 PREHOOK: type: DESCTABLE

http://git-wip-us.apache.org/repos/asf/hive/blob/1ab7fe14/ql/src/test/results/clientpositive/tez/empty_join.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/empty_join.q.out 
b/ql/src/test/results/clientpositive/tez/empty_join.q.out
index 06eea6f..4c89317 100644
--- a/ql/src/test/results/clientpositive/tez/empty_join.q.out
+++ b/ql/src/test/results/clientpositive/tez/empty_join.q.out
@@ -50,7 +50,7 @@ FROM test_1 t1
 LEFT JOIN test_2 t2 ON t1.id = t2.id
 INNER JOIN test_3 t3 ON t1.id = t3.id
 POSTHOOK: type: QUERY
-Plan not optimized by CBO.
+Plan optimized by CBO.
 
 Vertex dependency in root stage
 Map 1 <- Map 2 (BROADCAST_EDGE), Map 3 (BROADCAST_EDGE)
@@ -60,23 +60,33 @@ Stage-0
 limit:-1
 Stage-1
   Map 1
-  File Output Operator [FS_8]
-Select Operator [SEL_7] (rows=2 width=1)
-  Output:["_col0","_col1","_col2"]
-  Map Join Operator [MAPJOIN_16] (rows=2 width=1)
-Conds:TS_0.id=RS_4.id(Left 
Outer),TS_0.id=RS_5.id(Inner),Output:["_col0","_col4","_col8"]
-  <-Map 2 [BROADCAST_EDGE]
-BROADCAST [RS_4]
-  PartitionCols:id
-  TableScan [TS_1] (rows=1 width=0)
-default@test_2,t2,Tbl:PARTIAL,Col:NONE,Output:["id"]
-  <-Map 3 [BROADCAST_EDGE]
-BROADCAST [RS_5]
-  PartitionCols:id
-  TableScan [TS_2] (rows=1 width=1)
-default@test_3,t3,Tbl:COMPLETE,Col:NONE,Output:["id"]
-  <-TableScan [TS_0] (rows=1 width=1)
-  default@test_1,t1,Tbl:COMPLETE,Col:NONE,Output:["id"]
+  File Output Operator [FS_14]
+Map Join Operator [MAPJOIN_22] (rows=2 width=1)
+  Conds:SEL_2._col0=RS_10._col0(Left 
Outer),SEL_2._col0=RS_11._col0(Inner),Output:["_col0","_col1","_col2"]
+<-Map 2 [BROADCAST_EDGE]
+  BROADCAST [RS_10]
+PartitionCols:_col0
+Select Operator [SEL_5] (rows=1 width=0)
+  Output:["_col0"]
+  Filter Operator [FIL_20] (rows=1 width=0)
+predicate:id is not null
+TableScan [TS_3] (rows=1 width=0)
+  default@test_2,t2,Tbl:PARTIAL,Col:NONE,Output:["id"]
+<-Map 3 [BROADCAST_EDGE]
+  BROADCAST [RS_11]
+PartitionCols:_col0
+Select Operator [SEL_8] (rows=1 width=1)
+  Output:["_col0"]
+  Filter Operator [FIL_21] (rows=1 width=1)
+predicate:id is not null
+TableScan [TS_6] (rows=1 width=1)
+  default@test_3,t3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+<-Select Operator [SEL_2] (rows=1 width=1)
+Output:["_col0"]
+Filter Operator [FIL_19] (rows=1 width=1)
+  predicate:id is not null
+  TableScan [TS_0] (rows=1 width=1)
+default@test_1,t1,Tbl:COMPLETE,Col:NONE,Output:["id"]
 
 PREHOOK: query: SELECT t1.id, t2.id, t3.id
 FROM test_1 t1

http://git-wip-us.apache.org/repos/asf/hive/blob/1ab7fe14/ql/src/test/results/clientpositive/tez/vector_when_case_null.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/vector_when_case_null.q.out 
b/ql/src/test/results/clientpositive/tez/vector_when_case_null.q.out
index b32dd30..eb5806a 100644
--- a/ql/src/test/results/clientpositive/tez/vector_when_case_null.q.out
+++ b/ql/src/test/results/clientpositive/tez/vector_when_case_null.q.out
@@ -44,12 +44,12 @@ STAGE PLANS:
   alias: count_case_groupby
   Statistics: Num rows: 5 Data size: 452 Basic stats: COMPLETE 
Column stats: NONE
   Select

[3/4] hive git commit: HIVE-12727 : refactor Hive strict checks to be more granular, allow order by no limit and no partition filter by default for now (Sergey Shelukhin, reviewed by Xuefu Zhang)

2016-01-29 Thread sershe 
http://git-wip-us.apache.org/repos/asf/hive/blob/27800976/ql/src/test/results/clientpositive/tez/empty_join.q.out
--
diff --git a/ql/src/test/results/clientpositive/tez/empty_join.q.out 
b/ql/src/test/results/clientpositive/tez/empty_join.q.out
index ca351ec..61f6be8 100644
--- a/ql/src/test/results/clientpositive/tez/empty_join.q.out
+++ b/ql/src/test/results/clientpositive/tez/empty_join.q.out
@@ -50,7 +50,7 @@ FROM test_1 t1
 LEFT JOIN test_2 t2 ON t1.id = t2.id
 INNER JOIN test_3 t3 ON t1.id = t3.id
 POSTHOOK: type: QUERY
-Plan not optimized by CBO.
+Plan optimized by CBO.
 
 Vertex dependency in root stage
 Map 1 <- Map 2 (BROADCAST_EDGE), Map 3 (BROADCAST_EDGE)
@@ -60,39 +60,54 @@ Stage-0
   limit:-1
   Stage-1
  Map 1
- File Output Operator [FS_8]
+ File Output Operator [FS_14]
 compressed:false
 Statistics:Num rows: 2 Data size: 2 Basic stats: COMPLETE Column 
stats: NONE
 table:{"input 
format:":"org.apache.hadoop.mapred.TextInputFormat","output 
format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
-Select Operator [SEL_7]
-   outputColumnNames:["_col0","_col1","_col2"]
-   Statistics:Num rows: 2 Data size: 2 Basic stats: COMPLETE 
Column stats: NONE
-   Map Join Operator [MAPJOIN_16]
-   |  condition map:[{"":"Left Outer Join0 to 1"},{"":"Inner Join 
0 to 2"}]
-   |  keys:{"Map 1":"id (type: int)","Map 2":"id (type: int)","Map 
3":"id (type: int)"}
-   |  outputColumnNames:["_col0","_col4","_col8"]
-   |  Statistics:Num rows: 2 Data size: 2 Basic stats: COMPLETE 
Column stats: NONE
-   |<-Map 2 [BROADCAST_EDGE]
-   |  Reduce Output Operator [RS_4]
-   | key expressions:id (type: int)
-   | Map-reduce partition columns:id (type: int)
-   | sort order:+
-   | Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
-   | TableScan [TS_1]
-   |alias:t2
-   |Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: NONE
-   |<-Map 3 [BROADCAST_EDGE]
-   |  Reduce Output Operator [RS_5]
-   | key expressions:id (type: int)
-   | Map-reduce partition columns:id (type: int)
-   | sort order:+
-   | Statistics:Num rows: 1 Data size: 1 Basic stats: COMPLETE 
Column stats: NONE
-   | TableScan [TS_2]
-   |alias:t3
-   |Statistics:Num rows: 1 Data size: 1 Basic stats: 
COMPLETE Column stats: NONE
-   |<-TableScan [TS_0]
- alias:t1
+Map Join Operator [MAPJOIN_22]
+|  condition map:[{"":"Left Outer Join0 to 1"},{"":"Inner Join 0 
to 2"}]
+|  keys:{"Map 1":"_col0 (type: int)","Map 2":"_col0 (type: 
int)","Map 3":"_col0 (type: int)"}
+|  outputColumnNames:["_col0","_col1","_col2"]
+|  Statistics:Num rows: 2 Data size: 2 Basic stats: COMPLETE 
Column stats: NONE
+|<-Map 2 [BROADCAST_EDGE]
+|  Reduce Output Operator [RS_10]
+| key expressions:_col0 (type: int)
+| Map-reduce partition columns:_col0 (type: int)
+| sort order:+
+| Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
+| Select Operator [SEL_5]
+|outputColumnNames:["_col0"]
+|Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL 
Column stats: NONE
+|Filter Operator [FIL_20]
+|   predicate:id is not null (type: boolean)
+|   Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: NONE
+|   TableScan [TS_3]
+|  alias:t2
+|  Statistics:Num rows: 1 Data size: 0 Basic stats: 
PARTIAL Column stats: NONE
+|<-Map 3 [BROADCAST_EDGE]
+|  Reduce Output Operator [RS_11]
+| key expressions:_col0 (type: int)
+| Map-reduce partition columns:_col0 (type: int)
+| sort order:+
+| Statistics:Num rows: 1 Data size: 1 Basic stats: COMPLETE 
Column stats: NONE
+| Select Operator [SEL_8]
+|outputColumnNames:["_col0"]
+|Statistics:Num rows: 1 Data size: 1 Basic stats: COMPLETE 
Column stats: NONE
+|Filter Operator [FIL_21]
+|   predicate:id is not null (type: boolean)
+|   Statistics:Num rows: 1 Data size: 1 Basic stats: 
COMPLETE Column stats: NONE
+

hive git commit: HIVE-12948 : Update branch-2.0 before cutting the RC (Sergey Shelukhin, reviewed by Ashutosh Chauhan)

2016-01-29 Thread sershe 
Repository: hive
Updated Branches:
  refs/heads/branch-2.0 27800976d -> d574c13eb


HIVE-12948 : Update branch-2.0 before cutting the RC (Sergey Shelukhin, 
reviewed by Ashutosh Chauhan)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/d574c13e
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/d574c13e
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/d574c13e

Branch: refs/heads/branch-2.0
Commit: d574c13eb9e1702682ebab974e23416e99a6ad86
Parents: 2780097
Author: Sergey Shelukhin 
Authored: Fri Jan 29 12:59:00 2016 -0800
Committer: Sergey Shelukhin 
Committed: Fri Jan 29 12:59:00 2016 -0800

--
 README.txt|2 +-
 RELEASE_NOTES.txt | 1020 
 2 files changed, 1021 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/d574c13e/README.txt
--
diff --git a/README.txt b/README.txt
index 2d164e3..fb6dc53 100644
--- a/README.txt
+++ b/README.txt
@@ -81,7 +81,7 @@ Requirements
 
 - Java 1.7
 
-- Hadoop 1.x, 2.x
+- Hadoop 2.x
 
 
 Upgrading from older versions of Hive

http://git-wip-us.apache.org/repos/asf/hive/blob/d574c13e/RELEASE_NOTES.txt
--
diff --git a/RELEASE_NOTES.txt b/RELEASE_NOTES.txt
index 1e92566..9746892 100644
--- a/RELEASE_NOTES.txt
+++ b/RELEASE_NOTES.txt
@@ -1,4 +1,1024 @@
 
+Release Notes - Hive - Version 2.0.0
+
+** Sub-task
+* [HIVE-8858] - Visualize generated Spark plan [Spark Branch]
+* [HIVE-9139] - Clean up GenSparkProcContext.clonedReduceSinks and related 
code [Spark Branch]
+* [HIVE-9152] - Dynamic Partition Pruning [Spark Branch]
+* [HIVE-9605] - Remove parquet nested objects from wrapper writable objects
+* [HIVE-10166] - Merge Spark branch to master 7/30/2015
+* [HIVE-10180] - Loop optimization for SIMD in ColumnArithmeticColumn.txt
+* [HIVE-10235] - Loop optimization for SIMD in ColumnDivideColumn.txt
+* [HIVE-10238] - Loop optimization for SIMD in IfExprColumnColumn.txt
+* [HIVE-10434] - Cancel connection when remote Spark driver process has 
failed [Spark Branch] 
+* [HIVE-10458] - Enable parallel order by for spark [Spark Branch]
+* [HIVE-10460] - change the key of Parquet Record to Nullwritable instead 
of void
+* [HIVE-10476] - Hive query should fail when it fails to initialize a 
session in SetSparkReducerParallelism [Spark Branch]
+* [HIVE-10479] - CBO: Calcite Operator To Hive Operator (Calcite Return 
Path) Empty tabAlias in columnInfo which triggers PPD
+* [HIVE-10520] - LLAP: Must reset small table result columns for Native 
Vectorization of Map Join
+* [HIVE-10527] - NPE in SparkUtilities::isDedicatedCluster [Spark Branch]
+* [HIVE-10533] - CBO (Calcite Return Path): Join to MultiJoin support for 
outer joins
+* [HIVE-10550] - Dynamic RDD caching optimization for HoS.[Spark Branch]
+* [HIVE-10553] - Remove hardcoded Parquet references from 
SearchArgumentImpl
+* [HIVE-10565] - LLAP: Native Vector Map Join doesn't handle filtering and 
matching on LEFT OUTER JOIN repeated key correctly
+* [HIVE-10594] - Remote Spark client doesn't use Kerberos keytab to 
authenticate [Spark Branch]
+* [HIVE-10643] - Refactoring Windowing for sum() to pass WindowFrameDef 
instead of two numbers (1 for number of preceding and 1 for number of following)
+* [HIVE-10702] - COUNT(*) over windowing 'x preceding and y preceding' 
doesn't work properly
+* [HIVE-10717] - Fix failed qtest encryption_insert_partition_static test 
in Jenkin
+* [HIVE-10752] - Revert HIVE-5193
+* [HIVE-10755] - Rework on HIVE-5193 to enhance the column oriented table 
access
+* [HIVE-10793] - Hybrid Hybrid Grace Hash Join : Don't allocate all hash 
table memory upfront
+* [HIVE-10794] - Remove the dependence from ErrorMsg to HiveUtils
+* [HIVE-10795] - Remove use of PerfLogger from Orc
+* [HIVE-10796] - Remove dependencies on NumericHistogram and 
NumDistinctValueEstimator from JavaDataModel
+* [HIVE-10799] - Refactor the SearchArgumentFactory to remove the 
dependence on ExprNodeGenericFuncDesc
+* [HIVE-10800] - CBO (Calcite Return Path): Setup correct information if 
CBO succeeds
+* [HIVE-10825] - Add parquet branch profile to jenkins-submit-build.sh
+* [HIVE-10826] - Support min()/max() functions over x preceding and y 
preceding windowing 
+* [HIVE-10834] - Support First_value()/last_value() over x preceding and y 
preceding windowing
+* [HIVE-10844] - Combine equivalent Works for HoS[Spark Branch]
+* [HIVE-10855] - Make HIVE-10568 work with Spark [Spark Branch]
+* [HIVE-10864] - CBO (Calcite Return

hive git commit: HIVE-12948 : Update branch-2.0 before cutting the RC (Sergey Shelukhin, reviewed by Ashutosh Chauhan) ADDENDUM

2016-01-29 Thread sershe 
Repository: hive
Updated Branches:
  refs/heads/branch-2.0 d574c13eb -> b8119b0d9


HIVE-12948 : Update branch-2.0 before cutting the RC (Sergey Shelukhin, 
reviewed by Ashutosh Chauhan) ADDENDUM


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/b8119b0d
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/b8119b0d
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/b8119b0d

Branch: refs/heads/branch-2.0
Commit: b8119b0d99ce23621357e88967cff651affec518
Parents: d574c13
Author: Sergey Shelukhin 
Authored: Fri Jan 29 13:06:22 2016 -0800
Committer: Sergey Shelukhin 
Committed: Fri Jan 29 13:06:22 2016 -0800

--
 .reviewboardrc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/hive/blob/b8119b0d/.reviewboardrc
--
diff --git a/.reviewboardrc b/.reviewboardrc
index abc33f9..466e75b 100644
--- a/.reviewboardrc
+++ b/.reviewboardrc
@@ -28,6 +28,6 @@
 
 REPOSITORY='hive-git'
 REVIEWBOARD_URL='https://reviews.apache.org'
-TRACKING_BRANCH='origin/master'
+TRACKING_BRANCH='origin/branch-2.0'
 TARGET_GROUPS='hive'
 GUESS_FIELDS='true'