Review Request 26296: HIVE-8331 - HIVE-8303 followup, investigate result diff [Spark Branch]

2014-10-02 Thread Chao Sun
/26296/diff/ Testing --- Thanks, Chao Sun

Review Request 26181: HIVE-8262 - Create CacheTran that transforms the input RDD by caching it [Spark Branch]

2014-09-30 Thread Chao Sun
this, which can be used to plug in Spark Plan when caching is desirable. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/spark/CachedTran.java PRE-CREATION Diff: https://reviews.apache.org/r/26181/diff/ Testing --- Thanks, Chao Sun

Review Request 26211: HIVE-8314 - Restore thrift string interning of HIVE-7975

2014-09-30 Thread Chao Sun
/apache/hadoop/hive/metastore/api/SerDeInfo.java 24d65bb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java d0b9843 Diff: https://reviews.apache.org/r/26211/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 26120: HIVE-8278 - Restoring a graph representation of SparkPlan [Spark Branch]

2014-09-29 Thread Chao Sun
/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java 15af0f9 Diff: https://reviews.apache.org/r/26120/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 26120: HIVE-8278 - Restoring a graph representation of SparkPlan [Spark Branch]

2014-09-29 Thread Chao Sun
forgot ... now fixed. - Chao --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26120/#review54904 --- On Sept. 29, 2014, 8:16 p.m., Chao Sun

Re: Review Request 26120: HIVE-8278 - Restoring a graph representation of SparkPlan [Spark Branch]

2014-09-29 Thread Chao Sun
f8b3283 ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java 15af0f9 Diff: https://reviews.apache.org/r/26120/diff/ Testing --- Thanks, Chao Sun

Review Request 26120: HIVE-8278 - Restoring a graph representation of SparkPlan [Spark Branch]

2014-09-27 Thread Chao Sun
://reviews.apache.org/r/26120/diff/ Testing --- Thanks, Chao Sun

Review Request 26047: HIVE-8256 - Add SORT_QUERY_RESULTS for test that doesn't guarantee order #2

2014-09-25 Thread Chao Sun
/table_access_keys_stats.q.out adea0f6 Diff: https://reviews.apache.org/r/26047/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 26001: HIVE-8233 - multi-table insertion doesn't work with ForwardOperator [Spark Branch]

2014-09-25 Thread Chao Sun
/spark_groupby_multi_insert_common_distinct.q.out PRE-CREATION ql/src/test/results/clientpositive/spark/spark_union17.q.out PRE-CREATION Diff: https://reviews.apache.org/r/26001/diff/ Testing --- Thanks, Chao Sun

Review Request 26001: HIVE-8233 - multi-table insertion doesn't work with ForwardOperator [Spark Branch]

2014-09-24 Thread Chao Sun
/spark_groupby_multi_insert_common_distinct.q.out PRE-CREATION ql/src/test/results/clientpositive/spark/spark_union17.q.out PRE-CREATION Diff: https://reviews.apache.org/r/26001/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 26001: HIVE-8233 - multi-table insertion doesn't work with ForwardOperator [Spark Branch]

2014-09-24 Thread Chao Sun
--- Thanks, Chao Sun

Review Request 26007: HIVE-8249 - Refactoring SparkPlan and SparkPlanGenerator [Spark Branch]

2014-09-24 Thread Chao Sun
://reviews.apache.org/r/26007/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 26007: HIVE-8249 - Refactoring SparkPlan and SparkPlanGenerator [Spark Branch]

2014-09-24 Thread Chao Sun
/#review54481 --- On Sept. 24, 2014, 10:50 p.m., Chao Sun wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26007

Re: Review Request 26007: HIVE-8249 - Refactoring SparkPlan and SparkPlanGenerator [Spark Branch]

2014-09-24 Thread Chao Sun
/ql/exec/spark/SparkPlan.java 46e4b6d ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java 7ab2ca0 ql/src/java/org/apache/hadoop/hive/ql/exec/spark/UnionTran.java 546b448 Diff: https://reviews.apache.org/r/26007/diff/ Testing --- Thanks, Chao Sun

Review Request 25943: HIVE-8207 - Add .q tests for multi-table insertion [Spark Branch]

2014-09-23 Thread Chao Sun
/subquery_multiinsert.q.out PRE-CREATION ql/src/test/results/clientpositive/spark/table_access_keys_stats.q.out PRE-CREATION Diff: https://reviews.apache.org/r/25943/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25943: HIVE-8207 - Add .q tests for multi-table insertion [Spark Branch]

2014-09-23 Thread Chao Sun
/test/results/clientpositive/spark/ppd_transform.q.out PRE-CREATION ql/src/test/results/clientpositive/spark/subquery_multiinsert.q.out PRE-CREATION Diff: https://reviews.apache.org/r/25943/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
p.m., Chao Sun wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25394/ --- (Updated Sept. 18, 2014, 6:38 p.m.) Review

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
is this necessary, as you can find the chidren of lca later without the map. Cannot we just store lca here? Chao Sun wrote: The problem is because we are only generating one FS but multiple TSs. After the FS and the first TS is generated, the relation between child-parent is lost (since

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
/#review54004 --- On Sept. 18, 2014, 6:38 p.m., Chao Sun wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25394

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
it after the original op tree is broken into several trees. From each tree, we compile it into a spark task. In the end, we hook up parent child relation ship. The current logic is a little complicated and hard to understand. 2. Tests 3. Optimizations Chao Sun wrote: I agree. I

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
/SparkProcessAnalyzeTable.java 5fcaf64 ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkTableScanProcessor.java PRE-CREATION Diff: https://reviews.apache.org/r/25394/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
1bc55f4 Diff: https://reviews.apache.org/r/25394/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-19 Thread Chao Sun
patch. - Chao --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25394/#review54065 --- On Sept. 20, 2014, 1:33 a.m., Chao Sun wrote

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-18 Thread Chao Sun
://reviews.apache.org/r/25394/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-05 Thread Chao Sun
-CREATION Diff: https://reviews.apache.org/r/25394/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25280: Refactoring GraphTran to make it conform to SparkTran interface. [Spark Branch]

2014-09-05 Thread Chao Sun
-CREATION ql/src/test/results/clientpositive/spark/union27.q.out.sorted PRE-CREATION Diff: https://reviews.apache.org/r/25280/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25394: HIVE-7503: Support Hive's multi-table insert query with Spark [Spark Branch]

2014-09-05 Thread Chao Sun
, Chao Sun

Review Request 25404: NPE while reading null decimal value

2014-09-05 Thread Chao Sun
(ListSinkOperator.java:87) ... 19 more Diffs - common/src/java/org/apache/hadoop/hive/common/type/HiveDecimal.java 00ea481c2eed84de12815eedb079e965aa2ee701 Diff: https://reviews.apache.org/r/25404/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25404: NPE while reading null decimal value

2014-09-05 Thread Chao Sun
://reviews.apache.org/r/25404/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25404: NPE while reading null decimal value

2014-09-05 Thread Chao Sun
/java/org/apache/hadoop/hive/common/type/HiveDecimal.java 00ea481c2eed84de12815eedb079e965aa2ee701 common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java 769410d474fdc0ecbd63c7fe8944b2f6d23d5e5a Diff: https://reviews.apache.org/r/25404/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25404: NPE while reading null decimal value

2014-09-05 Thread Chao Sun
/HiveDecimal.java 00ea481c2eed84de12815eedb079e965aa2ee701 common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java 769410d474fdc0ecbd63c7fe8944b2f6d23d5e5a Diff: https://reviews.apache.org/r/25404/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 25404: NPE while reading null decimal value

2014-09-05 Thread Chao Sun
. To reply, visit: https://reviews.apache.org/r/25404/#review52529 --- On Sept. 5, 2014, 11:54 p.m., Chao Sun wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 25404: NPE while reading null decimal value

2014-09-05 Thread Chao Sun
/HiveDecimal.java 00ea481c2eed84de12815eedb079e965aa2ee701 common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java 769410d474fdc0ecbd63c7fe8944b2f6d23d5e5a Diff: https://reviews.apache.org/r/25404/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 24297: Spark Explain should give useful information on dependencies

2014-08-05 Thread Chao Sun
type. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java e238ff1 Diff: https://reviews.apache.org/r/24297/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 24297: Spark Explain should give useful information on dependencies

2014-08-05 Thread Chao Sun
, I'll make change to it. - Chao --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24297/#review49570 --- On Aug. 5, 2014, 6:09 a.m., Chao Sun

Re: Review Request 24297: Spark Explain should give useful information on dependencies

2014-08-05 Thread Chao Sun
this change might modify quite a few .q file tests. In which case it might be better to do a smaller change which only impacts Spark. Chao Sun wrote: I worried about that too - after a little grep I found it might affect quite a few places in MapWork. Perhaps we'll have to do something

Re: Review Request 24297: Spark Explain should give useful information on dependencies

2014-08-05 Thread Chao Sun
type. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java e238ff1 Diff: https://reviews.apache.org/r/24297/diff/ Testing --- Thanks, Chao Sun

Review Request 24352: StarterProject: Fix exception handling in POC code

2014-08-05 Thread Chao Sun
/hadoop/hive/ql/exec/spark/SparkClient.java 358cbc7 Diff: https://reviews.apache.org/r/24352/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 24352: StarterProject: Fix exception handling in POC code

2014-08-05 Thread Chao Sun
: https://reviews.apache.org/r/24352/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 24195: StarterProject: Move from assert to Guava Preconditions.* in Hive on Spark

2014-08-04 Thread Chao Sun
/diff/ Testing --- Thanks, Chao Sun

Review Request 24195: StarterProject: Move from assert to Guava Preconditions.* in Hive on Spark

2014-08-01 Thread Chao Sun
/hive/ql/plan/SparkWork.java 86d14f1 Diff: https://reviews.apache.org/r/24195/diff/ Testing --- Thanks, Chao Sun

Re: Review Request 24195: StarterProject: Move from assert to Guava Preconditions.* in Hive on Spark

2014-08-01 Thread Chao Sun
/apache/hadoop/hive/ql/parse/spark/SparkProcessAnalyzeTable.java 3a0f4c9 ql/src/java/org/apache/hadoop/hive/ql/plan/SparkWork.java 86d14f1 Diff: https://reviews.apache.org/r/24195/diff/ Testing --- Thanks, Chao Sun

Review Request 24127: Research to use groupby transformation to replace Hive existing partitionByKey and SparkCollector combination

2014-07-30 Thread Chao Sun
440dd93 ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTran.java 6aa732f Diff: https://reviews.apache.org/r/24127/diff/ Testing --- Thanks, Chao Sun

Review Request 23530: HIVE-6560: varchar and char types cannot be cast to binary

2014-07-15 Thread Chao Sun
/results/clientpositive/udf_binary.q.out PRE-CREATION Diff: https://reviews.apache.org/r/23530/diff/ Testing --- N/A Thanks, Chao Sun

<    1   2   3   4