[jira] [Updated] (HIVE-10009) LazyObjectInspectorFactory is not thread safe [Spark Branch]

2015-03-18 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10009: --- Affects Version/s: spark-branch LazyObjectInspectorFactory is not thread safe [Spark Branch]

[jira] [Commented] (HIVE-9970) Hive on spark

2015-03-16 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363377#comment-14363377 ] Jimmy Xiang commented on HIVE-9970: --- Any exception in your hive.log? If you can attach

[jira] [Updated] (HIVE-9993) Retrying task could use cached bad operators [Spark Branch]

2015-03-17 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9993: -- Attachment: HIVE-9993.1-spark.patch Retrying task could use cached bad operators [Spark Branch]

[jira] [Updated] (HIVE-10023) Fix more cache related concurrency issue [Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10023: --- Attachment: HIVE-10023.1-spark.patch Fix more cache related concurrency issue [Spark Branch]

[jira] [Commented] (HIVE-10023) Fix more cache related concurrency issue [Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370236#comment-14370236 ] Jimmy Xiang commented on HIVE-10023: Posted a patch that make sure we always use the

[jira] [Updated] (HIVE-10006) RSC has memory leak while execute multi queries.[Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10006: --- Attachment: HIVE-10006.4-spark.patch The test failures are not related. Discussed it with Xuefu a

[jira] [Commented] (HIVE-10006) RSC has memory leak while execute multi queries.[Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370704#comment-14370704 ] Jimmy Xiang commented on HIVE-10006: For the CombinedInputFormat, is it invoked in the

[jira] [Updated] (HIVE-10023) Fix more cache related concurrency issue [Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10023: --- Attachment: HIVE-10023.2-spark.patch Attached v2 that added space after the cast. Fix more cache

[jira] [Commented] (HIVE-10006) RSC has memory leak while execute multi queries.[Spark Branch]

2015-03-20 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371274#comment-14371274 ] Jimmy Xiang commented on HIVE-10006: Thanks for the clarification. Yes, we should be

[jira] [Updated] (HIVE-10006) RSC has memory leak while execute multi queries.[Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10006: --- Attachment: HIVE-10006.3-spark.patch RSC has memory leak while execute multi queries.[Spark Branch]

[jira] [Commented] (HIVE-10006) RSC has memory leak while execute multi queries.[Spark Branch]

2015-03-19 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369799#comment-14369799 ] Jimmy Xiang commented on HIVE-10006: The test failure (union31) is not related. Patch

[jira] [Updated] (HIVE-10009) LazyObjectInspectorFactory is not thread safe [Spark Branch]

2015-03-18 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10009: --- Fix Version/s: spark-branch LazyObjectInspectorFactory is not thread safe [Spark Branch]

[jira] [Updated] (HIVE-10009) LazyObjectInspectorFactory is not thread safe [Spark Branch]

2015-03-18 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10009: --- Attachment: HIVE-10009.1-spark.patch LazyObjectInspectorFactory is not thread safe [Spark Branch]

[jira] [Updated] (HIVE-9601) New Beeline queries will hang If Beeline terminates in-properly [Spark Branch]

2015-03-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9601: -- Attachment: HIVE-9601.1-spark.patch Re-attach the patch to trigger QA test. New Beeline queries will

[jira] [Updated] (HIVE-9601) New Beeline queries will hang If Beeline terminates in-properly [Spark Branch]

2015-03-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9601: -- Attachment: HIVE-9601.2-spark.patch Attached v2 that fixed unit test failures. The patch works only for

[jira] [Updated] (HIVE-9902) Map join small table files need more replications [Spark Branch]

2015-03-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9902: -- Attachment: HIVE-9902.1-spark.patch Map join small table files need more replications [Spark Branch]

[jira] [Updated] (HIVE-9929) StatsUtil#getAvailableMemory could return negative value

2015-03-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9929: -- Attachment: HIVE-9929.1.patch StatsUtil#getAvailableMemory could return negative value

[jira] [Updated] (HIVE-9935) Fix tests for java 1.8 [Spark Branch]

2015-03-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9935: -- Attachment: HIVE-9935.1-spark.patch Fix tests for java 1.8 [Spark Branch]

[jira] [Commented] (HIVE-9929) StatsUtil#getAvailableMemory could return negative value

2015-03-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357744#comment-14357744 ] Jimmy Xiang commented on HIVE-9929: --- This test is ok on my box.

[jira] [Updated] (HIVE-9935) Fix tests for java 1.8 [Spark Branch]

2015-03-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9935: -- Description: In spark branch, these tests don't have java 1.8 golden file: join0.q list_bucket_dml_2.q

[jira] [Commented] (HIVE-9935) Fix tests for java 1.8 [Spark Branch]

2015-03-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357826#comment-14357826 ] Jimmy Xiang commented on HIVE-9935: --- The results for 1.8 is similar to those for 1.7.

[jira] [Updated] (HIVE-9601) New Beeline queries will hang If Beeline terminates in-properly [Spark Branch]

2015-03-05 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9601: -- Attachment: HIVE-9601.1-spark.patch New Beeline queries will hang If Beeline terminates in-properly

[jira] [Commented] (HIVE-9601) New Beeline queries will hang If Beeline terminates in-properly [Spark Branch]

2015-03-05 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349327#comment-14349327 ] Jimmy Xiang commented on HIVE-9601: --- Attached a patch that is also on RB:

[jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on failed async queries

2015-03-05 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349331#comment-14349331 ] Jimmy Xiang commented on HIVE-9839: --- I attached a patch for HIVE-9601 which should be

[jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on async queries which fail at compile phase

2015-03-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350548#comment-14350548 ] Jimmy Xiang commented on HIVE-9839: --- I see. That means HIVE-9842 can not fix the leakage

[jira] [Commented] (HIVE-10073) Runtime exception when querying HBase with Spark [Spark Branch]

2015-03-24 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378743#comment-14378743 ] Jimmy Xiang commented on HIVE-10073: It looks like property hbase.mapred.outputtable

[jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on async queries which fail at compile phase

2015-03-24 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378764#comment-14378764 ] Jimmy Xiang commented on HIVE-9839: --- The patch looks good to me. However, the test is

[jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on async queries which fail at compile phase

2015-03-24 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379241#comment-14379241 ] Jimmy Xiang commented on HIVE-9839: --- I was wondering if we can use Mockito.mock instead

[jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on async queries which fail at compile phase

2015-03-25 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379998#comment-14379998 ] Jimmy Xiang commented on HIVE-9839: --- Thanks for making the change. +1 pending on test.

[jira] [Updated] (HIVE-10073) Runtime exception when querying HBase with Spark [Spark Branch]

2015-03-25 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10073: --- Attachment: HIVE-10073.1-spark.patch Runtime exception when querying HBase with Spark [Spark Branch]

[jira] [Assigned] (HIVE-10109) Index created in a different db can't be dropped

2015-03-30 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reassigned HIVE-10109: -- Assignee: Jimmy Xiang Index created in a different db can't be dropped

[jira] [Commented] (HIVE-10073) Runtime exception when querying HBase with Spark [Spark Branch]

2015-03-26 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382253#comment-14382253 ] Jimmy Xiang commented on HIVE-10073: [~xuefuz], I think it's an issue on Hive side. In

[jira] [Updated] (HIVE-10167) HS2 logs the server started only before the server is shut down

2015-03-31 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10167: --- Attachment: HIVE-10167.1.patch HS2 logs the server started only before the server is shut down

[jira] [Updated] (HIVE-10108) Index#getIndexTableName() returns db.index_table_name

2015-03-31 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10108: --- Attachment: HIVE-10108.1.patch These test failures are not related. Renamed the patch for trunk.

[jira] [Updated] (HIVE-10108) Index#getIndexTableName() returns db.index_table_name

2015-03-31 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10108: --- Attachment: HIVE-10108.2.patch Attached patch v2 that's rebased to trunk latest.

[jira] [Commented] (HIVE-9839) HiveServer2 leaks OperationHandle on async queries which fail at compile phase

2015-03-23 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376078#comment-14376078 ] Jimmy Xiang commented on HIVE-9839: --- [~ychena], to clarify, I submitted a patch for

[jira] [Commented] (HIVE-10108) Index#getIndexTableName() returns db.index_table_name

2015-04-01 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390793#comment-14390793 ] Jimmy Xiang commented on HIVE-10108: The patch (1) fixed Index#getIndexTableName() to

[jira] [Updated] (HIVE-9830) Test auto_sortmerge_join_8 is flaky [Spark Branch]

2015-03-02 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9830: -- Summary: Test auto_sortmerge_join_8 is flaky [Spark Branch] (was: Test auto_sortmerge_join_8 is flaky)

[jira] [Updated] (HIVE-9830) Map join could dump a small table multiple times [Spark Branch]

2015-03-02 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9830: -- Attachment: HIVE-9830.2-spark.patch Map join could dump a small table multiple times [Spark Branch]

[jira] [Updated] (HIVE-9861) Add spark-assembly on Hive's classpath [Spark Branch]

2015-03-04 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9861: -- Attachment: HIVE-9861.1-spark.patch Add spark-assembly on Hive's classpath [Spark Branch]

[jira] [Reopened] (HIVE-9601) New Beeline queries will hang If Beeline terminates in-properly [Spark Branch]

2015-03-03 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reopened HIVE-9601: --- Assignee: Jimmy Xiang I think we need to fix this. These properties are good. However, it is really

[jira] [Commented] (HIVE-9781) Utilize spark.kryo.registrator [Spark Branch]

2015-02-25 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336683#comment-14336683 ] Jimmy Xiang commented on HIVE-9781: --- Cool. Let me handle the other place which doesn't

[jira] [Assigned] (HIVE-9781) Utilize spark.kryo.registrator [Spark Branch]

2015-02-25 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reassigned HIVE-9781: - Assignee: Jimmy Xiang Utilize spark.kryo.registrator [Spark Branch]

[jira] [Updated] (HIVE-9781) Utilize spark.kryo.classesToRegister [Spark Branch]

2015-02-26 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9781: -- Attachment: HIVE-9781.2.patch Utilize spark.kryo.classesToRegister [Spark Branch]

[jira] [Updated] (HIVE-9781) Utilize spark.kryo.classesToRegister [Spark Branch]

2015-02-26 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9781: -- Attachment: HIVE-9781.3.patch Utilize spark.kryo.classesToRegister [Spark Branch]

[jira] [Updated] (HIVE-9781) Utilize spark.kryo.classesToRegister [Spark Branch]

2015-02-26 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-9781: -- Attachment: HIVE-9781.4.patch Utilize spark.kryo.classesToRegister [Spark Branch]

[jira] [Commented] (HIVE-8851) Broadcast files for small tables via SparkContext.addFile() and SparkFiles.get() [Spark Branch]

2015-03-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351025#comment-14351025 ] Jimmy Xiang commented on HIVE-8851: --- Patch v2 is on RB:

[jira] [Commented] (HIVE-10031) Modify the using of jobConf variable in ParquetRecordReaderWrapper constructor

2015-03-23 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376123#comment-14376123 ] Jimmy Xiang commented on HIVE-10031: Good. If Parquet already has its own copy of

[jira] [Resolved] (HIVE-10065) jp

2015-03-23 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang resolved HIVE-10065. Resolution: Invalid jp -- Key: HIVE-10065 URL:

[jira] [Updated] (HIVE-10146) Not count session as idle if query is running

2015-04-02 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10146: --- Attachment: HIVE-10146.1.patch Not count session as idle if query is running

[jira] [Commented] (HIVE-10108) Index#getIndexTableName() returns db.index_table_name

2015-04-02 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393382#comment-14393382 ] Jimmy Xiang commented on HIVE-10108: Chatted with Xuefu, Chao. We think it's ok to

[jira] [Updated] (HIVE-10146) Not count session as idle if query is running

2015-04-03 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10146: --- Attachment: HIVE-10146.4.patch Attached v4 that fixed the description as Lefty suggested. Thanks.

[jira] [Commented] (HIVE-8858) Visualize generated Spark plan [Spark Branch]

2015-04-21 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505572#comment-14505572 ] Jimmy Xiang commented on HIVE-8858: --- Cool, looks good to me. +1 Visualize generated

[jira] [Updated] (HIVE-10302) Load small tables (for map join) in executor memory only once [Spark Branch]

2015-04-23 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10302: --- Attachment: HIVE-10302.2-spark.patch Load small tables (for map join) in executor memory only once

[jira] [Updated] (HIVE-10302) Load small tables (for map join) in executor memory only once [Spark Branch]

2015-04-23 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10302: --- Attachment: HIVE-10302.3-spark.patch Load small tables (for map join) in executor memory only once

[jira] [Updated] (HIVE-10473) Spark client is recreated even spark configuration is not changed

2015-04-23 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10473: --- Attachment: HIVE-10473.1.patch Spark client is recreated even spark configuration is not changed

[jira] [Commented] (HIVE-10473) Spark client is recreated even spark configuration is not changed

2015-04-25 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512601#comment-14512601 ] Jimmy Xiang commented on HIVE-10473: If the new value is null, the set will fail at

[jira] [Updated] (HIVE-10302) Load small tables (for map join) in executor memory only once [Spark Branch]

2015-04-24 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10302: --- Attachment: HIVE-10302.4-spark.patch Thanks a lot for the discussion and review. Yes, the spark

[jira] [Commented] (HIVE-8858) Visualize generated Spark plan [Spark Branch]

2015-04-21 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505272#comment-14505272 ] Jimmy Xiang commented on HIVE-8858: --- Looks good. Can this be part of explain extended? If

[jira] [Updated] (HIVE-10473) Spark client is recreated even spark configuration is not changed

2015-04-24 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10473: --- Attachment: HIVE-10473.1-spark.patch Spark client is recreated even spark configuration is not

[jira] [Commented] (HIVE-10499) Ensure Session/ZooKeeperClient instances are closed

2015-04-28 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517410#comment-14517410 ] Jimmy Xiang commented on HIVE-10499: The test failures are not related. They fail in

[jira] [Commented] (HIVE-10307) Support to use number literals in partition column

2015-04-27 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515161#comment-14515161 ] Jimmy Xiang commented on HIVE-10307: +1 Support to use number literals in partition

[jira] [Updated] (HIVE-10499) Ensure Session/ZooKeeperClient instances are closed

2015-04-27 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10499: --- Attachment: HIVE-10499.patch Ensure Session/ZooKeeperClient instances are closed

[jira] [Commented] (HIVE-10499) Ensure Session/ZooKeeperClient instances are closed

2015-04-28 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517532#comment-14517532 ] Jimmy Xiang commented on HIVE-10499: Sure. The patch is on RB:

[jira] [Commented] (HIVE-10527) NPE in SparkUtilities::isDedicatedCluster [Spark Branch]

2015-04-29 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14519503#comment-14519503 ] Jimmy Xiang commented on HIVE-10527: Good catch. +1 NPE in

[jira] [Commented] (HIVE-10626) Spark paln need to be updated [Spark Branch]

2015-05-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531320#comment-14531320 ] Jimmy Xiang commented on HIVE-10626: I see. In order to use toString, we have to add

[jira] [Commented] (HIVE-10626) Spark paln need to be updated [Spark Branch]

2015-05-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531382#comment-14531382 ] Jimmy Xiang commented on HIVE-10626: Thanks for making the change. +1 pending on test.

[jira] [Commented] (HIVE-10626) Spark paln need to be updated [Spark Branch]

2015-05-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531275#comment-14531275 ] Jimmy Xiang commented on HIVE-10626: Why do we need SparkPlan.toString()? Spark paln

[jira] [Commented] (HIVE-10626) Spark paln need to be updated [Spark Branch]

2015-05-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531101#comment-14531101 ] Jimmy Xiang commented on HIVE-10626: Is it better to make sparkPlan (StringBuilder) a

[jira] [Updated] (HIVE-10060) Provide more informative stage description in Spark Web UI [Spark Branch]

2015-05-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10060: --- Assignee: (was: Jimmy Xiang) Provide more informative stage description in Spark Web UI [Spark

[jira] [Commented] (HIVE-10626) Spark paln need to be updated [Spark Branch]

2015-05-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530964#comment-14530964 ] Jimmy Xiang commented on HIVE-10626: Can you use StringBuilder in constructing the

[jira] [Assigned] (HIVE-10060) Provide more informative stage description in Spark Web UI [Spark Branch]

2015-05-05 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reassigned HIVE-10060: -- Assignee: Jimmy Xiang Provide more informative stage description in Spark Web UI [Spark

[jira] [Updated] (HIVE-10721) SparkSessionManagerImpl leaks SparkSessions [Spark Branch]

2015-05-15 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10721: --- Attachment: HIVE-10721.1-spark.patch SparkSessionManagerImpl leaks SparkSessions [Spark Branch]

[jira] [Commented] (HIVE-10718) Update committer list

2015-05-18 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548198#comment-14548198 ] Jimmy Xiang commented on HIVE-10718: [~Ferd], thanks for adding me. My details are

[jira] [Updated] (HIVE-10302) Cache small tables in memory [Spark Branch]

2015-04-15 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10302: --- Attachment: HIVE-10302.spark-1.patch Cache small tables in memory [Spark Branch]

[jira] [Commented] (HIVE-10302) Cache small tables in memory [Spark Branch]

2015-04-15 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497390#comment-14497390 ] Jimmy Xiang commented on HIVE-10302: The patch is on RB:

[jira] [Updated] (HIVE-10365) First job fails with StackOverflowError [Spark Branch]

2015-04-16 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10365: --- Summary: First job fails with StackOverflowError [Spark Branch] (was: First job fails with

[jira] [Commented] (HIVE-10313) Literal Decimal ExprNodeConstantDesc should contain value of HiveDecimal instead of String

2015-04-16 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498280#comment-14498280 ] Jimmy Xiang commented on HIVE-10313: +1 Literal Decimal ExprNodeConstantDesc should

[jira] [Resolved] (HIVE-10365) First job fails with StackOverflowError [Spark Branch]

2015-04-17 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang resolved HIVE-10365. Resolution: Not A Problem Not a code issue. It works after increasing the thread stack size.

[jira] [Commented] (HIVE-10206) Improve Alter Table to not initialize Serde unnecessarily

2015-04-06 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481431#comment-14481431 ] Jimmy Xiang commented on HIVE-10206: +1. Looks good to me too. Improve Alter Table

[jira] [Commented] (HIVE-10265) Hive CLI crashes on != inequality

2015-04-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487869#comment-14487869 ] Jimmy Xiang commented on HIVE-10265: +1 Hive CLI crashes on != inequality

[jira] [Commented] (HIVE-10209) FetchTask with VC may fail because ExecMapper.done is true

2015-04-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488018#comment-14488018 ] Jimmy Xiang commented on HIVE-10209: +1 FetchTask with VC may fail because

[jira] [Commented] (HIVE-10956) HS2 leaks HMS connections

2015-06-08 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577731#comment-14577731 ] Jimmy Xiang commented on HIVE-10956: New patch does not use this threadlocal any more.

[jira] [Commented] (HIVE-6991) History not able to disable/enable after session started

2015-06-08 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578076#comment-14578076 ] Jimmy Xiang commented on HIVE-6991: --- +1. You tested it on a live cluster and it works for

[jira] [Commented] (HIVE-10415) hive.start.cleanup.scratchdir configuration is not taking effect

2015-06-08 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578081#comment-14578081 ] Jimmy Xiang commented on HIVE-10415: +1. Ok, this setting is off by default.

[jira] [Updated] (HIVE-10956) HS2 leaks HMS connections

2015-06-05 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10956: --- Attachment: HIVE-10956.1.patch HS2 leaks HMS connections -

[jira] [Commented] (HIVE-10956) HS2 leaks HMS connections

2015-06-08 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577401#comment-14577401 ] Jimmy Xiang commented on HIVE-10956: We have a pool of thrift worker threads. If idle

[jira] [Commented] (HIVE-10415) hive.start.cleanup.scratchdir configuration is not taking effect

2015-06-03 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571229#comment-14571229 ] Jimmy Xiang commented on HIVE-10415: If multiple HS2 instances start up, is there any

[jira] [Commented] (HIVE-6991) History not able to disable/enable after session started

2015-06-03 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571241#comment-14571241 ] Jimmy Xiang commented on HIVE-6991: --- Can the user change the setting and start a new

[jira] [Updated] (HIVE-10956) HS2 leaks HMS connections

2015-06-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10956: --- Attachment: HIVE-10956.2.patch Attached patch v2. It is also on RB:

[jira] [Updated] (HIVE-10956) HS2 leaks HMS connections

2015-06-09 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10956: --- Attachment: HIVE-10956.3.patch HS2 leaks HMS connections -

[jira] [Commented] (HIVE-10976) Redundant HiveMetaStore connect check in HS2 CLIService start

2015-06-10 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580586#comment-14580586 ] Jimmy Xiang commented on HIVE-10976: +1 Redundant HiveMetaStore connect check in HS2

[jira] [Commented] (HIVE-11109) Replication factor is not properly set in SparkHashTableSinkOperator [Spark Branch]

2015-06-25 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-11109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601418#comment-14601418 ] Jimmy Xiang commented on HIVE-11109: +1 Replication factor is not properly set in

[jira] [Commented] (HIVE-10976) Redundant HiveMetaStore connect check in HS2 CLIService start

2015-06-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14582135#comment-14582135 ] Jimmy Xiang commented on HIVE-10976: [~ctang.ma], is it possible to remove the whole

[jira] [Commented] (HIVE-10976) Redundant HiveMetaStore connect check in HS2 CLIService start

2015-06-11 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14582190#comment-14582190 ] Jimmy Xiang commented on HIVE-10976: I see. The other choice is to remove the stop()

[jira] [Updated] (HIVE-10956) HS2 leaks HMS connections

2015-06-10 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-10956: --- Attachment: HIVE-10956.4.patch Changed the patch a little. Not synchronized for close(). It should be

[jira] [Commented] (HIVE-10863) Merge trunk to Spark branch 5/28/2015 [Spark Branch]

2015-05-29 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565035#comment-14565035 ] Jimmy Xiang commented on HIVE-10863: I resolved the mj.patch conflicts and pushed it

[jira] [Commented] (HIVE-10870) Merge Spark branch to trunk 5/29/2015

2015-06-01 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567568#comment-14567568 ] Jimmy Xiang commented on HIVE-10870: I tried udaf_histogram_numeric which is fine

[jira] [Commented] (HIVE-10302) Load small tables (for map join) in executor memory only once [Spark Branch]

2015-06-01 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568197#comment-14568197 ] Jimmy Xiang commented on HIVE-10302: The file was lost during rebasing. I pushed it to

[jira] [Commented] (HIVE-10877) TestUtil class name confuses ptest2

2015-06-01 Thread Jimmy Xiang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568208#comment-14568208 ] Jimmy Xiang commented on HIVE-10877: [~thejas], I got compilation error. Forgot to add

  1   2   3   4   >