[jira] [Commented] (HIVE-18340) Dynamic Min-Max/BloomFilter runtime-filtering in HoS

2018-02-07 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356365#comment-16356365 ] liyunzhang commented on HIVE-18340: --- [~stakiar]: {quote} Hive-on-Tez's has an implementa

[jira] [Commented] (HIVE-14285) Explain outputs: map-entry ordering of non-primitive objects.

2018-02-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359784#comment-16359784 ] liyunzhang commented on HIVE-14285: --- [~kgyrtkirk]: I found some problem with {{ExplainTa

[jira] [Commented] (HIVE-14285) Explain outputs: map-entry ordering of non-primitive objects.

2018-02-11 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360203#comment-16360203 ] liyunzhang commented on HIVE-14285: --- [~kgyrtkirk]: I want to ask a question that the wo

[jira] [Commented] (HIVE-14285) Explain outputs: map-entry ordering of non-primitive objects.

2018-02-12 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361692#comment-16361692 ] liyunzhang commented on HIVE-14285: --- [~kgyrtkirk]: {code} I'm starting to remember...I

[jira] [Updated] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2018-02-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17879: -- Attachment: HIVE-17879.patch > Can not find java.sql.date in JDK9 when building hive > -

[jira] [Commented] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2018-02-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363616#comment-16363616 ] liyunzhang commented on HIVE-17879: --- Most work of HIVE-17879.patch is belongs to https

[jira] [Comment Edited] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2018-02-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363616#comment-16363616 ] liyunzhang edited comment on HIVE-17879 at 2/14/18 8:08 AM: Mo

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-08 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283241#comment-16283241 ] liyunzhang commented on HIVE-17486: --- [~xuefuz] and [~lirui]: the second problem I found

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285468#comment-16285468 ] liyunzhang commented on HIVE-18148: --- [~lirui]: I can not reproduce because you did not p

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285470#comment-16285470 ] liyunzhang commented on HIVE-17486: --- [~lirui] and [~xuefuz]: Please help view the [First

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285473#comment-16285473 ] liyunzhang commented on HIVE-18150: --- [~ferd] : as we have finished the review of the pat

[jira] [Commented] (HIVE-18150) Upgrade Spark Version to 2.2.0

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285549#comment-16285549 ] liyunzhang commented on HIVE-18150: --- [~Ferd]: [~stakiar] has uploaded the spark-2.2.0 p

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285551#comment-16285551 ] liyunzhang commented on HIVE-17486: --- [~lirui]: {quote} could you provide more design and

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285558#comment-16285558 ] liyunzhang commented on HIVE-17486: --- [~xuefuz] {quote} It seems that we can do so whene

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-10 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285558#comment-16285558 ] liyunzhang edited comment on HIVE-17486 at 12/11/17 6:41 AM: -

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-12 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288814#comment-16288814 ] liyunzhang commented on HIVE-18148: --- [~lirui]: tried to use the example you provided. {

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288864#comment-16288864 ] liyunzhang commented on HIVE-18148: --- {quote} you can just run the new test with master b

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288899#comment-16288899 ] liyunzhang commented on HIVE-18148: --- sorry i have not run the qtest. just copy the query

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290243#comment-16290243 ] liyunzhang commented on HIVE-18148: --- [~lirui]: If this is found in common join please a

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-13 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290256#comment-16290256 ] liyunzhang commented on HIVE-18148: --- {code} grep -C2 "hive.auto.convert.join" $HIVE_SO

[jira] [Commented] (HIVE-18282) Spark tar is downloaded every time for itest

2017-12-14 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292052#comment-16292052 ] liyunzhang commented on HIVE-18282: --- [~stakiar]: http://d3jw87u4immizc.cloudfront.net/s

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.2.patch [~lirui] and [~xuefuz]: I have updated HIVE-17486.2.patch and designed

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292203#comment-16292203 ] liyunzhang edited comment on HIVE-17486 at 12/15/17 9:15 AM: -

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292286#comment-16292286 ] liyunzhang commented on HIVE-17486: --- [~xuefuz]: before you mentioned that the reason to

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Status: Patch Available (was: Open) > Enable SharedWorkOptimizer in tez on HOS > --

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-15 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.3.patch add spark_optimize_shared_work.q.out and update HIVE-17486.3.patch. Trigg

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-12-16 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.4.patch update HIVE-17486.4.patch to fix the compilation error. > Enable SharedW

[jira] [Assigned] (HIVE-18289) Fix jar dependency when enable rdd cache in Hive on Spark

2017-12-17 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18289: - > Fix jar dependency when enable rdd cache in Hive on Spark > ---

[jira] [Assigned] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-18 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18301: - > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-18 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Description: Before IOContext problem is found in MapTran when spark rdd cache is enabled in HIVE-8920.

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-18 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Description: Before IOContext problem is found in MapTran when spark rdd cache is enabled in HIVE-8920.

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297875#comment-16297875 ] liyunzhang commented on HIVE-18148: --- sorry for reply late. still have 1 question about t

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297897#comment-16297897 ] liyunzhang commented on HIVE-18301: --- yes. I can achieve the merge the same tables into

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297897#comment-16297897 ] liyunzhang edited comment on HIVE-18301 at 12/20/17 5:23 AM: -

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297899#comment-16297899 ] liyunzhang commented on HIVE-18148: --- I can understand the reason why you need to remove

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297921#comment-16297921 ] liyunzhang commented on HIVE-18148: --- it's better to remove the DPP which the statistics

[jira] [Commented] (HIVE-18289) Fix jar dependency when enable rdd cache in Hive on Spark

2017-12-19 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297977#comment-16297977 ] liyunzhang commented on HIVE-18289: --- if running in parquet, the exception is {code} Job

[jira] [Commented] (HIVE-18148) NPE in SparkDynamicPartitionPruningResolver

2017-12-20 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299384#comment-16299384 ] liyunzhang commented on HIVE-18148: --- +1 > NPE in SparkDynamicPartitionPruningResolver >

[jira] [Resolved] (HIVE-17755) NPE exception when running TestAcidOnTez#testGetSplitsLocks with "

2017-12-20 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang resolved HIVE-17755. --- Resolution: Not A Problem > NPE exception when running TestAcidOnTez#testGetSplitsLocks with " > -

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-21 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300999#comment-16300999 ] liyunzhang commented on HIVE-18301: --- The reason why have this NPE is because [org.apach

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-21 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16301027#comment-16301027 ] liyunzhang commented on HIVE-18301: --- [~xuefuz],[~lirui]: Is there any way to store IOcon

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-22 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16301713#comment-16301713 ] liyunzhang commented on HIVE-18301: --- [~xuefuz],[~csun]: I read jiras about MapInput IOCo

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2017-12-27 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304359#comment-16304359 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote} My understanding is these informa

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307619#comment-16307619 ] liyunzhang commented on HIVE-18301: --- thanks for Rui's comment, will spend some time to i

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308995#comment-16308995 ] liyunzhang commented on HIVE-17486: --- [~stakiar]: the original purpose to change M->R to

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308994#comment-16308994 ] liyunzhang commented on HIVE-17486: --- [~stakiar]: the original purpose to change M->R to

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308995#comment-16308995 ] liyunzhang edited comment on HIVE-17486 at 1/3/18 1:40 AM: --- [~st

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: HIVE-17486.5.patch > Enable SharedWorkOptimizer in tez on HOS >

[jira] [Issue Comment Deleted] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2018-01-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Comment: was deleted (was: [~stakiar]: the original purpose to change M->R to M->M->R is to let Combine

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16309361#comment-16309361 ] liyunzhang commented on HIVE-18301: --- Here some update about NPE: the normal case when e

[jira] [Commented] (HIVE-17573) LLAP: JDK9 support fixes

2018-01-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16310797#comment-16310797 ] liyunzhang commented on HIVE-17573: --- [~gopalv]: It seems that on tpch_text_10(10g), ther

[jira] [Commented] (HIVE-17573) LLAP: JDK9 support fixes

2018-01-04 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312596#comment-16312596 ] liyunzhang commented on HIVE-17573: --- [~gopalv]: thanks for your reply and tool. bq.JDK

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-17 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330190#comment-16330190 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote}My understanding is if the HadoopR

[jira] [Commented] (HIVE-8436) Modify SparkWork to split works with multiple child works [Spark Branch]

2018-01-21 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333870#comment-16333870 ] liyunzhang commented on HIVE-8436: -- [~csun]:  can you spend some time to explain why need

[jira] [Updated] (HIVE-18289) Support Parquet/Orc format when enable rdd cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18289: -- Summary: Support Parquet/Orc format when enable rdd cache in Hive on Spark (was: Fix jar dependency whe

[jira] [Commented] (HIVE-8436) Modify SparkWork to split works with multiple child works [Spark Branch]

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336431#comment-16336431 ] liyunzhang commented on HIVE-8436: -- [~csun]:  thanks for reply, {quote} without the copy

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336941#comment-16336941 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote} And we won't be able to tell the

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.patch > Investigate to enable MapInput cache in Hive on Spark > -

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336946#comment-16336946 ] liyunzhang commented on HIVE-18301: --- In HIVE-18301.patch, it provides one solution to tr

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336946#comment-16336946 ] liyunzhang edited comment on HIVE-18301 at 1/24/18 5:39 AM: In

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-23 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336946#comment-16336946 ] liyunzhang edited comment on HIVE-18301 at 1/24/18 5:47 AM: In

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338586#comment-16338586 ] liyunzhang commented on HIVE-18301: --- [~lirui]: {quote} does the proposed solution mean

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338586#comment-16338586 ] liyunzhang edited comment on HIVE-18301 at 1/25/18 4:05 AM: [~

[jira] [Commented] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached

2018-01-25 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340647#comment-16340647 ] liyunzhang commented on HIVE-18378: --- [~stakiar]: can you assign this jira to me ? I gues

[jira] [Assigned] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached

2018-01-25 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-18378: - Assignee: liyunzhang > Explain plan should show if a Map/Reduce Work is being cached > --

[jira] [Comment Edited] (HIVE-18378) Explain plan should show if a Map/Reduce Work is being cached

2018-01-25 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340647#comment-16340647 ] liyunzhang edited comment on HIVE-18378 at 1/26/18 7:21 AM: [~

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.1.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Status: Patch Available (was: Open) HIVE-18301.1.patch to trigger Hive QA > Investigate to enable MapI

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: (was: HIVE-18301.1.patch) > Investigate to enable MapInput cache in Hive on Spark >

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.1.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-26 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340717#comment-16340717 ] liyunzhang commented on HIVE-18301: --- for the problem it may make the RDD bigger: origin

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-31 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346418#comment-16346418 ] liyunzhang commented on HIVE-18301: --- [~lirui]: thanks for question. {quote} The IOCon

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-01-31 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.2.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Commented] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350077#comment-16350077 ] liyunzhang commented on HIVE-18301: --- [~xuefuz]: {quote} Instead of putting the input pa

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350077#comment-16350077 ] liyunzhang edited comment on HIVE-18301 at 2/2/18 9:54 AM: --- [~xu

[jira] [Updated] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-18301: -- Attachment: HIVE-18301.3.patch > Investigate to enable MapInput cache in Hive on Spark > ---

[jira] [Comment Edited] (HIVE-18301) Investigate to enable MapInput cache in Hive on Spark

2018-02-02 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350077#comment-16350077 ] liyunzhang edited comment on HIVE-18301 at 2/2/18 9:59 AM: --- [~xu

[jira] [Commented] (HIVE-17193) HoS: don't combine map works that are targets of different DPPs

2017-10-22 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16214728#comment-16214728 ] liyunzhang commented on HIVE-17193: --- [~lirui]: {quote} 1. The simplest solution is, if t

[jira] [Commented] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2017-10-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216538#comment-16216538 ] liyunzhang commented on HIVE-17879: --- [~kgyrtkirk]:thanks! i have build successfully by u

[jira] [Commented] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2017-10-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216570#comment-16216570 ] liyunzhang commented on HIVE-17879: --- [~kgyrtkirk]: have you built hadoop with jdk9? I h

[jira] [Commented] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2017-10-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217983#comment-16217983 ] liyunzhang commented on HIVE-17879: --- [~kgyrtkirk]: thanks for your suggestion. will try.

[jira] [Comment Edited] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2017-10-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217983#comment-16217983 ] liyunzhang edited comment on HIVE-17879 at 10/25/17 3:31 AM: -

[jira] [Commented] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2017-10-24 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16218084#comment-16218084 ] liyunzhang commented on HIVE-17879: --- [~gopalv]: {quote} the test was using build artifac

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-10-27 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221989#comment-16221989 ] liyunzhang commented on HIVE-17486: --- mapjoin.q {code} set hive.mapred.mode=nonstrict; se

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-10-27 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221992#comment-16221992 ] liyunzhang commented on HIVE-17486: --- In tez, Map can be connected 2 Reducers while in sp

[jira] [Resolved] (HIVE-17879) Can not find java.sql.date in JDK9 when building hive

2017-10-30 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang resolved HIVE-17879. --- Resolution: Duplicate duplicated with sub jiras under HIVE-17632. > Can not find java.sql.date in JD

[jira] [Commented] (HIVE-16602) Implement shared scans with Tez

2017-10-31 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233668#comment-16233668 ] liyunzhang commented on HIVE-16602: --- [~jcamachorodriguez]: I indeed found the performan

[jira] [Updated] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17486: -- Attachment: scanshare.after.svg scanshare.before.svg > Enable SharedWorkOptimizer in tez

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235054#comment-16235054 ] liyunzhang commented on HIVE-17486: --- Now HoS does not support multiple edge between two

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235094#comment-16235094 ] liyunzhang commented on HIVE-17486: --- [~xuefuz]: {quote} My gut feeling is that this ne

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235125#comment-16235125 ] liyunzhang commented on HIVE-17486: --- [~lirui]: {quote} My understanding is HoS also supp

[jira] [Comment Edited] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235125#comment-16235125 ] liyunzhang edited comment on HIVE-17486 at 11/2/17 3:35 AM: [~

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-01 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235131#comment-16235131 ] liyunzhang commented on HIVE-17486: --- [~lirui]: what i want to ask is there any possiblit

[jira] [Assigned] (HIVE-17973) Fix small bug in multi_insert_union_src.q

2017-11-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang reassigned HIVE-17973: - > Fix small bug in multi_insert_union_src.q > - > >

[jira] [Updated] (HIVE-17973) Fix small bug in multi_insert_union_src.q

2017-11-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17973: -- Attachment: HIVE-17973.patch > Fix small bug in multi_insert_union_src.q > -

[jira] [Updated] (HIVE-17973) Fix small bug in multi_insert_union_src.q

2017-11-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17973: -- Status: Patch Available (was: Open) > Fix small bug in multi_insert_union_src.q > -

[jira] [Commented] (HIVE-17973) Fix small bug in multi_insert_union_src.q

2017-11-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237188#comment-16237188 ] liyunzhang commented on HIVE-17973: --- [~lirui], [~xuefuz]: can you help review? thanks!

[jira] [Updated] (HIVE-17973) Fix small bug in multi_insert_union_src.q

2017-11-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17973: -- Description: in ql\src\test\queries\clientpositive\multi_insert_union_src.q, There are two problems in

[jira] [Commented] (HIVE-17486) Enable SharedWorkOptimizer in tez on HOS

2017-11-03 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237271#comment-16237271 ] liyunzhang commented on HIVE-17486: --- [~lirui]: {quote} I also think that's possible in t

[jira] [Updated] (HIVE-17973) Fix small bug in multi_insert_union_src.q

2017-11-05 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated HIVE-17973: -- Attachment: HIVE-17973.2.patch update the patch, need also modify $HIVE_SOURCE/ql/src/test/results/clie

  1   2   >