[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-06-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503674#comment-16503674 ] Sahil Takiar commented on HIVE-18533: - Thanks for catching that. Pushed an addendum commit to fix the

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-06-06 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502983#comment-16502983 ] Rui Li commented on HIVE-18533: --- [~stakiar], please take a look at the Yetus report. I think some warnings

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-06-04 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501063#comment-16501063 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-06-04 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500969#comment-16500969 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-06-01 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497842#comment-16497842 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-06-01 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497818#comment-16497818 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-29 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494623#comment-16494623 ] Rui Li commented on HIVE-18533: --- [~stakiar] sorry I overlooked your previous comments. +1 pending tests bq.

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-29 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494016#comment-16494016 ] Sahil Takiar commented on HIVE-18533: - [~lirui] any other comments? > Add option to use

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-15 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476305#comment-16476305 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-15 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476249#comment-16476249 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474738#comment-16474738 ] Sahil Takiar commented on HIVE-18533: - [~lirui] attached an updated patch and updated the RB.

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-14 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474138#comment-16474138 ] Rui Li commented on HIVE-18533: --- Seems the new qtest has unstable output. Please look into it. Some more

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-14 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473955#comment-16473955 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-14 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473926#comment-16473926 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473684#comment-16473684 ] Sahil Takiar commented on HIVE-18533: - [~lirui] good catch. Updated. > Add option to use

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-12 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473112#comment-16473112 ] Rui Li commented on HIVE-18533: --- Thanks for the update [~stakiar]. There's some issue with

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-11 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471743#comment-16471743 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-11 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471709#comment-16471709 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-09 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469407#comment-16469407 ] Sahil Takiar commented on HIVE-18533: - [~lirui] attached an updated patch that uses {{FutureTask}}

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-07 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466816#comment-16466816 ] Rui Li commented on HIVE-18533: --- Hi [~stakiar], my concern for SparkLauncherFuture is mainly about the

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466648#comment-16466648 ] Sahil Takiar commented on HIVE-18533: - [~lirui] yeah that might be slightly simpler. Although the

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-07 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465624#comment-16465624 ] Rui Li commented on HIVE-18533: --- Hi [~stakiar], for SparkLauncherSparkClient, how about we use a thread to

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-07 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465530#comment-16465530 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-06 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465462#comment-16465462 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-06 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465355#comment-16465355 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-05 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464849#comment-16464849 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-03 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462702#comment-16462702 ] Sahil Takiar commented on HIVE-18533: - I think {{Future}} is a better abstraction than a {{Thread}} in

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-03 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462612#comment-16462612 ] Rui Li commented on HIVE-18533: --- Hi [~stakiar], could you explain why we now need to monitor the driver with

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-03 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462109#comment-16462109 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-03 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462054#comment-16462054 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-05-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461318#comment-16461318 ] Rui Li commented on HIVE-18533: --- [~stakiar], thanks for working on this and sorry for the delay. I had a

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-04-27 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456664#comment-16456664 ] Sahil Takiar commented on HIVE-18533: - [~lirui] could you take a look? Below is a brief description of

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-27 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415447#comment-16415447 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-27 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415398#comment-16415398 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-25 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413364#comment-16413364 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-25 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413320#comment-16413320 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-25 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413100#comment-16413100 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-25 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413059#comment-16413059 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-23 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411987#comment-16411987 ] Sahil Takiar commented on HIVE-18533: - Thanks filed SPARK-23785 to fix the issue. > Add option to use

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410347#comment-16410347 ] Marcelo Vanzin commented on HIVE-18533: --- That looks like a bug in Spark's {{LauncherBackend}}. It

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-21 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408654#comment-16408654 ] Sahil Takiar commented on HIVE-18533: - [~vanzin] I'm seeing a race condition in one of the tests that

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-15 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400291#comment-16400291 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-15 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400236#comment-16400236 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-13 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396682#comment-16396682 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-13 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396654#comment-16396654 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-11 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394826#comment-16394826 ] Hive QA commented on HIVE-18533: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-11 Thread Hive QA (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394813#comment-16394813 ] Hive QA commented on HIVE-18533: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-11 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394793#comment-16394793 ] Sahil Takiar commented on HIVE-18533: - Attaching WIP patch with the {{SparkLauncher}} option enabled