[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599265#comment-16599265 ] Sahil Takiar commented on HIVE-14162: - [~Tagar] it was pushed to master, which is Hive 4.0.0. I can

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599261#comment-16599261 ] Ruslan Dautkhanov commented on HIVE-14162: -- Thank you. Which branch this is going to be applied?

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599260#comment-16599260 ] Sahil Takiar commented on HIVE-14162: - {{TestCliDriver.testCliDriver[test_teradatabinaryfile]}} was

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599085#comment-16599085 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599055#comment-16599055 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598793#comment-16598793 ] Sahil Takiar commented on HIVE-14162: - Thanks Adam. Test failures don't look related, but rebased the

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598667#comment-16598667 ] Adam Szita commented on HIVE-14162: --- Thanks [~stakiar]. I think it's good to go now: +1 on

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598368#comment-16598368 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598340#comment-16598340 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597637#comment-16597637 ] Sahil Takiar commented on HIVE-14162: - [~szita] thanks for taking a look. Addressed your comments and

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597413#comment-16597413 ] Adam Szita commented on HIVE-14162: --- Hi [~stakiar], I left some comments on RB > Allow disabling of

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588781#comment-16588781 ] Sahil Takiar commented on HIVE-14162: - [~szita], [~asinkovits] could you take a look? > Allow

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569874#comment-16569874 ] Sahil Takiar commented on HIVE-14162: - [~ngangam], [~aihuaxu] could you take a look at this patch. I

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569531#comment-16569531 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569529#comment-16569529 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568478#comment-16568478 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568451#comment-16568451 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567557#comment-16567557 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567538#comment-16567538 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559424#comment-16559424 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559407#comment-16559407 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558400#comment-16558400 ] Naveen Gangam commented on HIVE-14162: -- Thanks for the clarification. Having a single thread that

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558366#comment-16558366 ] Sahil Takiar commented on HIVE-14162: - [~naven084k] thanks for taking a look. A new Spark session is

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556785#comment-16556785 ] Naveen Gangam commented on HIVE-14162: -- [~stakiar] I took a quick look at the patch. Just thinking

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548213#comment-16548213 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547628#comment-16547628 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547607#comment-16547607 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545945#comment-16545945 ] Hive QA commented on HIVE-14162: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545936#comment-16545936 ] Hive QA commented on HIVE-14162: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545673#comment-16545673 ] Sahil Takiar commented on HIVE-14162: - The high level idea is to introduce a new config called

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545560#comment-16545560 ] Sahil Takiar commented on HIVE-14162: - [~aihuaxu] was working on this recently, so assigned to

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329692#comment-16329692 ] Xuefu Zhang commented on HIVE-14162: Thanks, [~belugabehr]. I liked your thoughts and agreed that live

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324429#comment-16324429 ] BELUGA BEHR commented on HIVE-14162: The size of the driver is only configurable before the

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324408#comment-16324408 ] Xuefu Zhang commented on HIVE-14162: The size of the driver is configurable. Minimum number of

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324405#comment-16324405 ] BELUGA BEHR commented on HIVE-14162: [~xuefuz] I am familiar with dynamic allocation. So at a minimum

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324349#comment-16324349 ] Xuefu Zhang commented on HIVE-14162: [~belugabehr], Spark on Yarn is powered by a feature call dynamic

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324121#comment-16324121 ] BELUGA BEHR commented on HIVE-14162: [~xuefuz] Thanks for the review. The issue here is that holding

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321254#comment-16321254 ] Xuefu Zhang commented on HIVE-14162: I don't quite follow why existing configurations cannot meet what

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321226#comment-16321226 ] BELUGA BEHR commented on HIVE-14162: [~aihuaxu] We realy need another timeout that applies only to

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138890#comment-16138890 ] Aihua Xu commented on HIVE-14162: - Such requirement can be achieved by the following configuration

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839740#comment-15839740 ] Aihua Xu commented on HIVE-14162: - Yeah I also have such kind of configuration in mind. Feels it's better

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838224#comment-15838224 ] BELUGA BEHR commented on HIVE-14162: How about some sort of idle Spark timeout value that can be

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667675#comment-15667675 ] Aihua Xu commented on HIVE-14162: - Let me check that case. So Spark session is closed for that? > Allow

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667691#comment-15667691 ] Thomas Scott commented on HIVE-14162: - [~aihuaxu][~xuefuz] if i remember correctly the case in

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667609#comment-15667609 ] Xuefu Zhang commented on HIVE-14162: If they are switching engines, I think spark resources are

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667558#comment-15667558 ] Aihua Xu commented on HIVE-14162: - Thanks Xuefu. There are some cases that the session is still needed.

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667464#comment-15667464 ] Xuefu Zhang commented on HIVE-14162: Hi [~aihuaxu], I'm not 100% sure if this request is needed even

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667393#comment-15667393 ] Aihua Xu commented on HIVE-14162: - [~xuefuz] My first patch doesn't work. I'm thinking of creating a new

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15626698#comment-15626698 ] Aihua Xu commented on HIVE-14162: - I will take a look. > Allow disabling of long running job on Hive On

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362559#comment-15362559 ] Thomas Scott commented on HIVE-14162: - This is equivalent to running: set