Re: Yarn queues for spark jobs submitted via livy
Are you hitting https://issues.apache.org/jira/browse/KNOX-1098
Bikas
From: Lian Jiang
Sent: Wednesday, November 28, 2018 2:21 PM
To: user@livy.incubator.apache.org
Subject: Yarn queues for spark jobs submitted via livy
We are using HDP3 and knox. Users can submit spark job using livy api like:
curl -k -u user1:"" -H "Content-Type: application/json" -H "X-Requested-By:
admin" -d
'{"className":"my.job",
"file":"/tmp/sparkApp-assembly-1.0.jar"}'
-X POST
https://knox.com/gateway/ui/livy/v1/batches
The jobs submitted by user1 and user2 both show livy as user on yarn UI. Now I
setup a yarn queue for user1 and another for user2. However can I let these two
users use their corresponding yarn queue? Appreciate any help!
Re: Yarn queues for spark jobs submitted via livy
Meanwhile, user1 cannot user2's queue and vice versa. Thanks.
On Wed, Nov 28, 2018 at 2:21 PM Lian Jiang wrote:
> We are using HDP3 and knox. Users can submit spark job using livy api like:
>
> curl -k -u user1:"" -H "Content-Type: application/json" -H "X-Requested-By:
> admin" -d '{"className":"my.job
> ",
> "file":"/tmp/sparkApp-assembly-1.0.jar
>
> "}' -X POST https://knox.com/gateway/ui/livy/v1/batches
>
>
> The jobs submitted by user1 and user2 both show livy as user on yarn UI.
> Now I setup a yarn queue for user1 and another for user2. However can I let
> these two users use their corresponding yarn queue? Appreciate any help!
>
