Hi, Kevin
Thanks for your reply.
Could you help with the documentation about parameter
external.config.location ？
When the Griffin submits some jobs with the same session name at the same
time, Livy server will reject the other jobs, except the first job, because the
session is already started. So I think this problem is not related to where
the configuration files are located at.
At 2019-09-18 14:17:00, "Kevin Yao" <ahutsunsh...@gmail.com> wrote:
>Seems pr closed. I cannot comment it.
>I agree with you. we should have document to introduce such function for
>users.
>
>Thanks,
>Kevin
>
>On Wed, Sep 18, 2019 at 12:38 PM William Guo <gu...@apache.org> wrote:
>
>> hi Kevin,
>>
>> the PR is https://github.com/apache/griffin/pull/534 , Could you review
>> the
>> PR again?
>>
>> external.config.location need more documentation for users.
>>
>>
>> Thanks,
>> William
>>
>> On Wed, Sep 18, 2019 at 10:27 AM Kevin Yao <ahutsunsh...@gmail.com> wrote:
>>
>> > Hi William, wankun,
>> >
>> > Actually, I don't think it's a bug for Griffin. `sparkProperties.json`
>> is a
>> > config file which can be replaced by user in external location. We have
>> > already supported such function with `external.config.location` in
>> > `application.properties` file.
>> >
>> > Thanks,
>> > Kevin
>> >
>> > On Wed, Sep 18, 2019 at 7:30 AM ASF GitHub Bot (Jira) <j...@apache.org>
>> > wrote:
>> >
>> > >
>> > > [
>> > >
>> >
>> https://issues.apache.org/jira/browse/GRIFFIN-290?focusedWorklogId=314053&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314053
>> > > ]
>> > >
>> > > ASF GitHub Bot logged work on GRIFFIN-290:
>> > > ------------------------------------------
>> > >
>> > > Author: ASF GitHub Bot
>> > > Created on: 17/Sep/19 23:29
>> > > Start Date: 17/Sep/19 23:29
>> > > Worklog Time Spent: 10m
>> > > Work Description: asfgit commented on pull request #534:
>> > > [GRIFFIN-290]Fix bug for submitting job to livy
>> > > URL: https://github.com/apache/griffin/pull/534
>> > >
>> > >
>> > >
>> > >
>> > > ----------------------------------------------------------------
>> > > This is an automated message from the Apache Git Service.
>> > > To respond to the message, please log on to GitHub and use the
>> > > URL above to go to the specific comment.
>> > >
>> > > For queries about this service, please contact Infrastructure at:
>> > > us...@infra.apache.org
>> > >
>> > >
>> > > Issue Time Tracking
>> > > -------------------
>> > >
>> > > Worklog Id: (was: 314053)
>> > > Time Spent: 50m (was: 40m)
>> > >
>> > > > Fix bug for submitting job to livy
>> > > > ----------------------------------
>> > > >
>> > > > Key: GRIFFIN-290
>> > > > URL:
>> https://issues.apache.org/jira/browse/GRIFFIN-290
>> > > > Project: Griffin
>> > > > Issue Type: Task
>> > > > Reporter: wan kun
>> > > > Priority: Major
>> > > > Fix For: 0.6.0
>> > > >
>> > > > Time Spent: 50m
>> > > > Remaining Estimate: 0h
>> > > >
>> > > > When griffin submit multiple DQ jobs to livy, the http parameter
>> `name`
>> > > is always griffin.
>> > > > So livy will reject them.
>> > > > job request :
>> > > >
>> > > > {quote}[owner: null, request: [proxyUser: None, file:
>> > > hdfs://nameservice-standby/user/kun.wan/measure-0.6.0-SNAPSHOT.jar,
>> > > > args: {
>> > > > "spark" :
>> > > > Unknown macro: \{ "log.level" }
>> > > > ,
>> > > > "sinks" : [
>> > > > Unknown macro: \{ .... }
>> > > > ],
>> > > > "griffin.checkpoint" : [ ]
>> > > > },{
>> > > > "measure.type" : "griffin",
>> > > > "id" : 5202,
>> > > > "name" : "spu_null_check",
>> > > > "owner" : "test",
>> > > > "description" : "check null value for store and category",
>> > > > "deleted" : false,
>> > > > "timestamp" : 1568195100000,
>> > > > "dq.type" : "PROFILING",
>> > > > "sinks" : [ "ELASTICSEARCH", "HDFS" ],
>> > > > "process.type" : "BATCH",
>> > > > "rule.description" :
>> > > > ,
>> > > > "data.sources" : [
>> > > > Unknown macro: \{ .... }
>> > > > ],
>> > > > "evaluate.rule" :
>> > > > ,
>> > > > "measure.type" : "griffin"
>> > > > },raw,raw, driverMemory: 1g, executorMemory: 6g, executorCores: 2,
>> > > numExecutors: 6, queue: root.users.kun_dot_wan, {color:#ff0000}*name:
>> > > griffin*{color}]]
>> > > > {quote}
>> > > >
>> > > > livy Response :
>> > > > {quote}400 Bad Request
>> > > > [Date:"Thu, 12 Sep 2019 10:00:00 GMT",
>> > > Content-Type:"application/json;charset=utf-8", Content-Length:"47",
>> > > Server:"Jetty(9.3.24.v20180605)"]
>> > > > {"msg":"Duplicate session name: Some(griffin)"}
>> > > > {quote}
>> > >
>> > >
>> > >
>> > > --
>> > > This message was sent by Atlassian Jira
>> > > (v8.3.4#803005)
>> > >
>> >
>>
