Seems pr closed. I cannot comment it. I agree with you. we should have document to introduce such function for users.
Thanks, Kevin On Wed, Sep 18, 2019 at 12:38 PM William Guo <[email protected]> wrote: > hi Kevin, > > the PR is https://github.com/apache/griffin/pull/534 , Could you review > the > PR again? > > external.config.location need more documentation for users. > > > Thanks, > William > > On Wed, Sep 18, 2019 at 10:27 AM Kevin Yao <[email protected]> wrote: > > > Hi William, wankun, > > > > Actually, I don't think it's a bug for Griffin. `sparkProperties.json` > is a > > config file which can be replaced by user in external location. We have > > already supported such function with `external.config.location` in > > `application.properties` file. > > > > Thanks, > > Kevin > > > > On Wed, Sep 18, 2019 at 7:30 AM ASF GitHub Bot (Jira) <[email protected]> > > wrote: > > > > > > > > [ > > > > > > https://issues.apache.org/jira/browse/GRIFFIN-290?focusedWorklogId=314053&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314053 > > > ] > > > > > > ASF GitHub Bot logged work on GRIFFIN-290: > > > ------------------------------------------ > > > > > > Author: ASF GitHub Bot > > > Created on: 17/Sep/19 23:29 > > > Start Date: 17/Sep/19 23:29 > > > Worklog Time Spent: 10m > > > Work Description: asfgit commented on pull request #534: > > > [GRIFFIN-290]Fix bug for submitting job to livy > > > URL: https://github.com/apache/griffin/pull/534 > > > > > > > > > > > > > > > ---------------------------------------------------------------- > > > This is an automated message from the Apache Git Service. > > > To respond to the message, please log on to GitHub and use the > > > URL above to go to the specific comment. > > > > > > For queries about this service, please contact Infrastructure at: > > > [email protected] > > > > > > > > > Issue Time Tracking > > > ------------------- > > > > > > Worklog Id: (was: 314053) > > > Time Spent: 50m (was: 40m) > > > > > > > Fix bug for submitting job to livy > > > > ---------------------------------- > > > > > > > > Key: GRIFFIN-290 > > > > URL: > https://issues.apache.org/jira/browse/GRIFFIN-290 > > > > Project: Griffin > > > > Issue Type: Task > > > > Reporter: wan kun > > > > Priority: Major > > > > Fix For: 0.6.0 > > > > > > > > Time Spent: 50m > > > > Remaining Estimate: 0h > > > > > > > > When griffin submit multiple DQ jobs to livy, the http parameter > `name` > > > is always griffin. > > > > So livy will reject them. > > > > job request : > > > > > > > > {quote}[owner: null, request: [proxyUser: None, file: > > > hdfs://nameservice-standby/user/kun.wan/measure-0.6.0-SNAPSHOT.jar, > > > > args: { > > > > "spark" : > > > > Unknown macro: \{ "log.level" } > > > > , > > > > "sinks" : [ > > > > Unknown macro: \{ .... } > > > > ], > > > > "griffin.checkpoint" : [ ] > > > > },{ > > > > "measure.type" : "griffin", > > > > "id" : 5202, > > > > "name" : "spu_null_check", > > > > "owner" : "test", > > > > "description" : "check null value for store and category", > > > > "deleted" : false, > > > > "timestamp" : 1568195100000, > > > > "dq.type" : "PROFILING", > > > > "sinks" : [ "ELASTICSEARCH", "HDFS" ], > > > > "process.type" : "BATCH", > > > > "rule.description" : > > > > , > > > > "data.sources" : [ > > > > Unknown macro: \{ .... } > > > > ], > > > > "evaluate.rule" : > > > > , > > > > "measure.type" : "griffin" > > > > },raw,raw, driverMemory: 1g, executorMemory: 6g, executorCores: 2, > > > numExecutors: 6, queue: root.users.kun_dot_wan, {color:#ff0000}*name: > > > griffin*{color}]] > > > > {quote} > > > > > > > > livy Response : > > > > {quote}400 Bad Request > > > > [Date:"Thu, 12 Sep 2019 10:00:00 GMT", > > > Content-Type:"application/json;charset=utf-8", Content-Length:"47", > > > Server:"Jetty(9.3.24.v20180605)"] > > > > {"msg":"Duplicate session name: Some(griffin)"} > > > > {quote} > > > > > > > > > > > > -- > > > This message was sent by Atlassian Jira > > > (v8.3.4#803005) > > > > > >
