wan kun created GRIFFIN-290:
-------------------------------
Summary: Fix bug for submitting job to livy
Key: GRIFFIN-290
URL: https://issues.apache.org/jira/browse/GRIFFIN-290
Project: Griffin
Issue Type: Task
Reporter: wan kun
When griffin submit multiple DQ jobs to livy, the http parameter `name` is
always griffin.
So livy will reject them.
job request :
{quote}[owner: null, request: [proxyUser: None, file:
hdfs://nameservice-standby/user/kun.wan/measure-0.6.0-SNAPSHOT.jar,
args: {
"spark" : {
"log.level" : "WARN"
},
"sinks" : [ {
....
} ],
"griffin.checkpoint" : [ ]
},{
"measure.type" : "griffin",
"id" : 5202,
"name" : "spu_null_check",
"owner" : "test",
"description" : "check null value for store and category",
"deleted" : false,
"timestamp" : 1568195100000,
"dq.type" : "PROFILING",
"sinks" : [ "ELASTICSEARCH", "HDFS" ],
"process.type" : "BATCH",
"rule.description" : {
....
},
"data.sources" : [ {
....
} ],
"evaluate.rule" : {
....
},
"measure.type" : "griffin"
},raw,raw, driverMemory: 1g, executorMemory: 6g, executorCores: 2,
numExecutors: 6, queue: root.users.kun_dot_wan, {color:#FF0000}*name:
griffin*{color}]]
{quote}
livy Response :
{quote}400 Bad Request
[Date:"Thu, 12 Sep 2019 10:00:00 GMT",
Content-Type:"application/json;charset=utf-8", Content-Length:"47",
Server:"Jetty(9.3.24.v20180605)"]
Unknown macro: \{"msg"}{quote}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
