Cory Woytasik created GRIFFIN-189:
-------------------------------------
Summary: Griffin - Livy error
Key: GRIFFIN-189
URL: https://issues.apache.org/jira/browse/GRIFFIN-189
Project: Griffin (Incubating)
Issue Type: Bug
Affects Versions: 0.2.0-incubating
Reporter: Cory Woytasik
We are trying to get griffin set up and after creating measure and jobs and
letting them run we have noticed the results are not available via the DQ
metrics link or metric link from the job itself. We have noticed when the job
gets submitted the following spark context and error message are generated. We
assume we must have a setting in one of the directories set incorrectly.
Thoughts?
INFO 20972 --- [ryBean_Worker-2] o.a.g.c.j.SparkSubmitJob : {
"measure.type" : "griffin",
"id" : 13,
"name" : "LineageAccuracy",
"owner" : "test",
"description" : "AccuracyTest",
"organization" : null,
"deleted" : false,
"timestamp" : 1535998320000,
"dq.type" : "accuracy",
"process.type" : "batch",
"data.sources" : [ {
"id" : 16,
"name" : "source",
"connectors" : [ {
"id" : 17,
"name" : "source1535741016027",
"type" : "HIVE",
"version" : "1.2",
"predicates" : [ ],
"data.unit" : "1day",
"config" : {
"database" : "default",
"table.name" : "lineage"
}
} ]
}, {
"id" : 18,
"name" : "target",
"connectors" : [ {
"id" : 19,
"name" : "target1535741022277",
"type" : "HIVE",
"version" : "1.2",
"predicates" : [ ],
"data.unit" : "1day",
"config" : {
"database" : "default",
"table.name" : "lineageload"
}
} ]
} ],
"evaluate.rule" : {
"id" : 14,
"rules" : [ {
"id" : 15,
"rule" : "source.asset=target.asset AND source.element=target.element AND
source.elementtype=target.elementtype AND source.object=target.object AND
source.objecttype=target.objecttype AND source.objectfield=target.objectfield
AND source.sourceelement=target.sourceelement AND
source.sourceobject=target.sourceobject AND
source.sourcefield=target.sourcefield AND
source.sourcefieldname=target.sourcefieldname AND
source.transformationtext=target.transformationtext AND
source.displayindicator=target.displayindicator",
"name" : "accuracy",
"dsl.type" : "griffin-dsl",
"dq.type" : "accuracy"
} ]
},
"measure.type" : "griffin"
}
{color:#FF0000}2018-09-04 13:12:00.752 ERROR 20972 --- [ryBean_Worker-2]
o.a.g.c.j.SparkSubmitJob : Post to livy error. 500 Internal
Server Error{color}
[EL Fine]: sql: 2018-09-04
13:12:00.754--ClientSession(787879814)--Connection(1389579691)--UPDATE
JOBINSTANCEBEAN SET predicate_job_deleted = ?, STATE = ? WHERE (ID = ?)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
