Hi Lionel,
I think in the previous mail people can’t see the image so I’m attaching it.
Also I don’t see measure name when I click on DQ Matrix and My
Dashboards.Please see atched.
Regards
Ashwini
From: Ashwini Kumar Gupta <ashwini.gu...@enquero.com>
Sent: 15 April 2018 12:07
To: dev@griffin.incubator.apache.org; Lionel Liu <lionel...@apache.org>
Subject: RE: Re:RE: Can't get output after creating measure
Update:
Griffin was not able to reach out to HDFS because while I was creating DONE
file in measure I gave path as /user/warehouse . How ever I should have fully
qualified name. Well I think this was the issue.
Now that I have corrected it I do not get the “can’t reach hdfs error” but I’m
still not getting the output.
Livy dashboard:
[cid:image001.png@01D3D4B2.50398350]
Livy log file:
Warning: Skip remote jar hdfs:///griffin/griffin-measure.jar.
Warning: Skip remote jar hdfs:///livy/datanucleus-api-jdo-3.2.6.jar.
Warning: Skip remote jar hdfs:///livy/datanucleus-core-3.2.10.jar.
Warning: Skip remote jar hdfs:///livy/datanucleus-rdbms-3.2.9.jar.
java.lang.ClassNotFoundException: org.apache.griffin.measure.Application
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:176)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:689)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
NOTE: I have placed in griffin-measure in hdfs:///griffin/griffin-measure.jar.
And datanucleus files in hdfs:///livy/datanucleus-api-jdo-3.2.6.jar
Also I have copied datanucleus jar files from Hive library folder. Livy 0.4.0
doesn’t have these jars files.
Please suggest corrections
Regards
Ashwini
From: Ashwini Kumar Gupta
<ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com>>
Sent: 15 April 2018 11:41
To: Lionel Liu <lionel...@apache.org<mailto:lionel...@apache.org>>
Cc: dev@griffin.incubator.apache.org<mailto:dev@griffin.incubator.apache.org>
Subject: RE: Re:RE: Can't get output after creating measure
Hello Lionel,
As suggested I kept only one configuration i.e
sparkJob.jars =
hdfs:///livy/datanucleus-api-jdo-3.2.6.jar;\
hdfs:///livy/datanucleus-core-3.2.10.jar;\
hdfs:///livy/datanucleus-rdbms-3.2.9.jar
* I kept env.json , griffin-measure.jar ,hive-site.xml in
hdfs:///griffin/griffin-measure.jar .
* I created an accuracy measure and created a job with 1 minute cron
expression.
Attached is the log file. It seems it can’t get to HDFS although my cloudera
HDFS is up.
Regards
Ashwini
From: Lionel Liu <lionel...@apache.org<mailto:lionel...@apache.org>>
Sent: 13 April 2018 15:14
To: Ashwini Kumar Gupta
<ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com>>
Cc: dev@griffin.incubator.apache.org<mailto:dev@griffin.incubator.apache.org>
Subject: Re: Re:RE: Can't get output after creating measure
Hi Ashwini,
I've read your document, and here lists my answers:
Question
· Do I keep both of them?
You should only keep the effective “sparkJob.jars” parameter.
· Do I have to copy hive-site.xml to HDFS and give the HDFS path in
spark.yarn.dist.files?
You’d better copy hive-site.xml to HDFS, cause livy could only submit spark
applications in cluster mode, so the hive-site.xml should be accessed by each
node.
About the livy log:
According to livy log, it seems that your configuration of sparkJob.properties
doesn’t work, livy is trying to find hdfs:///griffin/griffin-measure.jar, not
hdfs:///user/griffin/griffin-measure.jar.
Pls correct the sparkJob.properties, and rebuild the service module and have a
try.
Thanks,
Lionel
On Fri, Apr 13, 2018 at 4:16 PM, Ashwini Kumar Gupta
<ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com>> wrote:
Hello Lionel,
Apologies for delayed reply. I was trying all my options before raising an
issue.
I’m attaching my installation steps. Please let me know what’s wrong with them.
Regards
Ashwin
From: bhlx3l...@163.com<mailto:bhlx3l...@163.com>
<bhlx3l...@163.com<mailto:bhlx3l...@163.com>> On Behalf Of Lionel Liu
Sent: 10 April 2018 18:11
To: dev@griffin.incubator.apache.org<mailto:dev@griffin.incubator.apache.org>;
Ashwini Kumar Gupta
<ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com>>
Subject: Re:RE: Can't get output after creating measure
Hi Ashwini,
It works the same in linux OS, we need to check the log to figure out what
happened, it might be some configure mistake or input mistake.
I recommend you try our docker image first, by following this doc:
https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/griffin-docker-guide.md
--
Regards,
Lionel, Liu
At 2018-04-10 19:25:01, "Ashwini Kumar Gupta"
<ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com>> wrote:
>Hello Lionel,
>
>I’m running this in cloudera VM. Will that change anything?
>
>Regards
>Ashwin
>
>From: Lionel Liu <lionel...@apache.org<mailto:lionel...@apache.org>>
>Sent: 10 April 2018 15:26
>To: dev@griffin.incubator.apache.org<mailto:dev@griffin.incubator.apache.org>;
>Ashwini Kumar Gupta
><ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com>>
>Subject: Re: Can't get output after creating measure
>
>Hi Ashwini,
>
>First, you could check the log of griffin service, to know if it has triggered
>the job instance.
>Then, griffin service will submit a spark application with configuration to
>livy, you can check the log of livy, to verify if it has been submitted
>correctly.
>After that, you need to check the spark cluster, to verify the application has
>been accepted by the cluster, if it runs, you can get the application log
>through yarn.
>
>Each step error might block the result.
>
>Thanks,
>Lionel
>
>On Tue, Apr 10, 2018 at 5:01 PM, William Guo
><gu...@apache.org<mailto:gu...@apache.org<mailto:gu...@apache.org%3cmailto:gu...@apache.org>>>
> wrote:
>hi Ashwin,
>
>Could you show us your log here?
>
>Thanks,
>William
>
>On Tue, Apr 10, 2018 at 3:35 PM, Ashwini Kumar Gupta <
>ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com<mailto:ashwini.gu...@enquero.com%3cmailto:ashwini.gu...@enquero.com>>>
> wrote:
>
>> Hello Team,
>>
>> I have been trying to install and use griffin but I cannot get output when
>> I click on DQ Matrix.
>>
>> I created a measure, created a job to run.
>> The sequence in which I run all services are:
>>
>>
>> 1. Elasticsearch
>> 2. Jar file
>>
>> I also noticed that griffin is not creating mapping in ES.
>>
>> Can you please tell me where I'm going wrong.
>>
>> Thanks
>> Ashwin
>>
>
