Griffin service error

Hi Griffin Team, I am trying to run the service jar for griffin but I am encountering the below error, [cid:image001.png@01D3D0DF.A114DDA0] Need guidance on the same. Thank you, Karan Gupta From: Lionel, Liu Sent: Saturday, April 7, 2018 1:32 PM To: Vinod Raina ; dev

RE: Griffin Service Error

Hi Lionel, hive.metastore.uris is correctly set as per my knowledge in the application.properties. Could you suggest any alternative or a work around? Thank you, Karan Gupta From: Lionel, Liu Sent: Tuesday, April 10, 2018 6:00 PM To: Karan Gupta ; dev@griffin.incubator.apache.org Subject: RE

RE: Griffin Service Error

, Karan Gupta From: Lionel Liu Sent: Wednesday, April 11, 2018 10:34 AM To: dev@griffin.incubator.apache.org; Karan Gupta Subject: Re: Griffin Service Error Hi Karan, I've read your log again, found error happens as steps bellow: 1. You've configured hive.metastore.uris as "thrif

RE: Griffin Service Error

Hortonworks HDP distribution Thank you, Karan Gupta From: Lionel Liu Sent: Thursday, April 12, 2018 3:19 PM To: Karan Gupta Cc: dev@griffin.incubator.apache.org Subject: Re: Griffin Service Error Hi Karan, Is your hive cluster based on kerberised cluster? I doubt that it was caused by that. How about

RE: Re:RE: Griffin Service Error

Hi Lionel, Placing hive-site.xml into the service directory did the trick. Thank you, Karan Gupta From: bhlx3l...@163.com On Behalf Of Lionel Liu Sent: Friday, April 13, 2018 9:33 AM To: Karan Gupta Cc: dev@griffin.incubator.apache.org Subject: Re:RE: Griffin Service Error Hi Karan, Griffin

Partition Configuration

Hi Lionel, Could you help me understand the partition configuration under accuracy measure better with an example. Thank you, Karan Gupta Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information

RE: Partition Configuration

Thanks Lionel, That was helpful. Thank you, Karan Gupta From: Lionel Liu Sent: Wednesday, April 25, 2018 3:49 PM To: Karan Gupta Cc: dev@griffin.incubator.apache.org Subject: Re: Partition Configuration Hi Karan, For most hive table in production environment, there're partition co

Livy Error

Authentication required Could you help me out with the same. Thank you, Karan Gupta Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is

RE: Re:Livy Error

om: bhlx3l...@163.com On Behalf Of Lionel Liu Sent: Friday, April 27, 2018 8:40 PM To: Karan Gupta Cc: dev@griffin.incubator.apache.org Subject: Re:Livy Error Hi Karan, I need some more information, how did you configure in sparkJob.properties? Can you post to livy directly by cur

Custom Rules For Existing Data

Hi Lionel, Can we create custom rules and profiles for the existing data? If so, could you guide me as to how? Thank you, Karan Gupta Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is

No Index Formation in Elastic Search

Hi Lionel, While the Spark Application gets finished, I do not see any Index getting created in the elastic search, hence I do not see the data quality metrics getting populated. Could you help me out with a possible solution? Thank you, Karan Gupta Any

RE: No Index Formation in Elastic Search

. They appear as “array” in JSON… but the “EmailParam” and “SmsParam” do not expect a List… This was causing Spark jobs not to launch. We edited the env.json accordingly…. We hope we did the right thing… Can you confirm this? Thank you, Karan Gupta From: Lionel Liu Sent: Friday, May 4, 2018 11:46 AM

RE: No Index Formation in Elastic Search

have created the index, there are errors which are suggestive of missing “mappings” Is there an auto index create property that we need to enable somewhere in ES? I could not find anything in the config yml file though…. Thank you, Karan Gupta From: Lionel Liu Sent: Friday, May 4, 2018 2:11 PM To

RE: No Index Formation in Elastic Search

Hi Lionel, We are not using Docker Image, hence we want to set it up manually. Could you provide us the “CREATE” statement for griffin indices along with “mappings”. Thank you, Karan Gupta From: Lionel Liu Sent: Friday, May 4, 2018 2:56 PM To: Karan Gupta Cc: dev

RE: No Index Formation in Elastic Search

,"tmst" : {"type": "long"},"value" : {"properties": {"total": {"type": "long"},"miss": {"type": "long"},"matched": {"type": "long"}}}' When I try to G

RE: No Index Formation in Elastic Search

anks, Best, Karan From: Lionel Liu Sent: Wednesday, May 9, 2018 11:36 AM To: dev@griffin.incubator.apache.org; Karan Gupta Subject: Re: No Index Formation in Elastic Search Hi Karan, Sorry for the missing field "__tmst", which is the timestamp with each output value record. The m

RE: No Index Formation in Elastic Search

metrics. Any suggestions here? Env.json has "method": "post" for ES persist part. Should it be POST? Thanks, Best, Karan From: Lionel Liu Sent: Friday, May 11, 2018 3:45 PM To: Karan Gupta Cc: dev@griffin.incubator.apache.org Subject: Re: No Index Formation in Elastic Sea

RE: No Index Formation in Elastic Search

.addMetricValues(java.util.List) Thanks, Best, Karan -Original Message- From: Karan Gupta Sent: Friday, May 11, 2018 4:12 PM To: Lionel Liu Cc: dev@griffin.incubator.apache.org Subject: RE: No Index Formation in Elastic Search Hi Lionel, Thank you for your quick revert. I recreated

Griffin DQ Metric Populated

raries that we use, probably are older… may be…….Not sure.. Anyway, All is well that ends well 😊 A Big THANKS to @Lionel Liu<mailto:lionel...@apache.org> for all his help Thank you, Karan Gupta Any comments or statements made in this email are not

RE: Re:RE: No Index Formation in Elastic Search

:lionel...@apache.org> for all his help Thank you, Karan Gupta From: bhlx3l...@163.com On Behalf Of Lionel Liu Sent: Saturday, May 12, 2018 4:38 PM To: Karan Gupta ; dev@griffin.incubator.apache.org Subject: Re:RE: No Index Formation in Elastic Search Hi Karan, In Griffin service module,

Apache Griffin Mappings For Profiling

: "/griffin/profiling" in the env.json? Thank you, Karan Gupta Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is addre

RE: Apache Griffin Mappings For Profiling

Hi Lionel, Sorry for the inconvenience, I can now see the tabular representation of the profiling. Thank you, Karan Gupta -Original Message- From: Lionel, Liu Sent: Monday, May 21, 2018 6:50 PM To: dev@griffin.incubator.apache.org; Karan Gupta Subject: RE: Apache Griffin Mappings

Unable to persist profiling results in HDFS

Hi Lionel, I created a custom config.json and defined a custom rule to run and submitted it through spark. The spark submit runs fine without any issues. On HDFS, I could see the directory with the custom rule name but I am unable to see the _METRIC file where the results will be persisted, I on

RE: Unable to persist profiling results in HDFS

Hi Lionel, Thank you for the response, well this helped. Cheers !! Thank you, Karan Gupta From: Lionel Liu Sent: Friday, May 25, 2018 7:07 AM To: Karan Gupta Cc: dev@griffin.incubator.apache.org Subject: Re: Unable to persist profiling results in HDFS Hi Karan, It looks good to me, but

RE: Apache Griffin Profiling

Does profiling support files instead of hive tables? If yes, can I point my “data.source” to an HDFS directory instead of specifying a file each time, so that the griffin will run the profiling job on each newly added file in that HDFS location. Thank you, Karan Gupta From: Lionel Liu Sent

RE: Re:Profiling Job for multiple tables

(*) from ${source}" 1. > Can I invoke my profiling rule through a web API? Thank you, Karan Gupta From: bhlx3l...@163.com On Behalf Of Lionel Liu Sent: Friday, May 25, 2018 7:12 PM To: Karan Gupta ; dev@griffin.incubator.apache.org Subject: Re:Profiling Job for multiple tables Hi Kar