RE: Re:Profiling Job for multiple tables

2018-05-31 Thread Karan Gupta
uot; 1. > Can I invoke my profiling rule through a web API? Thank you, Karan Gupta From: bhlx3l...@163.com On Behalf Of Lionel Liu Sent: Friday, May 25, 2018 7:12 PM To: Karan Gupta ; dev@griffin.incubator.apache.org Subject: Re:Profiling Job for multiple tables Hi Karan, I think it coul

RE: Apache Griffin Profiling

2018-05-28 Thread Karan Gupta
Does profiling support files instead of hive tables? If yes, can I point my “data.source” to an HDFS directory instead of specifying a file each time, so that the griffin will run the profiling job on each newly added file in that HDFS location. Thank you, Karan Gupta From: Lionel Liu <lio

RE: Unable to persist profiling results in HDFS

2018-05-24 Thread Karan Gupta
Hi Lionel, Thank you for the response, well this helped. Cheers !! Thank you, Karan Gupta From: Lionel Liu <lionel...@apache.org> Sent: Friday, May 25, 2018 7:07 AM To: Karan Gupta <karan.gu...@tavant.com> Cc: dev@griffin.incubator.apache.org Subject: Re: Unable to persist profi

RE: Re:RE: No Index Formation in Elastic Search

2018-05-15 Thread Karan Gupta
:lionel...@apache.org> for all his help Thank you, Karan Gupta From: bhlx3l...@163.com <bhlx3l...@163.com> On Behalf Of Lionel Liu Sent: Saturday, May 12, 2018 4:38 PM To: Karan Gupta <karan.gu...@tavant.com>; dev@griffin.incubator.apache.org Subject: Re:RE: No Index Formation in

Griffin DQ Metric Populated

2018-05-15 Thread Karan Gupta
.http libraries that we use, probably are older… may be…….Not sure.. Anyway, All is well that ends well  A Big THANKS to @Lionel Liu<mailto:lionel...@apache.org> for all his help Thank you, Karan Gupta Any comments or statements made in this email are not

RE: No Index Formation in Elastic Search

2018-05-11 Thread Karan Gupta
.addMetricValues(java.util.List) Thanks, Best, Karan -Original Message- From: Karan Gupta <karan.gu...@tavant.com> Sent: Friday, May 11, 2018 4:12 PM To: Lionel Liu <lionel...@apache.org> Cc: dev@griffin.incubator.apache.org Subject: RE: No Index Formation in Elastic Sea

RE: No Index Formation in Elastic Search

2018-05-11 Thread Karan Gupta
metrics. Any suggestions here? Env.json has "method": "post" for ES persist part. Should it be POST? Thanks, Best, Karan From: Lionel Liu <lionel...@apache.org> Sent: Friday, May 11, 2018 3:45 PM To: Karan Gupta <karan.gu...@tavant.com> Cc: dev@griffin.incubator.

RE: No Index Formation in Elastic Search

2018-05-11 Thread Karan Gupta
ity? Thanks, Best, Karan From: Lionel Liu <lionel...@apache.org> Sent: Wednesday, May 9, 2018 11:36 AM To: dev@griffin.incubator.apache.org; Karan Gupta <karan.gu...@tavant.com> Subject: Re: No Index Formation in Elastic Search Hi Karan, Sorry for the missing field "__tmst", which is

RE: No Index Formation in Elastic Search

2018-05-08 Thread Karan Gupta
t;type": "long"},"value" : {"properties": {"total": {"type": "long"},"miss": {"type": "long"},"matched": {"type": "long"}}}' When I try to GET the indexes, I can see that

RE: No Index Formation in Elastic Search

2018-05-04 Thread Karan Gupta
Hi Lionel, We are not using Docker Image, hence we want to set it up manually. Could you provide us the “CREATE” statement for griffin indices along with “mappings”. Thank you, Karan Gupta From: Lionel Liu <lionel...@apache.org> Sent: Friday, May 4, 2018 2:56 PM To: Karan Gupta <

RE: No Index Formation in Elastic Search

2018-05-04 Thread Karan Gupta
have created the index, there are errors which are suggestive of missing “mappings” Is there an auto index create property that we need to enable somewhere in ES? I could not find anything in the config yml file though…. Thank you, Karan Gupta From: Lionel Liu <lionel...@apache.org> Sent:

No Index Formation in Elastic Search

2018-05-04 Thread Karan Gupta
Hi Lionel, While the Spark Application gets finished, I do not see any Index getting created in the elastic search, hence I do not see the data quality metrics getting populated. Could you help me out with a possible solution? Thank you, Karan Gupta Any

Custom Rules For Existing Data

2018-04-30 Thread Karan Gupta
Hi Lionel, Can we create custom rules and profiles for the existing data? If so, could you guide me as to how? Thank you, Karan Gupta Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted

RE: Re:Livy Error

2018-04-30 Thread Karan Gupta
lx3l...@163.com <bhlx3l...@163.com> On Behalf Of Lionel Liu Sent: Friday, April 27, 2018 8:40 PM To: Karan Gupta <karan.gu...@tavant.com> Cc: dev@griffin.incubator.apache.org Subject: Re:Livy Error Hi Karan, I need some more information, how did you configure in sparkJob.properties? Can

RE: Re:RE: Griffin Service Error

2018-04-16 Thread Karan Gupta
Hi Lionel, Placing hive-site.xml into the service directory did the trick. Thank you, Karan Gupta From: bhlx3l...@163.com <bhlx3l...@163.com> On Behalf Of Lionel Liu Sent: Friday, April 13, 2018 9:33 AM To: Karan Gupta <karan.gu...@tavant.com> Cc: dev@griffin.incubator.apache.org Su

RE: Griffin Service Error

2018-04-12 Thread Karan Gupta
the Hortonworks HDP distribution Thank you, Karan Gupta From: Lionel Liu <lionel...@apache.org> Sent: Thursday, April 12, 2018 3:19 PM To: Karan Gupta <karan.gu...@tavant.com> Cc: dev@griffin.incubator.apache.org Subject: Re: Griffin Service Error Hi Karan, Is your hive cluster based on kerberise

RE: Griffin Service Error

2018-04-12 Thread Karan Gupta
, Karan Gupta From: Lionel Liu <lionel...@apache.org> Sent: Wednesday, April 11, 2018 10:34 AM To: dev@griffin.incubator.apache.org; Karan Gupta <karan.gu...@tavant.com> Subject: Re: Griffin Service Error Hi Karan, I've read your log again, found error happens as steps bellow: 1. You'v

RE: Griffin Service Error

2018-04-10 Thread Karan Gupta
Hi Lionel, hive.metastore.uris is correctly set as per my knowledge in the application.properties. Could you suggest any alternative or a work around? Thank you, Karan Gupta From: Lionel, Liu <bhlx3l...@163.com> Sent: Tuesday, April 10, 2018 6:00 PM To: Karan Gupta <karan.gu...@t

Griffin service error

2018-04-10 Thread Karan Gupta
Hi Griffin Team, I am trying to run the service jar for griffin but I am encountering the below error, [cid:image001.png@01D3D0DF.A114DDA0] Need guidance on the same. Thank you, Karan Gupta From: Lionel, Liu <bhlx3l...@163.com> Sent: Saturday, April 7, 2018 1:32 PM To: Vinod