Hi William/Lionel, Could you please help us to understand on the *profiling* feature availability, we see the code traces on profiling, could you let us know if we can starting using it from some branch which could be used so that we could leverage griffin on two dimensions (accuracy & profiling). If its not available a tentative date so that we could decide upon the same.
Regards, Ananthanarayanan.M.S On Wed, Aug 2, 2017 at 10:47 PM, Mara Preotescu <mara.preote...@nielsen.com> wrote: > Thank you William. > > Ananth will follow up with a few more questions on the roadmap. > > Thanks, > Mara > > On Wed, Aug 2, 2017 at 9:20 AM, William Guo <gu...@apache.org> wrote: > >> >> Meeting minutes with Nielsen: >> >> >> >> - Discuss griffin to support filters for metastore tables or >> navigation assistance for table selection on UI. >> - Griffin provides RESTful API for backend. >> - Discuss griffin to support multiple source or target tables. >> - Discuss more supporting file types, such as parquet. >> - In griffin, the partition field is optional, it just helps to >> provide the specific part of data, it will get all the data of a table >> without any partition information. >> - Config json file provides the parameters for griffin measure >> calculation, you can also submit a spark job with it directly. >> - Currently, griffin can only reuse measure, not rule. We’ll discuss >> about this, if we need to support reusing rules. >> - Sample ratio field in config file is optional, in batch mode we >> don’t need to configure it. >> - In griffin, mapping of columns are limited, discuss to support >> advanced features like joining between tables , or advanced sql script. >> - At current, the rule parser doesn’t support customized rules, >> griffin has the plan to support this. //TODO document it and send it to >> dev >> list >> - Griffin doesn’t support metrics alert function, it posts all the >> metrics to elasticsearch, es supports such feature. //TODO, write a >> solution for it based on elastic search >> - In griffin, you can’t modify the exist rules or measure at current. >> >> >> >> >> Thanks, >> William >> >> ------------------------------ >> *From:* William GUO <guo...@outlook.com> on behalf of William Guo < >> gu...@apache.org> >> *Sent:* Wednesday, August 2, 2017 10:02:15 AM >> *To:* Mara Preotescu >> *Cc:* dev@griffin.incubator.apache.org; Ananthanarayanan Ms; Kunduru, >> Abishek >> *Subject:* Re: Griffin support & roadmap >> >> hi mara, >> >> >> Are you join? >> >> >> Thanks, >> >> William >> >> ________________________________ >> From: Mara Preotescu <mara.preote...@nielsen.com> >> Sent: Monday, July 31, 2017 11:22:00 PM >> To: William Guo >> Cc: dev@griffin.incubator.apache.org; Ananthanarayanan Ms; Kunduru, >> Abishek >> Subject: Re: Griffin support & roadmap >> >> Hi William, >> >> Would 10:00 am CST (Beijing) work for you on Wednesday 08/02? >> >> Thanks, >> Mara >> >> On Sun, Jul 30, 2017 at 10:59 PM, William Guo <gu...@apache.org<mailto: >> gu...@apache.org>> wrote: >> >> hi Mara, >> >> >> We are in China, it is hard to arrange a meeting for US, CHINA, INDIA >> together. >> >> >> China day time is fine for me. >> >> >> Thanks, >> >> William >> >> ________________________________ >> From: Mara Preotescu <mara.preote...@nielsen.com<mailto: >> mara.preote...@nielsen.com>> >> Sent: Monday, July 31, 2017 10:54:25 AM >> >> To: William Guo >> Cc: Lv, Alex; Guo, William; dev@griffin.incubator.apache.org<mailto: >> dev@griffin.incubator.apache.org> >> Subject: Re: Griffin support & roadmap >> >> Hi William, >> >> Either Wednesday or Thursday will work for us. Any better time working >> for you? What time zone are you in? I am in US ET time, a colleague of >> mine who I would like to join our discussion is in India, Chennai. >> >> Thanks, >> Mara >> >> On Sun, Jul 30, 2017 at 7:34 PM, William Guo <gu...@apache.org<mailto: >> gu...@apache.org>> wrote: >> >> hi Mara, >> >> >> Sure, We could schedule a meeting to discuss background, requirements, >> status and milestone. >> >> >> We should be fine in Wednesday or Thursday, what is your proposal? >> >> >> >> Thanks, >> >> William >> >> ________________________________ >> From: Mara Preotescu <mara.preote...@nielsen.com<mailto: >> mara.preote...@nielsen.com>> >> Sent: Friday, July 28, 2017 7:57:45 PM >> To: William Guo >> Cc: Lv, Alex; Guo, William; dev@griffin.incubator.apache.org<mailto: >> dev@griffin.incubator.apache.org> >> Subject: Re: Griffin support & roadmap >> >> HI Alex, William, >> >> THANK YOU so much for your responses. Thank you for the links. And, I >> hope you don't mind if I'll take up your offer to contact you if needed. >> We are considering, here at Nielsen, using Griffin for our new Data Quality >> framework ... we know the project is still in the incubator but we would >> like give it a try and even contributing, if needed. We already install >> it and ran a few tests. >> >> If your time permits I would like scheduling a quick call so we could >> understand the current status and, most importantly if the roadmap stays as >> in the published documents. >> >> Thanks again, >> Mara >> >> >> On Fri, Jul 28, 2017 at 4:21 AM, William Guo <gu...@apache.org<mailto: >> gu...@apache.org>> wrote: >> >> hi Mara, >> >> Few links might help, you can contact us by >> dev@griffin.incubator.apache.org<mailto:dev@griffin.incubator.apache.org> >> or my personal account gu...@apache.org<mailto:gu...@apache.org >> <gu...@apache.org>> >> >> >> GitHub : https://github.com/apache/incubator-griffin<https://github. >> com/eBay/griffin> >> Website : https://griffin.incubator.apache.org<https://griffin.incubat >> or.apache.org/> >> Contact: mailto://subscribe-...@griffin.incubator.apache.org<mailto: >> subscribe-...@griffin.incubator.apache.org> >> Apache Griffin JIRA: https://issues.apache.org/jira/browse/GRIFFIN >> Apache Griffin Wiki :https://cwiki.apache.org/conf >> luence/display/GRIFFIN/Griffin >> >> Thanks, William >> >> ________________________________ >> From: Lv, Alex <lzhix...@ebay.com<mailto:lzhix...@ebay.com>> >> Sent: Friday, July 28, 2017 9:10:09 AM >> To: Mara Preotescu; Guo, William; gu...@apache.org<mailto:guoyp@ >> apache.org <gu...@apache.org>> >> Cc: dev@griffin.incubator.apache.org<mailto:dev@griffin.incubato >> r.apache.org> >> Subject: RE: Griffin support & roadmap >> >> <<Move Amber to BCC>> >> Hi Mara, >> >> Glad to hear from you, you may discuss the details with William. >> Thx. >> >> Best regards, >> Alex Lv >> >> From: Mara Preotescu [mailto:mara.preotescu@nielsen >> .com<mailto:mara.preote...@nielsen.com >> <mara.preote...@nielsen.com%3Cmailto:mara.preote...@nielsen.com>>] >> Sent: 2017年7月28日 6:14 >> To: Lv, Alex <lzhix...@ebay.com<mailto:lzhix...@ebay.com>>; Vaidya, >> Amber <amvai...@ebay.com<mailto:amvai...@ebay.com>> >> Subject: Griffin support & roadmap >> >> Hello Alex, Amber, >> >> I am writing you trying to reach the support for Griffin, both support >> e-mails for the product returned as invalid addresses ( >> subscribe-...@griffin.incubator.apache.org<mailto:subscribe >> -dev@griffin.incubator.apache.org>, ebay-griffin-d...@googlegroups.com >> <mailto:ebay-griffin-d...@googlegroups.com>). >> >> Could you please let me know who should we contact to discuss about >> Griffin's roadmap? >> >> We are looking, here at Nielsen, to use the Griffin framework for our DQ >> processes. As of today we learned, and tested, the only dimension >> available, Accuracy. Would you be able to share the roadmap for any other >> DQ dimensions availability? >> >> We are looking as well to add a few custom validations - does the tool >> offer any APIs that can be used for this purpose? >> >> Any information you could provide would be very, very helpful. >> >> >> Thank you in advance for your help and time. >> Mara Preotescu >> VP Technology, DevOps Nielsen >> >> >> >> >
