Re: Meeting minutes with Nielsen:

2017-08-03 Thread William Guo 
hi Ananthanarayanan,


For profiling, we have developed some samples based on our measures. Will make 
samples available in our repo next week.


Accuracy, we have test cases for it, but we will also make samples in our repo 
next week.


Thanks,

William


From: Ananthanarayanan Ms 
Sent: Friday, August 4, 2017 12:30:22 AM
To: Mara Preotescu
Cc: William Guo; dev@griffin.incubator.apache.org; Kunduru, Abishek
Subject: Re: Meeting minutes with Nielsen:

Hi William/Lionel,
  Could you please help us to understand on the profiling feature availability, 
we see the code traces on profiling, could you let us know if we can starting 
using it from some branch which could be used so that we could leverage griffin 
on two dimensions (accuracy & profiling). If its not available a tentative date 
so that we could decide upon the same.


Regards,
Ananthanarayanan.M.S

On Wed, Aug 2, 2017 at 10:47 PM, Mara Preotescu 
mailto:mara.preote...@nielsen.com>> wrote:
Thank you William.

Ananth will follow up with a few more questions on the roadmap.

Thanks,
Mara

On Wed, Aug 2, 2017 at 9:20 AM, William Guo 
mailto:gu...@apache.org>> wrote:


Meeting minutes with Nielsen:


  *   Discuss griffin to support filters for metastore tables or navigation 
assistance for table selection on UI.
  *   Griffin provides RESTful API for backend.
  *   Discuss griffin to support multiple source or target tables.
  *   Discuss more supporting file types, such as parquet.
  *   In griffin, the partition field is optional, it just helps to provide the 
specific part of data, it will get all the data of a table without any 
partition information.
  *   Config json file provides the parameters for griffin measure calculation, 
you can also submit a spark job with it directly.
  *   Currently, griffin can only reuse measure, not rule. We’ll discuss about 
this, if we need to support reusing rules.
  *   Sample ratio field in config file is optional, in batch mode we don’t 
need to configure it.
  *   In griffin, mapping of columns are limited, discuss to support advanced 
features like joining between tables , or advanced sql script.
  *   At current, the rule parser doesn’t support customized rules, griffin has 
the plan to support this. //TODO document it and send it to dev list
  *   Griffin doesn’t support metrics alert function, it posts all the metrics 
to elasticsearch, es supports such feature. //TODO, write a solution for it 
based on elastic search
  *   In griffin, you can’t modify the exist rules or measure at current.



Thanks,
William



From: William GUO mailto:guo...@outlook.com>> on behalf of 
William Guo mailto:gu...@apache.org>>
Sent: Wednesday, August 2, 2017 10:02:15 AM
To: Mara Preotescu
Cc: dev@griffin.incubator.apache.org; 
Ananthanarayanan Ms; Kunduru, Abishek
Subject: Re: Griffin support & roadmap

hi mara,


Are you join?


Thanks,

William


From: Mara Preotescu 
mailto:mara.preote...@nielsen.com>>
Sent: Monday, July 31, 2017 11:22:00 PM
To: William Guo
Cc: dev@griffin.incubator.apache.org; 
Ananthanarayanan Ms; Kunduru, Abishek
Subject: Re: Griffin support & roadmap

Hi William,

Would 10:00 am CST (Beijing) work for you on Wednesday 08/02?

Thanks,
Mara

On Sun, Jul 30, 2017 at 10:59 PM, William Guo 
mailto:gu...@apache.org>>>
 wrote:

hi Mara,


We are in China, it is hard to arrange a meeting for US, CHINA, INDIA together.


China day time is fine for me.


Thanks,

William


From: Mara Preotescu 
mailto:mara.preote...@nielsen.com>>>
Sent: Monday, July 31, 2017 10:54:25 AM

To: William Guo
Cc: Lv, Alex; Guo, William; 
dev@griffin.incubator.apache.org>
Subject: Re: Griffin support & roadmap

Hi William,

Either Wednesday or Thursday will work for us.  Any better time working for 
you?   What time zone are you in?   I am in US ET time, a colleague of mine who 
I would like to join our discussion is in India, Chennai.

Thanks,
Mara

On Sun, Jul 30, 2017 at 7:34 PM, William Guo 
mailto:gu...@apache.org>>>
 wrote:

hi Mara,


Sure, We could schedule a meeting to discuss background, requirements, status 
and milestone.


We should be fine in Wednesday or Thursday, what is your proposal?



Thanks,

William


From: Mara Preotescu 
mailto:mara.preote...@nielsen.com>>>
Sent: Friday, July 28, 2017 7:57:45 PM
To: William Guo
Cc: Lv, Alex; Guo, William; 
dev@griffin.incubator.apache.org

Re: Meeting minutes with Nielsen:

2017-08-03 Thread Ananthanarayanan Ms 
Hi William/Lionel,
  Could you please help us to understand on the *profiling* feature
availability, we see the code traces on profiling, could you let us know if
we can starting using it from some branch which could be used so that we
could leverage griffin on two dimensions (accuracy & profiling). If its not
available a tentative date so that we could decide upon the same.


Regards,
Ananthanarayanan.M.S

On Wed, Aug 2, 2017 at 10:47 PM, Mara Preotescu 
wrote:

> Thank you William.
>
> Ananth will follow up with a few more questions on the roadmap.
>
> Thanks,
> Mara
>
> On Wed, Aug 2, 2017 at 9:20 AM, William Guo  wrote:
>
>>
>> Meeting minutes with Nielsen:
>>
>>
>>
>>- Discuss griffin to support filters for metastore tables or
>>navigation assistance for table selection on UI.
>>- Griffin provides RESTful API for backend.
>>- Discuss griffin to support multiple source or target tables.
>>- Discuss more supporting file types, such as parquet.
>>- In griffin, the partition field is optional, it just helps to
>>provide the specific part of data, it will get all the data of a table
>>without any partition information.
>>- Config json file provides the parameters for griffin measure
>>calculation, you can also submit a spark job with it directly.
>>- Currently, griffin can only reuse measure, not rule. We’ll discuss
>>about this, if we need to support reusing rules.
>>- Sample ratio field in config file is optional, in batch mode we
>>don’t need to configure it.
>>- In griffin, mapping of columns are limited, discuss to support
>>advanced features like joining between tables , or advanced sql script.
>>- At current, the rule parser doesn’t support customized rules,
>>griffin has the plan to support this. //TODO document it and send it to 
>> dev
>>list
>>- Griffin doesn’t support metrics alert function, it posts all the
>>metrics to elasticsearch, es supports such feature. //TODO, write a
>>solution for it based on elastic search
>>- In griffin, you can’t modify the exist rules or measure at current.
>>
>>
>>
>>
>> Thanks,
>> William
>>
>> --
>> *From:* William GUO  on behalf of William Guo <
>> gu...@apache.org>
>> *Sent:* Wednesday, August 2, 2017 10:02:15 AM
>> *To:* Mara Preotescu
>> *Cc:* dev@griffin.incubator.apache.org; Ananthanarayanan Ms; Kunduru,
>> Abishek
>> *Subject:* Re: Griffin support & roadmap
>>
>> hi mara,
>>
>>
>> Are you join?
>>
>>
>> Thanks,
>>
>> William
>>
>> 
>> From: Mara Preotescu 
>> Sent: Monday, July 31, 2017 11:22:00 PM
>> To: William Guo
>> Cc: dev@griffin.incubator.apache.org; Ananthanarayanan Ms; Kunduru,
>> Abishek
>> Subject: Re: Griffin support & roadmap
>>
>> Hi William,
>>
>> Would 10:00 am CST (Beijing) work for you on Wednesday 08/02?
>>
>> Thanks,
>> Mara
>>
>> On Sun, Jul 30, 2017 at 10:59 PM, William Guo > gu...@apache.org>> wrote:
>>
>> hi Mara,
>>
>>
>> We are in China, it is hard to arrange a meeting for US, CHINA, INDIA
>> together.
>>
>>
>> China day time is fine for me.
>>
>>
>> Thanks,
>>
>> William
>>
>> 
>> From: Mara Preotescu > mara.preote...@nielsen.com>>
>> Sent: Monday, July 31, 2017 10:54:25 AM
>>
>> To: William Guo
>> Cc: Lv, Alex; Guo, William; dev@griffin.incubator.apache.org> dev@griffin.incubator.apache.org>
>> Subject: Re: Griffin support & roadmap
>>
>> Hi William,
>>
>> Either Wednesday or Thursday will work for us.  Any better time working
>> for you?   What time zone are you in?   I am in US ET time, a colleague of
>> mine who I would like to join our discussion is in India, Chennai.
>>
>> Thanks,
>> Mara
>>
>> On Sun, Jul 30, 2017 at 7:34 PM, William Guo > gu...@apache.org>> wrote:
>>
>> hi Mara,
>>
>>
>> Sure, We could schedule a meeting to discuss background, requirements,
>> status and milestone.
>>
>>
>> We should be fine in Wednesday or Thursday, what is your proposal?
>>
>>
>>
>> Thanks,
>>
>> William
>>
>> 
>> From: Mara Preotescu > mara.preote...@nielsen.com>>
>> Sent: Friday, July 28, 2017 7:57:45 PM
>> To: William Guo
>> Cc: Lv, Alex; Guo, William; dev@griffin.incubator.apache.org> dev@griffin.incubator.apache.org>
>> Subject: Re: Griffin support & roadmap
>>
>> HI Alex, William,
>>
>> THANK YOU so much for your responses.  Thank you for the links.  And, I
>> hope you don't mind if I'll take up your offer to contact you if needed.
>> We are considering, here at Nielsen, using Griffin for our new Data Quality
>> framework  ... we know the project is still in the incubator but we would
>> like give it a try and even contributing, if needed.   We already install
>> it and ran a few tests.
>>
>> If your time permits I would like scheduling a quick call so we could
>> understand the current status and, most importantly if the roadmap stays as
>> in the published documents.
>>
>> Thanks again,
>> Mara
>>
>>
>> On Fri, Ju

Re: hipchat

2017-08-03 Thread William Guo 

Unfortunately, it seems I cannot invite members to join this channel. 'John 
Ament' is the only admin for asf group.


Seems I only can invite people already in asf group.


Members need to login asf first, then I can invite.




From: Henry Saputra 
Sent: Thursday, August 3, 2017 1:36:55 AM
To: dev@griffin.incubator.apache.org
Subject: Re: hipchat

Problem with Slack that you need an invite to join the Team.

Do you know if ASF Slack team has auto invite?

On Mon, Jul 31, 2017 at 4:48 PM, William Guo  wrote:

> hi all,
>
>
> I have created our griffin slack channel under the-asf.slack.com.
>
>
> https://the-asf.slack.com/?redir=%2Fmessages%2FC6G77MZM0
>
>
> Please join and discuss if necessary.
>
>
>
> Thanks,
>
> William
>
>
> 
> From: William GUO 
> Sent: Tuesday, August 1, 2017 7:20:10 AM
> To: dev@griffin.incubator.apache.org
> Subject: Re: hipchat
>
> Slack is not blocked yet by China, we can use slack as real-time
> communication tool so far.
>
>
> Who knows what will happen in the future.
>
>
> Thanks,
>
> William
>
> 
> From: Henry Saputra 
> Sent: Tuesday, August 1, 2017 2:46:38 AM
> To: dev@griffin.incubator.apache.org
> Subject: Re: hipchat
>
> Is the Slack or Gitter blocked in China?
>
> If not we could probably set channel in one of the collab app above to get
> people real-time interactions.
>
> - Henry
>
> On Thu, Jul 27, 2017 at 4:11 PM, William Guo  wrote:
>
> > hi guys,
> >
> >
> > We can chat with hipchat apps , recommended chat solution in apache
> > community.
> >
> > You can find me by https://apache.hipchat.com/chat/user/5149522 or
Log in | HipChat
apache.hipchat.com
Log in to HipChat with your work email address. New to HipChat? Want to create 
a new team? Sign up for free at hipchat.com.



> mobile
> > apps.
> >
> >
> >
> >
> >
> > Thanks,
> >
> > William
> >
>