[Discuss]what is next for apache griffin

[William Guo]

hi all,

After discussed with some IPMCs, we think the next prioritized items for
apache griffin are:

1. more connectors as google cloud, es,...
2. simplify the deployment process.
3. provides more integration models.

What do you think?


Thanks,
William

Re: [jira] [Created] (GRIFFIN-214) [Measure] Support elasticsearch as data source.

[William Guo]

right, do you want to implement es connector?

Thanks,
William

On Tue, Nov 20, 2018 at 6:12 PM xiangrong,chen (JIRA) 
wrote:

> xiangrong,chen created GRIFFIN-214:
> --
>
>  Summary: [Measure] Support elasticsearch as data source.
>  Key: GRIFFIN-214
>  URL: https://issues.apache.org/jira/browse/GRIFFIN-214
>  Project: Griffin (Incubating)
>   Issue Type: Improvement
> Reporter: xiangrong,chen
>
>
> sometimes, user may choose elasticsearch as datasource, so supporting
> elasticsearch as data source may be necessary.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>

[jira] [Resolved] (GRIFFIN-212) [Measure] Add matchedFraction in streaming accuracy metrics as in batch mode

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-212.
-
   Resolution: Fixed
Fix Version/s: 0.3.1-incubating

Issue resolved by pull request 455
[https://github.com/apache/incubator-griffin/pull/455]

> [Measure] Add matchedFraction in streaming accuracy metrics as in batch mode
> 
>
> Key: GRIFFIN-212
> URL: https://issues.apache.org/jira/browse/GRIFFIN-212
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 0.3.1-incubating
>
>
> In batch mode, we've added matchedFraction in accuracy metric, we should keep 
> the metric format in streaming mode align with batch mode. So we need to add 
> this field in streaming accuracy metric as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [RESULT][VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Hi Justin,

Could you have a look whether Apache Griffin miss anything in the agenda
before board meeting?

Henry should have updated the resolution part for apache griffin graduation.

Please tell us if we miss anything, so we can update it accordingly before
the meeting.


Thanks,
William




On Wed, Nov 14, 2018 at 4:23 PM William Guo  wrote:

> Henry, Thanks!
>
> On Wed, Nov 14, 2018 at 2:03 PM Henry Saputra 
> wrote:
>
>> Hi Justin,
>>
>> I can add it to the resolution.
>>
>> Should I add it as New Resolution item?
>>
>> - Henry
>>
>> On Tue, Nov 13, 2018 at 12:45 PM Justin Mclean 
>> wrote:
>>
>> > Hi,
>> >
>> > I had you down as graduating this month but don't see the resolution on
>> > the board agenda yet. [1] Is someone on top of it?
>> >
>> > Thanks,
>> > Justin
>> >
>> > 1. https://whimsy.apache.org/board/agenda/2018-11-21/
>> >
>>
>

[jira] [Resolved] (GRIFFIN-211) [Service] JobInstance appUrl error

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-211.
-
Resolution: Fixed

> [Service] JobInstance appUrl error
> --
>
> Key: GRIFFIN-211
> URL: https://issues.apache.org/jira/browse/GRIFFIN-211
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: xiangrong,chen
>Assignee: xiangrong,chen
>Priority: Major
> Attachments: WeChat Image_20181116093345.png
>
>
> when I try to get jobInstance by calling API 
> "http://localhost:8080/api/v1/jobs/instances?jobId=8=0=10;, 
> response is:
> {
>  "id": 212,
>  "sessionId": 53,
>  "state": "SUCCESS",
>  "type": "BATCH",
>  "appId": "application_1542252279758_0053",
>  "appUri": "{color:#33}http://griffin:8088{color} 
> {color:#FF}/cluster/app/{color} application_1542252279758_0053",
>  "predicateGroup": "PG",
>  "predicateName": "accu-job1_predicate_154233028",
>  "timestamp": 154233028,
>  "expireTimestamp": 1542934800028
>  },
> I find there are 2 blanks on both sides of '/cluster/app/', and it make this 
> url can not work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (GRIFFIN-210) Measure need to integrate with upstream/downstream nodes when bad records are founded

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo updated GRIFFIN-210:

Summary: Measure need to integrate with upstream/downstream nodes when bad 
records are founded  (was: Measure need to notify up/down sides when bad 
records are founded)

> Measure need to integrate with upstream/downstream nodes when bad records are 
> founded
> -
>
> Key: GRIFFIN-210
> URL: https://issues.apache.org/jira/browse/GRIFFIN-210
> Project: Griffin (Incubating)
>  Issue Type: Wish
>        Reporter: William Guo
>    Assignee: William Guo
>Priority: Major
>
> In a typical data quality project, when apache griffin find some data quality 
> issue, usually, it need to integrate with upstream or downstream nodes
> it have labelled some potential records as "need to recover", it should 
> integrate with upstream or downstream to do some remedy action  automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (GRIFFIN-210) Measure need to integrate with upstream/downstream nodes when bad records are founded

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo updated GRIFFIN-210:

Description: 
In a typical data quality project, when Apache Griffin find some data quality 
issue, usually, it need to integrate with upstream or downstream nodes
it have labelled some potential records as "need to recover", it should 
integrate with upstream or downstream to do some remedy action  automatically.

  was:
In a typical data quality project, when apache griffin find some data quality 
issue, usually, it need to integrate with upstream or downstream nodes
it have labelled some potential records as "need to recover", it should 
integrate with upstream or downstream to do some remedy action  automatically.


> Measure need to integrate with upstream/downstream nodes when bad records are 
> founded
> -
>
> Key: GRIFFIN-210
> URL: https://issues.apache.org/jira/browse/GRIFFIN-210
> Project: Griffin (Incubating)
>  Issue Type: Wish
>    Reporter: William Guo
>Assignee: William Guo
>Priority: Major
>
> In a typical data quality project, when Apache Griffin find some data quality 
> issue, usually, it need to integrate with upstream or downstream nodes
> it have labelled some potential records as "need to recover", it should 
> integrate with upstream or downstream to do some remedy action  automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (GRIFFIN-210) [Measure] need to integrate with upstream/downstream nodes when bad records are founded

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo updated GRIFFIN-210:

Description: 
In a typical data quality project, when Apache Griffin find some data quality 
issue, usually, it need to integrate with upstream or downstream nodes.
So corresponding systems can have opportunities to automatically do some remedy 
action, such as retry...  


  was:
In a typical data quality project, when Apache Griffin find some data quality 
issue, usually, it need to integrate with upstream or downstream nodes
it have labelled some potential records as "need to recover", it should 
integrate with upstream or downstream to do some remedy action  automatically.


> [Measure] need to integrate with upstream/downstream nodes when bad records 
> are founded
> ---
>
> Key: GRIFFIN-210
> URL: https://issues.apache.org/jira/browse/GRIFFIN-210
> Project: Griffin (Incubating)
>  Issue Type: Wish
>Reporter: William Guo
>Assignee: William Guo
>Priority: Major
>
> In a typical data quality project, when Apache Griffin find some data quality 
> issue, usually, it need to integrate with upstream or downstream nodes.
> So corresponding systems can have opportunities to automatically do some 
> remedy action, such as retry...  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (GRIFFIN-210) [Measure] need to integrate with upstream/downstream nodes when bad records are founded

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo updated GRIFFIN-210:

Summary: [Measure] need to integrate with upstream/downstream nodes when 
bad records are founded  (was: Measure need to integrate with 
upstream/downstream nodes when bad records are founded)

> [Measure] need to integrate with upstream/downstream nodes when bad records 
> are founded
> ---
>
> Key: GRIFFIN-210
> URL: https://issues.apache.org/jira/browse/GRIFFIN-210
> Project: Griffin (Incubating)
>  Issue Type: Wish
>        Reporter: William Guo
>    Assignee: William Guo
>Priority: Major
>
> In a typical data quality project, when Apache Griffin find some data quality 
> issue, usually, it need to integrate with upstream or downstream nodes
> it have labelled some potential records as "need to recover", it should 
> integrate with upstream or downstream to do some remedy action  automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (GRIFFIN-210) Measure need to notify up/down sides when bad records are founded

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo updated GRIFFIN-210:

Description: 
In a typical data quality project, when apache griffin find some data quality 
issue, usually, it need to integrate with upstream or downstream nodes
it have labelled some potential records as "need to recover", it should 
integrate with upstream or downstream to do some remedy action  automatically.

  was:In typical data quality project, when griffin find some data quality 
issue, it have labelled some potential records as "need to recover", it should 
integrate with upstream or downstream to do some remedy action  automatically.


> Measure need to notify up/down sides when bad records are founded
> -
>
> Key: GRIFFIN-210
> URL: https://issues.apache.org/jira/browse/GRIFFIN-210
> Project: Griffin (Incubating)
>  Issue Type: Wish
>Reporter: William Guo
>Assignee: William Guo
>Priority: Major
>
> In a typical data quality project, when apache griffin find some data quality 
> issue, usually, it need to integrate with upstream or downstream nodes
> it have labelled some potential records as "need to recover", it should 
> integrate with upstream or downstream to do some remedy action  automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (GRIFFIN-210) Measure need to notify up/down sides when bad records are founded

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo updated GRIFFIN-210:

Summary: Measure need to notify up/down sides when bad records are founded  
(was: Measure need to notify downstream when bad records are founded)

> Measure need to notify up/down sides when bad records are founded
> -
>
> Key: GRIFFIN-210
> URL: https://issues.apache.org/jira/browse/GRIFFIN-210
> Project: Griffin (Incubating)
>  Issue Type: Wish
>        Reporter: William Guo
>        Assignee: William Guo
>Priority: Major
>
> In typical data quality project, when griffin find some data quality issue, 
> it have labelled some potential records as "need to recover", it should 
> integrate with upstream or downstream to do some remedy action  automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (GRIFFIN-210) Measure need to notify downstream when bad records are founded

[William Guo (JIRA)]

William Guo created GRIFFIN-210:
---

 Summary: Measure need to notify downstream when bad records are 
founded
 Key: GRIFFIN-210
 URL: https://issues.apache.org/jira/browse/GRIFFIN-210
 Project: Griffin (Incubating)
  Issue Type: Wish
Reporter: William Guo
Assignee: William Guo


In typical data quality project, when griffin find some data quality issue, it 
have labelled some potential records as "need to recover", it should integrate 
with upstream or downstream to do some remedy action  automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (GRIFFIN-209) [Measure] In paramUtil, the util function getParamStringMap doesn't work as expected

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-209.
-
Resolution: Fixed

Issue resolved by pull request 452
[https://github.com/apache/incubator-griffin/pull/452]

> [Measure] In paramUtil, the util function getParamStringMap doesn't work as 
> expected
> 
>
> Key: GRIFFIN-209
> URL: https://issues.apache.org/jira/browse/GRIFFIN-209
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 0.3.1-incubating
>
>
> Need to update paramUtil, to make getParamStringMap and the other util 
> functions work as expected.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [RESULT][VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Henry, Thanks!

On Wed, Nov 14, 2018 at 2:03 PM Henry Saputra 
wrote:

> Hi Justin,
>
> I can add it to the resolution.
>
> Should I add it as New Resolution item?
>
> - Henry
>
> On Tue, Nov 13, 2018 at 12:45 PM Justin Mclean  wrote:
>
> > Hi,
> >
> > I had you down as graduating this month but don't see the resolution on
> > the board agenda yet. [1] Is someone on top of it?
> >
> > Thanks,
> > Justin
> >
> > 1. https://whimsy.apache.org/board/agenda/2018-11-21/
> >
>

[DISCUSS] Build another release for 0.4.0

[William Guo]

hi all,

We have implemented several features and fix a lot of bugs recently,

I think it it time for apache griffin to build 0.4.0 release, what do you
think?


Thanks,
William

Re: [RESULT][VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

hi Justin,

I cannot access this, I think our mentors will look into this.



Thanks,
William

On Wed, Nov 14, 2018 at 4:45 AM Justin Mclean  wrote:

> Hi,
>
> I had you down as graduating this month but don't see the resolution on
> the board agenda yet. [1] Is someone on top of it?
>
> Thanks,
> Justin
>
> 1. https://whimsy.apache.org/board/agenda/2018-11-21/
>

[jira] [Resolved] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-200.
-
Resolution: Fixed

> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>Assignee: Eugene
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface GriffinHook {
> void onEvent(GriffinHookEvent event) throws Exception;
> }
> public interface GriffinHookEvent { ... }
> public interface JobEvent implements GriffinHookEvent { ... } 
> public class BeforeJobCreated implements JobEvent { ... }
> public class AfterJobCreated implements JobEvent { ... }
> public class BeforeJobDeleted implements JobEvent { ... }
> public class AfterJobDeleted implements JobEvent { ... }
> public interface JobInstanceEvent implements GriffinHookEvent { ... }
> public class BeforeJobInstanceStart implements JobInstanceEvent { ... }
> public class AfterJobInstanceEnd implements JobInstanceEvent { ... }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[RESULT][VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Hi all,

With 13 +1 non-binding votes, 5 +1 binding votes and no -1 or +/-0 votes,
This vote PASSES.

Thanks to everyone who voted.
The vote tally is attached.

Non-binding:
邵亮
Jenny Li
ShaoFeng Shi
Eugene Liu
Sheng Wu
Alex Lv
Yin Wan
John Liu
Kevin Yao
Matt Sicker
Xin Wang
Li De
He Wang


Binding:
Willem Jiang
Jean-Baptiste Onofré
Dave Fisher
Uma Gangumalla
Henry Saputra


Thanks,
William.

Fwd: [VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

+dev

-- Forwarded message -
From: William Guo 
Date: Thu, Nov 1, 2018 at 11:21 AM
Subject: [VOTE] Graduate Apache Griffin (incubating) as a TLP
To: 


Hi all,

Given that we've got positive feedback on the DISCUSS
thread, after we had made some changes based on DISCUSS feedback.
I'd like to start an official VOTE thread now.


Please vote on the resolution pasted below to graduate
Apache Griffin from the incubator to the top level project.

[ ] +1 Graduate Apache Griffin from the Incubator.
[ ] +0 Don't care.
[ ] -1 Don't graduate Apache Griffin from the Incubator because...

This vote will be open for at least 72 hours.


Many thanks to our mentors and everyone else for their support,

William Guo (on behalf of the Apache Griffin PPMC).


## Resolution to create a TLP from graduating Incubator podling

X. Establish the Apache Griffin Project

WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation's purpose to establish
a Project Management Committee charged with the creation and maintenance
of open-source software, for distribution at no charge to the public,
related to a data quality solution for big data, including both streaming
and batch mode.


NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache Griffin Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further

RESOLVED, that the Apache Griffin Project be and hereby is
responsible for the creation and maintenance of software related to
a data quality solution for big data, including both streaming and batch
mode;
and be it further

RESOLVED, that the office of "Vice President, Apache Griffin" be and
hereby is created, the person holding such office to serve at the
direction of the Board of Directors as the chair of the Apache
Griffin Project, and to have primary responsibility for management
of the projects within the scope of responsibility of the Apache
Griffin Project; and be it further

RESOLVED, that the persons listed immediately below be and hereby are
appointed to serve as the initial members of the Apache Griffin Project:

* Alex Lv 
* Deyi Yao 
* Eugene Liu 
* Grant Guo 
* He Wang 
* Henry Saputra 
* Jason Liao 
* John Liu 
* Juan Li 
* Liang Shao 
* Lionel Liu 
* Luciano Resende 
* Nick Sokolov 
* Shawn Sha 
* Vincent Zhao 
* William Guo 
* Yuqin Xuan 

NOW, THEREFORE, BE IT FURTHER RESOLVED, that William Guo be
appointed to the office of Vice President, Apache Griffin, to serve
in accordance with and subject to the direction of the Board of
Directors and the Bylaws of the Foundation until death, resignation,
retirement, removal or disqualification, or until a successor is
appointed; and be it further

RESOLVED, that the Apache Griffin Project be and hereby is tasked
with the migration and rationalization of the Apache Incubator
Griffin podling; and be it further

RESOLVED, that all responsibilities pertaining to the Apache Incubator
Griffin podling encumbered upon the Apache Incubator PMC are
here after discharged.

[jira] [Resolved] (GRIFFIN-207) LDAP auth is not supporting group filters and non-CN login names

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-207.
-
   Resolution: Fixed
Fix Version/s: 0.3.1-incubating

Issue resolved by pull request 441
[https://github.com/apache/incubator-griffin/pull/441]

> LDAP auth is not supporting group filters and non-CN login names
> 
>
> Key: GRIFFIN-207
> URL: https://issues.apache.org/jira/browse/GRIFFIN-207
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Nikolay Sokolov
>Assignee: Nikolay Sokolov
>Priority: Major
> Fix For: 0.3.1-incubating
>
>
> Currently LDAP auth performs bind to principal with name 
> "${username}${ldap.email}", and searches through user objects 
> ldap.searchPattern. Result of search then only used to retrieve fullName of 
> the user.
> There are two problems here:
>  * login username can not be different than CN, as it is used to perform LDAP 
> bind
>  * it is not possible to restrict access to specific groups
> Typical approach used in other software products is to use separate bind 
> account, which would search through LDAP objects using search pattern, and 
> then use found object's DN to perform password check.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Hooks support in JobService

[William Guo]

Sounds good to me.

   - application.properties would have a property containing list of hook
   class names (or probably, spring bean names?), configuring subset and
order
   of hook instances enabled at deployment time.
[William] I like a logic name as spring bean name.

   - Hook instantiation mechanism should create them as proper spring
   beans, so that they could use same property file for configuration, and
   access existing repositories and services. At the same time, only
   explicitly enabled hooks should be instantiated, to avoid initializing
any
   optional heavy hooks each time.
[William] We can launch our hook mechanism by leveraging spring bean
instantializtion mechanism.

   - Hooks would be implementing GriffinHook interface, and particular
   implementation would use any kind of internal logic, from spring
   integration to some asynchronous handling.
[William] Sounds good to me.

   - Hook interface would have single method accepting subclass of
   GriffinHookEvent. Hook implementation would pick events it would like to
   react on, and ignore the others. To give it some structure, events would
be
   organized in hierarchy, like JobEvents, JobInstanceEvents, MeasureEvents,
   etc.
[William] Events are common parts for the Event driven flow.

   - New extension points would be added by introducing new types of events
   corresponding to some point inside griffin services. Service instance
would
   be creating corresponding event object, and call onEvent method on all
hook
   instances defined in .properties file, according to defined order.
[William] Let's start with basic events, get it run in prototype, we can
refine this later.



On Mon, Oct 22, 2018 at 2:51 PM Nick Sokolov  wrote:

> I think this is best detail I can give right now without writing any code:
>
>- application.properties would have a property containing list of hook
>class names (or probably, spring bean names?), configuring subset and
> order
>of hook instances enabled at deployment time.
>- Hook instantiation mechanism should create them as proper spring
>beans, so that they could use same property file for configuration, and
>access existing repositories and services. At the same time, only
>explicitly enabled hooks should be instantiated, to avoid initializing
> any
>optional heavy hooks each time.
>- Hooks would be implementing GriffinHook interface, and particular
>implementation would use any kind of internal logic, from spring
>integration to some asynchronous handling.
>- Hook interface would have single method accepting subclass of
>GriffinHookEvent. Hook implementation would pick events it would like to
>react on, and ignore the others. To give it some structure, events
> would be
>organized in hierarchy, like JobEvents, JobInstanceEvents,
> MeasureEvents,
>etc.
>- New extension points would be added by introducing new types of events
>corresponding to some point inside griffin services. Service instance
> would
>be creating corresponding event object, and call onEvent method on all
> hook
>instances defined in .properties file, according to defined order.
>
> Probably it's time to build some technical prototype.
>
> On Sun, Oct 21, 2018 at 6:46 PM Eugene Liu  wrote:
>
> > Guys,
> >
> > you have agree to integration style that keeps two potential solutions,
> > Hive/Spring, leaves final decision to users. I'm fine on this agreement.
> >
> > but do you have comprehensive design schema, I think the topic discussion
> > is not closing in jira, we should keep in same page about
> > event/message/sync/async details.
> >
> > thanks
> > 
> > From: William Guo 
> > Sent: Monday, October 22, 2018 8:26 AM
> > To: dev@griffin.incubator.apache.org
> > Subject: Re: Hooks support in JobService
> >
> > Sounds good to me.
> >
> > You can implement your preferred hive-style, I can implement spring
> > integration based on your interface.
> >
> > William
> >
> > On Mon, Oct 22, 2018 at 2:10 AM Nick Sokolov 
> wrote:
> >
> > > Hi William,
> > >
> > > Totally agree on keeping interface as abstract as possible. Interfaces
> > feel
> > > like natural solution here.
> > > It should be relatively easy to provide spring integration
> implementation
> > > of hook interface.
> > > Probably we can even have default spring integration hook provided as
> an
> > > example.
> > >
> > > On Sun, Oct 21, 2018 at 6:33 AM William Guo  wrote:
> > >
> > > > hi Nick,
> > > >
> > > > I agree with you we need to have some lifecycle hooks for job

Re: Hooks support in JobService

[William Guo]

Sounds good to me.

You can implement your preferred hive-style, I can implement spring
integration based on your interface.

William

On Mon, Oct 22, 2018 at 2:10 AM Nick Sokolov  wrote:

> Hi William,
>
> Totally agree on keeping interface as abstract as possible. Interfaces feel
> like natural solution here.
> It should be relatively easy to provide spring integration implementation
> of hook interface.
> Probably we can even have default spring integration hook provided as an
> example.
>
> On Sun, Oct 21, 2018 at 6:33 AM William Guo  wrote:
>
> > hi Nick,
> >
> > I agree with you we need to have some lifecycle hooks for jobs.
> >
> > No matter which extension mechanism we choose at runtime,
> > but we can always abstract common lifecycle interfaces, and implements
> > different solutions for different environments.
> >
> > Say, at runtime someone can configure griffin extension mechanism as
> > 'Hive-style' and others can configure as spring integration model.
> >
> > We cannot decide which one is super than other, but we can implement
> > multiply solutions and let users decide at deployment time?
> >
> > What do you think?
> >
> > Thanks,
> > William
> >
> >
> >
> > On Fri, Oct 19, 2018 at 2:03 PM Nick Sokolov 
> wrote:
> >
> > > Hi all,
> > >
> > > I'd like invite the community to discussion of Job object lifecycle
> hooks
> > > design (GRIFFIN-200 <https://issues.apache.org/jira/browse/GRIFFIN-200
> > >).
> > >
> > > On a high level, idea is to have hooks executed triggered by JobService
> > > before/after job is created, or before/after job is deleted, and so on.
> > > Main point is to provide extension mechanism for JobService, allowing
> > > third-party implementations of following features:
> > >
> > >- validating naming conventions on the jobs
> > >- validating cron schedules (whether job is scheduled "not too
> > >frequently")
> > >- creating dashboards for each new job and/or alerts in third-party
> > >system (Grafana, Elastalert)
> > >- RBAC implementation (Hive-style, as plugin)
> > >- disabling alerts in third-party system if job is paused
> > >
> > > Right now several aspects are to be decided:
> > >
> > >1. particular extention mehanism (POJO interface implementations
> > >("Hive-style"), or Spring-Integration, etc)
> > >2. whether only one or multiple hooks could be configured at same
> time
> > >3. execution model: whether they should run synchronously, or
> > >asynchronously; sequentially, or in parallel
> > >
> > > Ticket already has some discussion going, so inviting everyone to Jira
> > > ticket, to avoid having same conversation in two places.
> > >
> >
>

Re: Hooks support in JobService

[William Guo]

hi Nick,

I agree with you we need to have some lifecycle hooks for jobs.

No matter which extension mechanism we choose at runtime,
but we can always abstract common lifecycle interfaces, and implements
different solutions for different environments.

Say, at runtime someone can configure griffin extension mechanism as
'Hive-style' and others can configure as spring integration model.

We cannot decide which one is super than other, but we can implement
multiply solutions and let users decide at deployment time?

What do you think?

Thanks,
William



On Fri, Oct 19, 2018 at 2:03 PM Nick Sokolov  wrote:

> Hi all,
>
> I'd like invite the community to discussion of Job object lifecycle hooks
> design (GRIFFIN-200 ).
>
> On a high level, idea is to have hooks executed triggered by JobService
> before/after job is created, or before/after job is deleted, and so on.
> Main point is to provide extension mechanism for JobService, allowing
> third-party implementations of following features:
>
>- validating naming conventions on the jobs
>- validating cron schedules (whether job is scheduled "not too
>frequently")
>- creating dashboards for each new job and/or alerts in third-party
>system (Grafana, Elastalert)
>- RBAC implementation (Hive-style, as plugin)
>- disabling alerts in third-party system if job is paused
>
> Right now several aspects are to be decided:
>
>1. particular extention mehanism (POJO interface implementations
>("Hive-style"), or Spring-Integration, etc)
>2. whether only one or multiple hooks could be configured at same time
>3. execution model: whether they should run synchronously, or
>asynchronously; sequentially, or in parallel
>
> Ticket already has some discussion going, so inviting everyone to Jira
> ticket, to avoid having same conversation in two places.
>

Re: [DISCUSS] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Hi Both,

We have updated name search ticket
https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-148

We think 'apache griffin' should be ok for us, What do you think?

Thanks,
William

On Fri, Oct 19, 2018 at 1:59 PM Lionel Liu  wrote:

> Hi all,
>
> We've updated the Podling Suitable Name Search ticket:
> https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-148
>
> BTW, I think we can remove the "creation of a set of bylaws" line, to just
> follow the ASF bylaws.
>
> Thanks,
> Lionel
>
> On Thu, Oct 18, 2018 at 10:37 AM Henry Saputra 
> wrote:
>
> > Hi Dave,
> >
> > You are right. Will wait until the name search been resolved.
> >
> > - Henry
> >
> > On Wed, Oct 17, 2018 at 8:46 AM Dave Fisher 
> wrote:
> >
> > > Hi -
> > >
> > > The Podling Suitable Name Search is unresolved:
> > > https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-148 <
> > > https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-148>
> > >
> > > Note that links are given, but some remarks about the results are
> needed.
> > >
> > > Regards,
> > > Dave
> > >
> > > > On Oct 17, 2018, at 8:35 AM, Henry Saputra 
> > wrote:
> > > >
> > > > Thanks for driving the discussion, William.
> > > >
> > > > As one of the mentors of the podling, I support the graduation
> proposal
> > > and
> > > > looking forward for the VOTE thread.
> > > >
> > > > - Henry
> > > >
> > > > On Mon, Oct 15, 2018 at 5:51 PM William Guo 
> wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> After an enthusiastic discussion with the community:
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/ba389dd1f7a9e8c82912d4dbf06abceda5461429806f5f7b112fc05d@%3Cdev.griffin.apache.org%3E
> > > >> culminating with a positive vote:
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/df9c3a36d66c140b82ae655d49e6d3437564cb895724689ff266d954@%3Cdev.griffin.apache.org%3E
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/3fdf92ae3510b6bca3ac25f51200c0b1e48a4ea1185945733259a7b7@%3Cdev.griffin.apache.org%3E
> > > >>
> > > >> We'd like to bring this to a discussing at the IPMC.
> > > >>
> > > >> Please see the proposed resolution below and let us know what do you
> > > think.
> > > >>
> > > >> A few stats to help with the discussion:
> > > >>
> > > >> Now we have the developer team[1,2] from
> > > >> eBay, VMWARE, NetEase, Pingan Bank, Huawei,
> > > >> Grid Dynamics, Paypal, Alipay, Yitu, JD, Ontario Institute for
> Cancer
> > > >> Research.
> > > >>
> > > >> 458 commits on development of the project.
> > > >> 437 PRs on the Github
> > > >> 31 contributors
> > > >> 206+ issues created
> > > >> 180+ issues resolved
> > > >>
> > > >> dev has 76 subscribers
> > > >> 285 emails sent by 24 people, divided into 139 topics in last
> > month.[3]
> > > >>
> > > >> Please check out Apache Maturity Model Assessment for Griffin[4] For
> > > more
> > > >> information.
> > > >>
> > > >> [1]http://griffin.incubator.apache.org/docs/community.html
> > > >> [2]http://griffin.incubator.apache.org/docs/contributors.html
> > > >> [3]
> https://lists.apache.org/trends.html?d...@griffin.apache.org:2018-9
> > > >> [4]
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/GRIFFIN/ASF+Maturity+Evaluation
> > > >>
> > > >> Thanks,
> > > >> William.
> > > >>
> > > >>
> > > >> Establish the Apache Griffin Project
> > > >>
> > > >> WHEREAS, the Board of Directors deems it to be in the best
> > > >> interests of the Foundation and consistent with the
> > > >> Foundation's purpose to establish a Project Management
> > > >> Committee charged with the creation and maintenance of
> > > >> open-source software, for distribution at no charge to
> > > >> the public, related to a data quality solution for big data,
> > &

[jira] [Comment Edited] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653729#comment-16653729
 ] 

William Guo edited comment on GRIFFIN-200 at 10/17/18 3:22 PM:
---

hi Nikolay Sokolov,

I suggest you make your proposal and discuss in our dev mailing list for the 
design.
Let the community discuss and decide.
When design is fully discussed and debated, we can do a quick implementation 
for this hook.
What do you think?


Thanks,
William


was (Author: guoyp):
hi Nikolay Sokolov,

I suggest you make your proposal and discuss in our dev mailing list for the 
design.
Let the community discuss and decide.

What do you think?

Thanks,
William

> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface JobLifecycleHook {
> void onJobEvent(JobLifecycleEvent event) throws Exception;
> }
> public class BeforeJobCreated implements JobLifecycleEvent { ... }
> public class AfterJobCreated implements JobLifecycleEvent { ... }
> public class BeforeJobDeleted implements JobLifecycleEvent { ... }
> public class AfterJobDeleted implements JobLifecycleEvent { ... }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653729#comment-16653729
 ] 

William Guo edited comment on GRIFFIN-200 at 10/17/18 3:21 PM:
---

hi Nikolay Sokolov,

I suggest you make your proposal and discuss in our dev mailing list for the 
design.
Let the community discuss and decide.

What do you think?

Thanks,
William


was (Author: guoyp):
hi Nikolay Sokolov,

I suggest you make your proposal and discuss in our dev mailing list for this 
design.
Let the community discuss and decide.

What do you think?

Thanks,
William

> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface JobLifecycleHook {
> void onJobEvent(JobLifecycleEvent event) throws Exception;
> }
> public class BeforeJobCreated implements JobLifecycleEvent { ... }
> public class AfterJobCreated implements JobLifecycleEvent { ... }
> public class BeforeJobDeleted implements JobLifecycleEvent { ... }
> public class AfterJobDeleted implements JobLifecycleEvent { ... }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653729#comment-16653729
 ] 

William Guo commented on GRIFFIN-200:
-

hi Nikolay Sokolov,

I suggest you make your proposal and discuss in our dev mailing list for this 
design.
Let the community discuss and decide.

What do you think?

Thanks,
William

> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface JobLifecycleHook {
> void onJobEvent(JobLifecycleEvent event) throws Exception;
> }
> public class BeforeJobCreated implements JobLifecycleEvent { ... }
> public class AfterJobCreated implements JobLifecycleEvent { ... }
> public class BeforeJobDeleted implements JobLifecycleEvent { ... }
> public class AfterJobDeleted implements JobLifecycleEvent { ... }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Hi Bertrand,

Thanks for the feedback!  We will rephrase this in coming voting phase.

William

On Tue, Oct 16, 2018 at 4:25 PM Bertrand Delacretaz <
bdelacre...@codeconsult.ch> wrote:

> Hi,
>
> On Tue, Oct 16, 2018 at 2:51 AM William Guo  wrote:
> > ...to establish a Project Management
> > Committee charged with the creation and maintenance of
> > open-source software, for distribution at no charge to
> > the public, related to a data quality solution for big data,
> > including both streaming and batch mode. It offers an unified
> > process to measure data quality from different perspectives...
>
> I suggest removing the last phrase, "It offers an unified...", I think
> it's good to avoid being too specific in these descriptions. That
> phrase appears twice in the resolution, as usual.
>
> Apart from that, +1 on graduation!
>
> -Bertrand
>

[RESULT][VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Hi all,

Vote passed with 6[+1] binding votes, 6[+1] votes and no[-1] votes ,
Please check the following tally.

+1 : [12]
  He Wang
  Raju Goud
  Jenny Li
  Vincent Zhao(binding)
  Lionel Liu(binding)
  Yin Wan
  Kevin Yao
  Uma Maheswara Rao Gangumalla (binding)
  Lv Alex (binding)
  John Liu(binding)
  Eugene Liu
  Shao Liang (binding)


0 : [0]

-1 : [0]

The vote thread lists here:

https://lists.apache.org/thread.html/df9c3a36d66c140b82ae655d49e6d3437564cb895724689ff266d954@%3Cdev.griffin.apache.org%3E

https://lists.apache.org/thread.html/859339a7f47003a0a715293abc8d156c8b4d9a9c782cf97a6809f9c9@%3Cdev.griffin.apache.org%3E

Thanks for voting, We will start the discussion in IPMC shortly.

Thanks,
William
On Behalf of Apache Griffin PPMC

[jira] [Resolved] (GRIFFIN-202) Allow viewing of raw measure details from UI

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-202.
-
   Resolution: Fixed
Fix Version/s: 0.3.1-incubating

Issue resolved by pull request 437
[https://github.com/apache/incubator-griffin/pull/437]

> Allow viewing of raw measure details from UI
> 
>
> Key: GRIFFIN-202
> URL: https://issues.apache.org/jira/browse/GRIFFIN-202
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Nikolay Sokolov
>Assignee: Nikolay Sokolov
>Priority: Minor
> Fix For: 0.3.1-incubating
>
>
> Sometimes it's desirable to see raw JSON of specific measure. This is 
> especially useful when custom measurements are created from API.
> Two implementations are possible (one or even both can be implemented):
> * show link with URL of measure details API on UI
> * add section under "Mapping Rules", showing formatted plain text of API 
> response on UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (GRIFFIN-206) Wrong sign of time zone offset when created from UI

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-206.
-
   Resolution: Fixed
Fix Version/s: 0.3.1-incubating

Issue resolved by pull request 435
[https://github.com/apache/incubator-griffin/pull/435]

> Wrong sign of time zone offset when created from UI
> ---
>
> Key: GRIFFIN-206
> URL: https://issues.apache.org/jira/browse/GRIFFIN-206
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Nikolay Sokolov
>Assignee: Nikolay Sokolov
>Priority: Major
> Fix For: 0.3.1-incubating
>
>
> When job is created from UI, time zone offset is created by this piece of 
> code:
> {code:none}
> let time = new Date().getTimezoneOffset() / 60;
> let timezone = "GMT" + time + ":00";
> {code}
> Problem is, that for GMT-7 it yields positive number, resulting in calculated 
> timezone to be GMT+7.
> Turns out that that some formats are not parsed properly, at least UTCXX:XX 
> format used by UI is not processed correctly, and GMTXX:XX (without signs):
> {code:none}
> scala> java.util.TimeZone.getTimeZone("GMT6")
> res9: java.util.TimeZone = 
> sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> scala> java.util.TimeZone.getTimeZone("GMT+6")
> res10: java.util.TimeZone = 
> sun.util.calendar.ZoneInfo[id="GMT+06:00",offset=2160,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> scala> java.util.TimeZone.getTimeZone("UTC+6")
> res11: java.util.TimeZone = 
> sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> scala> java.util.TimeZone.getTimeZone("UTC-6(CST)")
> res12: java.util.TimeZone = 
> sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [jira] [Commented] (GRIFFIN-206) Wrong sign of time zone offset when created from UI

[William Guo]

don't worry, we can do it in angular side.

On Sun, Oct 14, 2018 at 4:17 AM ASF GitHub Bot (JIRA) 
wrote:

>
> [
> https://issues.apache.org/jira/browse/GRIFFIN-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649102#comment-16649102
> ]
>
> ASF GitHub Bot commented on GRIFFIN-206:
> 
>
> Github user chemikadze commented on a diff in the pull request:
>
>
> https://github.com/apache/incubator-griffin/pull/435#discussion_r224970940
>
> --- Diff: ui/angular/src/app/job/create-job/batch/batch.component.ts
> ---
> @@ -134,8 +134,8 @@ export class BatchComponent implements OnInit,
> AfterViewChecked {
>return false;
>  }
>  this.measureid = this.getMeasureId();
> -let time = new Date().getTimezoneOffset() / 60;
> -let timezone = "GMT" + time + ":00";
> +let time = -(new Date().getTimezoneOffset() / 60);
> --- End diff --
>
> @guoyuepeng @ashutakGG Moved to utility function. Angular is not my
> specialty, I was really surprised how much complexity Angular community has
> about it.
>
>
> > Wrong sign of time zone offset when created from UI
> > ---
> >
> > Key: GRIFFIN-206
> > URL: https://issues.apache.org/jira/browse/GRIFFIN-206
> > Project: Griffin (Incubating)
> >  Issue Type: Bug
> >Reporter: Nikolay Sokolov
> >Assignee: Nikolay Sokolov
> >Priority: Major
> >
> > When job is created from UI, time zone offset is created by this piece
> of code:
> > {code:none}
> > let time = new Date().getTimezoneOffset() / 60;
> > let timezone = "GMT" + time + ":00";
> > {code}
> > Problem is, that for GMT-7 it yields positive number, resulting in
> calculated timezone to be GMT+7.
> > Turns out that that some formats are not parsed properly, at least
> UTCXX:XX format used by UI is not processed correctly, and GMTXX:XX
> (without signs):
> > {code:none}
> > scala> java.util.TimeZone.getTimeZone("GMT6")
> > res9: java.util.TimeZone =
> sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> > scala> java.util.TimeZone.getTimeZone("GMT+6")
> > res10: java.util.TimeZone =
> sun.util.calendar.ZoneInfo[id="GMT+06:00",offset=2160,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> > scala> java.util.TimeZone.getTimeZone("UTC+6")
> > res11: java.util.TimeZone =
> sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> > scala> java.util.TimeZone.getTimeZone("UTC-6(CST)")
> > res12: java.util.TimeZone =
> sun.util.calendar.ZoneInfo[id="GMT",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
> > {code}
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>

[VOTE] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

hi all,

Please vote on the proposal for Apache Griffin graduation to TLP to submit
to
the Incubator PMC.

Vote:
[ ] +1 - Recommend Graduation of Apache Griffin as a TLP
[ ] -1 - Do not recommend graduation of Apache Griffin because ….

This vote will stay open for at least 72 hours.


At the mentors request they did a maturity model analysis [1] and wrote
contribution guidelines. [2]

The Graduation Proposal was written and discussed on the dev list. [3]

[1]
https://cwiki.apache.org/confluence/display/GRIFFIN/ASF+Maturity+Evaluation

[2] http://griffin.incubator.apache.org/docs/contribute.html

[3]
https://lists.apache.org/thread.html/ba389dd1f7a9e8c82912d4dbf06abceda5461429806f5f7b112fc05d@%3Cdev.griffin.apache.org%3E



Establish the Apache Griffin Project

WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation's purpose to establish
a Project Management Committee charged with the creation and maintenance
of open-source software, for distribution at no charge to the public,
related to a data quality solution for big data, including both streaming
and batch mode.
It offers an unified process to measure data quality from different
perspectives.

NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache Griffin Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further

RESOLVED, that the Apache Griffin Project be and hereby is responsible
for the creation and maintenance of software related to a data quality
solution for big data,
including both streaming and batch mode.
It offers an unified process to measure data quality from different
perspectives; and be it further


RESOLVED, that the office of "Vice President, Apache Griffin" be and
hereby is created, the person holding such office to serve at the
direction of the Board of Directors as the chair of the Apache Griffin
Project, and to have primary responsibility for management of the
projects within the scope of responsibility of the Apache Griffin
Project; and be it further

RESOLVED, that the persons listed immediately below be and hereby are
appointed to serve as the initial members of the Apache Griffin Project:

* Alex Lv 
* Deyi Yao 
* Eugene Liu 
* Grant Guo 
* He Wang 
* Henry Saputra 
* Jason Liao 
* John Liu 
* Juan Li 
* Liang Shao 
* Lionel Liu 
* Luciano Resende 
* Nick Sokolov 
* Shawn Sha 
* Vincent Zhao 
* William Guo 
* Yuqin Xuan 


NOW, THEREFORE, BE IT FURTHER RESOLVED, that William Guo be appointed
to the office of Vice President, Apache Griffin, to serve in accordance
with and subject to the direction of the Board of Directors and the
Bylaws of the Foundation until death, resignation, retirement, removal
or disqualification, or until a successor is appointed; and be it
further

RESOLVED, that the initial Apache Griffin PMC be and hereby is tasked
with the creation of a set of bylaws intended to encourage open
development and increased participation in the Apache Griffin Project;
and be it further

RESOLVED, that the Apache Griffin Project be and hereby is tasked with
the migration and rationalization of the Apache Incubator Griffin
podling; and be it further

RESOLVED, that all responsibilities pertaining to the Apache Incubator
Griffin podling encumbered upon the Apache Incubator PMC are hereafter
discharged.


Thanks,
William Guo

Re: How to assign Griffin's Jira to myself?

[William Guo]

I have assigned GRIFFIN-205 to you.


William

On Wed, Oct 10, 2018 at 8:34 PM Artem Shutak  wrote:

> My Jira name is 'artem.shutak'.
>
> Thanks,
> Artem.
>
> ср, 10 окт. 2018 г. в 2:39, Henry Saputra :
>
> > Right now yes. Please indicate you will be working on it. What is your
> JIRA
> > username?
> >
> > I will file ticket to Apache INFRA to change this.
> >
> > Thanks,
> >
> > Henry
> >
> > On Tue, Oct 9, 2018 at 11:00 AM Artem Shutak 
> > wrote:
> >
> > > Hello!
> > >
> > > I've created a Jira (https://issues.apache.org/jira/browse/GRIFFIN-205
> )
> > > and
> > > I'm not able to assign it to myself.
> > > Is it okey? Should I ask someone to grant me additional Jira rights? Or
> > > only committers/PPC members have such rights?
> > >
> > > Thanks,
> > > Artem
> > >
> >
>

[jira] [Assigned] (GRIFFIN-205) Accuracy measure check should provide matchedFraction to store

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo reassigned GRIFFIN-205:
---

Assignee: Artem Shutak

> Accuracy measure check should provide matchedFraction to store
> --
>
> Key: GRIFFIN-205
> URL: https://issues.apache.org/jira/browse/GRIFFIN-205
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>  Components: accuracy-batch, accuracy-real-time
>Affects Versions: 1.0.0-incubating
>Reporter: Artem Shutak
>Assignee: Artem Shutak
>Priority: Major
>
> Currently, {{accuracy}} measure results contains "total", "miss" and 
> "matched" counts.
> As a result, It's hard to analyze accuracy fraction based on results stored 
> in ElasticSearch, because ElasticSearch does not provide straight forward 
> capability to get "field divided by field" query results.
> {{Accuracy}} measure results should also contain {{matchedFraction}} field. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Jira permissions

[William Guo]

hi Nick Sokolov,

It seems griffin ticket can be assigned to a contributor now.
Could you check it again?

Thanks,
William


On Wed, Oct 10, 2018 at 9:05 AM William Guo  wrote:

> Let me look into this.
>
> On Wed, Oct 10, 2018 at 1:55 AM Nick Sokolov  wrote:
>
>> Hi all,
>>
>> I've discovered, that it's not possible to reassign jira ticket to other
>> contributors (non-committers), and non-committers can not assign tickets
>> (even if they have created it and want to assign it to themselves). Is
>> that
>> expected behavior?
>>
>> I remember that in Hive jira non-committers have these permissions, and
>> ticket can be assigned to anyone within Apache jira. Should project
>> settings be adjusted?
>>
>

[DISCUSS] Graduate Apache Griffin (incubating) as a TLP

[William Guo]

Hi All,

With the 0.3.0-incubating release officially out, the Apache Griffin
community and its mentors believe it is time to consider graduation to the
TLP.

Apache Griffin entered incubation in December of 2016, since then, the
Griffin
community learned a lot about how to do things in Apache ways. Now we have
a healthy and engaged community, ready to help with all questions from the
Griffin community. We delivered five releases, now we can do self-driving
releases in good cadence. The PPMC has demonstrated a good understanding of
growing the community by electing 9 individuals as committers and PPMC
members.
The PPMC addressed the maturity issues one by one followed by
Apache Project Maturity Model, currently all the License and IP issues are
resolved.
This demonstrated our understanding of ASF's IP policies.

All in all, I believe this project is qualified as a true TLP and we should
recognize this fact by formally awarding it such a status. This thread
means to open up the very same discussion that we had among the mentors and
Griffin community to the rest of the IPMC. It is a DISCUSS thread so feel
free
to ask questions.

To get you all going, here are a few data points which may help:

Project status:
http://incubator.apache.org/projects/griffin.html

Project website:
http://griffin.incubator.apache.org/

Project documentation:
http://griffin.incubator.apache.org/docs/quickstart.html
http://griffin.incubator.apache.org/docs/download.html

Maturity assessment:
https://cwiki.apache.org/confluence/display/GRIFFIN/ASF+Maturity+Evaluation

DRAFT of the board resolution is at the bottom of this email

Proposed PMC size: 18 members

Total number of committers: 18 members

PMC affiliation (* indicated chair):

 - eBay * (7)
 - Meituan (1)
 - VMWare (1)
 - Ontario Institute for Cancer Research (1)
 - NetEase (1)
 - ASF (1)
 - Pingan Bank (1)
 - Satori Software (1)
 - Grid Dynamics (1)
 - JD.com (1)
 - IBM (1)
 - Huawei (1)


452 commits on develop
431 PR”s on GitHub
30 contributors across all branches

200 issues created
195 issues resolved

dev list averaged ~20 msgs/month over last 12 months


committer affiliations:

active

* eBay
* meituan
* VMWare
* Grid Dynamics

occasional

 * Ontario Institute for Cancer Research
 * NetEase
 * ASF
 * Pingan Bank
 * Satori Software
 * JD.com
 * IBM
 * Huawei


Thanks,
William


**Notes:**

* I'm proposing myself as initial PMC chair -- Please comment this or
propose other persons as well

* This draft includes all existing PPMC members and mentors into the new
PMC.

- For all: please indicate if you want to keep being part of the
PMC or if you prefer to be removed.


--

## Resolution to create a TLP from graduating Incubator podling

X. Establish the Apache Griffin Project

WHEREAS, the Board of Directors deems it to be in the best
interests of the Foundation and consistent with the
Foundation's purpose to establish a Project Management
Committee charged with the creation and maintenance of
open-source software, for distribution at no charge to
the public, related to a data quality solution for big data,
including both streaming and batch mode. It offers an unified
process to measure data quality from different perspectives.

NOW, THEREFORE, BE IT RESOLVED, that a Project Management
Committee (PMC), to be known as the "Apache Griffin Project",
be and hereby is established pursuant to Bylaws of the
Foundation; and be it further

RESOLVED, that the Apache Griffin Project be and hereby is
responsible for the creation and maintenance of software
related to a data quality solution for big data,
including both streaming and batch mode. It offers an unified
process to measure data quality from different perspectives;
and be it further

RESOLVED, that the office of "Vice President, Apache Griffin" be
and hereby is created, the person holding such office to
serve at the direction of the Board of Directors as the chair
of the Apache Griffin Project, and to have primary responsibility
for management of the projects within the scope of
responsibility of the Apache Griffin Project; and be it further

RESOLVED, that the persons listed immediately below be and
hereby are appointed to serve as the initial members of the
Apache Griffin Project:

* Alex Lv 
* Deyi Yao 
* Eugene Liu 
* Grant Guo 
* He Wang 
* Henry Saputra 
* Jason Liao 
* John Liu 
* Juan Li 
* Liang Shao 
* Lionel Liu 
* Luciano Resende 
* Nick Sokolov 
* Shawn Sha 
* Uma Maheswara Rao Gangumalla 
* Vincent Zhao 
* William Guo 
* Yuqin Xuan 

NOW, THEREFORE, BE IT FURTHER RESOLVED, that William Guo
be appointed to the office of Vice President, Apache Griffin, to
serve in accordance with and subject to the direction of the
Board of Directors and the Bylaws of the Foundation until
death, resignation, retirement, removal or disqualification,
or until a successor is appointed; and be it further

RESOLVED, that the initial Apache Griffin PMC be and hereby is
tasked with the creation of a se

[jira] [Comment Edited] (GRIFFIN-202) Allow viewing of raw measure details from UI

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642931#comment-16642931
 ] 

William Guo edited comment on GRIFFIN-202 at 10/9/18 8:10 AM:
--

we can leverage some open source widgets, but we need to pay attention to their 
license.

if you have some candidates, please discuss in dev mailing list, or we need to 
ensure its compliant with apache license 2.0.

please check this document for license segment.

https://cwiki.apache.org/confluence/display/GRIFFIN/5.+How+to+Verify+Release+Package

Thanks,
William 


was (Author: guoyp):
we can leverage some open source widgets, but we need to pay attention to their 
license.

if you have some candidates, please discuss in dev mailing list, or we need to 
ensure its compliance with apache license 2.0.

please check this document for license segment.

https://cwiki.apache.org/confluence/display/GRIFFIN/5.+How+to+Verify+Release+Package

Thanks,
William 

> Allow viewing of raw measure details from UI
> 
>
> Key: GRIFFIN-202
> URL: https://issues.apache.org/jira/browse/GRIFFIN-202
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Nikolay Sokolov
>Priority: Minor
>
> Sometimes it's desirable to see raw JSON of specific measure. This is 
> especially useful when custom measurements are created from API.
> Two implementations are possible (one or even both can be implemented):
> * show link with URL of measure details API on UI
> * add section under "Mapping Rules", showing formatted plain text of API 
> response on UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-202) Allow viewing of raw measure details from UI

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642931#comment-16642931
 ] 

William Guo commented on GRIFFIN-202:
-

we can leverage some open source widgets, but we need to pay attention to their 
license.

if you have some candidates, please discuss in dev mailing list, or we need to 
ensure its compliance with apache license 2.0.

please check this document for license segment.

https://cwiki.apache.org/confluence/display/GRIFFIN/5.+How+to+Verify+Release+Package

Thanks,
William 

> Allow viewing of raw measure details from UI
> 
>
> Key: GRIFFIN-202
> URL: https://issues.apache.org/jira/browse/GRIFFIN-202
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Nikolay Sokolov
>Priority: Minor
>
> Sometimes it's desirable to see raw JSON of specific measure. This is 
> especially useful when custom measurements are created from API.
> Two implementations are possible (one or even both can be implemented):
> * show link with URL of measure details API on UI
> * add section under "Mapping Rules", showing formatted plain text of API 
> response on UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-194) [service] Hive API improvement

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642921#comment-16642921
 ] 

William Guo commented on GRIFFIN-194:
-

Cool, we might can increase performance by limit network roundtrip.

> [service] Hive API improvement
> --
>
> Key: GRIFFIN-194
> URL: https://issues.apache.org/jira/browse/GRIFFIN-194
> Project: Griffin (Incubating)
>  Issue Type: Sub-task
>Reporter: Nikolay Sokolov
>Priority: Minor
>
> Purpose is mainly to support GRIFFIN-195 with single request to get table 
> list information, while avoiding transferring all table metadata and making 
> lots of metastore requests.
> Hive API provides following relevant APIs right now:
> * listing DBs
> * getting all table names in DB
> * listing all table _objects_ in all _dbs_
> What's seems to be missing, is API call for all table names in all DBs (as 
> middle ground between n+1 API requests and 1 API request with huge payload 
> and n*m+1 metastore requests on backend).
> This api request should take no parameters, and return Map List> in response.
> Proposed API endpoint: TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633577#comment-16633577
 ] 

William Guo commented on GRIFFIN-200:
-

Should hooks by synchronous or asynchronous?
-- It depends, if we want to 'control' the job rate limit, it should be sync 
call, if it is just for notification, async will be better.
Whether exceptions thrown are able to interrupt action execution?
--As you said, we need interrupt it. but if we need to verify jobs, we should 
refine our job submit pipeline such as add one verify phase.
Whether single or multiple hooks should be supported?
--It should be flexible, we can support multiple hooks. but it seems more like 
multiple message channels if those messages are not used to control the job, 
but for message only.

Does livy support all these features?



> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface JobLifecycleHook {
> void beforeAddJob(AbstractJob js) throws Exception;
> void afterAddJob(AbstractJob js) throws Exception;
> void beforeOnAction(AbstractJob job, String action) throws Exception;
> void afterOnAction(AbstractJob job, String action) throws Exception;
> void beforeDeleteJob(AbstractJob job) throws Exception;
> void afterDeleteJob(AbstractJob job) throws Exception;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629642#comment-16629642
 ] 

William Guo commented on GRIFFIN-200:
-

Agree with griffin users do need to integrate with griffin jobs.
but what kind of integration model should we use, we should think twice.

One proposal is spring integration model.

One disadvantage of run time class loader reflection is that 
there will be some compile time dependencies between griffin and customers 
logic, 
which might inject some griffin agreements into customer's code, not acceptable 
for them.



> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface JobLifecycleHook {
> void beforeAddJob(AbstractJob js) throws Exception;
> void afterAddJob(AbstractJob js) throws Exception;
> void beforeOnAction(AbstractJob job, String action) throws Exception;
> void afterOnAction(AbstractJob job, String action) throws Exception;
> void beforeDeleteJob(AbstractJob job) throws Exception;
> void afterDeleteJob(AbstractJob job) throws Exception;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (GRIFFIN-200) Lifecycle hooks support

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo reassigned GRIFFIN-200:
---

Assignee: William Guo

> Lifecycle hooks support
> ---
>
> Key: GRIFFIN-200
> URL: https://issues.apache.org/jira/browse/GRIFFIN-200
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Minor
>
> In some environments, users might want to perform certain actions 
> before/after job is created, before/after job is activated, before/after job 
> is deleted, and so on.
> To fullfill that need, some hook plugin mechanism can be provided, similar to 
> what Hive is doing. User would place respective jar files into Service module 
> classpath at deployment time, and would specify class names using some 
> annotation or using property listing class names (particular mechanism is yet 
> to be determined).
> Proposed signature:
> {code:none}
> public interface JobLifecycleHook {
> void beforeAddJob(AbstractJob js) throws Exception;
> void afterAddJob(AbstractJob js) throws Exception;
> void beforeOnAction(AbstractJob job, String action) throws Exception;
> void afterOnAction(AbstractJob job, String action) throws Exception;
> void beforeDeleteJob(AbstractJob job) throws Exception;
> void afterDeleteJob(AbstractJob job) throws Exception;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-198) HiveMetaStoreServiceImpl caches incorrect state of metastore

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624882#comment-16624882
 ] 

William Guo commented on GRIFFIN-198:
-

[~Kevin Yao]

What is your option on this?

Thanks,
William

> HiveMetaStoreServiceImpl caches incorrect state of metastore
> 
>
> Key: GRIFFIN-198
> URL: https://issues.apache.org/jira/browse/GRIFFIN-198
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Major
>
> Currently, HiveMetaStoreServiceImpl is caching:
> * nulls and empty lists in some methods, when metastore request failed for 
> some reason
> * empty list of tables as response to getAllTable(String) and getAllTable() 
> calls
> Both problems result in confusing output on large amount of tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-198) HiveMetaStoreServiceImpl caches incorrect state of metastore

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624880#comment-16624880
 ] 

William Guo commented on GRIFFIN-198:
-

hi [~chemikadze]

Sorry, I will reopen this ticket.

For this issue,
https://github.com/apache/incubator-griffin/blob/master/service/src/main/java/org/apache/griffin/core/metastore/hive/HiveMetaStoreServiceImpl.java#L119-L123

the bottom line is  we cannot return or cache incomplete db->tables map to 
users.

I think we need to make it transparent to our users, so let us retry it first, 
but we need to ensure it works well with cache.

Thanks,
William


> HiveMetaStoreServiceImpl caches incorrect state of metastore
> 
>
> Key: GRIFFIN-198
> URL: https://issues.apache.org/jira/browse/GRIFFIN-198
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Nikolay Sokolov
>    Assignee: William Guo
>Priority: Major
>
> Currently, HiveMetaStoreServiceImpl is caching:
> * nulls and empty lists in some methods, when metastore request failed for 
> some reason
> * empty list of tables as response to getAllTable(String) and getAllTable() 
> calls
> Both problems result in confusing output on large amount of tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (GRIFFIN-198) HiveMetaStoreServiceImpl caches incorrect state of metastore

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo reopened GRIFFIN-198:
-
  Assignee: William Guo

> HiveMetaStoreServiceImpl caches incorrect state of metastore
> 
>
> Key: GRIFFIN-198
> URL: https://issues.apache.org/jira/browse/GRIFFIN-198
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Nikolay Sokolov
>        Assignee: William Guo
>Priority: Major
>
> Currently, HiveMetaStoreServiceImpl is caching:
> * nulls and empty lists in some methods, when metastore request failed for 
> some reason
> * empty list of tables as response to getAllTable(String) and getAllTable() 
> calls
> Both problems result in confusing output on large amount of tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (GRIFFIN-198) HiveMetaStoreServiceImpl caches incorrect state of metastore

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo resolved GRIFFIN-198.
-
Resolution: Fixed

> HiveMetaStoreServiceImpl caches incorrect state of metastore
> 
>
> Key: GRIFFIN-198
> URL: https://issues.apache.org/jira/browse/GRIFFIN-198
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Nikolay Sokolov
>Priority: Major
>
> Currently, HiveMetaStoreServiceImpl is caching:
> * nulls and empty lists in some methods, when metastore request failed for 
> some reason
> * empty list of tables as response to getAllTable(String) and getAllTable() 
> calls
> Both problems result in confusing output on large amount of tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (GRIFFIN-194) [service] Hive API improvement

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620172#comment-16620172
 ] 

William Guo edited comment on GRIFFIN-194 at 9/19/18 6:31 AM:
--

yeah, 

List dbs = getAllDatabases();
foreach db in dbs:
getAllTableNames(db)
we can get all tables db by db, that can fix the payload issue.

BTW,   I like+ your idea for narrow down navigation table path in UI Part. 

Thanks,
William



was (Author: guoyp):
yeah, 

List dbs = getAllDatabases();
foreach db in dbs:
getAllTableNames(db)
we can get all tables db by db, that can fix the payload issue.

BTW,   I like your idea for narrow down navigation table path in UI Part.

Thanks,
William


> [service] Hive API improvement
> --
>
> Key: GRIFFIN-194
> URL: https://issues.apache.org/jira/browse/GRIFFIN-194
> Project: Griffin (Incubating)
>  Issue Type: Sub-task
>Reporter: Nikolay Sokolov
>Priority: Minor
>
> Purpose is mainly to support GRIFFIN-195 with single request to get table 
> list information, while avoiding transferring all table metadata and making 
> lots of metastore requests.
> Hive API provides following relevant APIs right now:
> * listing DBs
> * getting all table names in DB
> * listing all table _objects_ in all _dbs_
> What's seems to be missing, is API call for all table names in all DBs (as 
> middle ground between n+1 API requests and 1 API request with huge payload 
> and n*m+1 metastore requests on backend).
> This api request should take no parameters, and return Map List> in response.
> Proposed API endpoint: TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-194) [service] Hive API improvement

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620172#comment-16620172
 ] 

William Guo commented on GRIFFIN-194:
-

yeah, 

List dbs = getAllDatabases();
foreach db in dbs:
getAllTableNames(db)
we can get all tables db by db, that can fix the payload issue.

BTW,   I like your idea for narrow down navigation table path in UI Part.

Thanks,
William


> [service] Hive API improvement
> --
>
> Key: GRIFFIN-194
> URL: https://issues.apache.org/jira/browse/GRIFFIN-194
> Project: Griffin (Incubating)
>  Issue Type: Sub-task
>Reporter: Nikolay Sokolov
>Priority: Minor
>
> Purpose is mainly to support GRIFFIN-195 with single request to get table 
> list information, while avoiding transferring all table metadata and making 
> lots of metastore requests.
> Hive API provides following relevant APIs right now:
> * listing DBs
> * getting all table names in DB
> * listing all table _objects_ in all _dbs_
> What's seems to be missing, is API call for all table names in all DBs (as 
> middle ground between n+1 API requests and 1 API request with huge payload 
> and n*m+1 metastore requests on backend).
> This api request should take no parameters, and return Map List> in response.
> Proposed API endpoint: TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (GRIFFIN-194) [service] Hive API improvement

[William Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/GRIFFIN-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620153#comment-16620153
 ] 

William Guo commented on GRIFFIN-194:
-

Hi Sokolov,


Based on my understanding, griffin service has already supported your 
requirement for all tables.

https://github.com/apache/incubator-griffin/blob/master/service/src/main/java/org/apache/griffin/core/metastore/hive/HiveMetaStoreServiceImpl.java#L105-L122

Could you check this?


Thanks,
William

> [service] Hive API improvement
> --
>
> Key: GRIFFIN-194
> URL: https://issues.apache.org/jira/browse/GRIFFIN-194
> Project: Griffin (Incubating)
>  Issue Type: Sub-task
>Reporter: Nikolay Sokolov
>Priority: Minor
>
> Hive API provides following relevant APIs right now:
> * listing DBs
> * getting all table names in DB
> * listing all table _objects_ in all _dbs_
> What's seems to be missing, is API call for all table names in all DBs (as 
> middle ground between n+1 API requests and 1 API request with huge payload 
> and n*m+1 metastore requests on backend).
> This api request should take no parameters, and return Map List> in response.
> Proposed API endpoint: TBD



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-185) [UI] download miss records

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-185.
---

> [UI] download miss records
> --
>
> Key: GRIFFIN-185
> URL: https://issues.apache.org/jira/browse/GRIFFIN-185
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-183) [UI] add home page link under user setting

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-183.
---

> [UI] add home page link under user setting
> --
>
> Key: GRIFFIN-183
> URL: https://issues.apache.org/jira/browse/GRIFFIN-183
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-184) [Service] download miss records

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-184.
---

> [Service] download miss records
> ---
>
> Key: GRIFFIN-184
> URL: https://issues.apache.org/jira/browse/GRIFFIN-184
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-178) [UI] job state not update after performing start/stop actions on job page

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-178.
---

> [UI] job state not update after performing start/stop actions on job page
> -
>
> Key: GRIFFIN-178
> URL: https://issues.apache.org/jira/browse/GRIFFIN-178
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
> Attachments: 178.png
>
>
> job state is not update properly after performing start/stop job action, 
> still previouse state
> attached issue snapshot
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-177) [UI] measure page wrong display

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-177.
---

> [UI] measure page wrong display 
> 
>
> Key: GRIFFIN-177
> URL: https://issues.apache.org/jira/browse/GRIFFIN-177
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
> Attachments: 177.png
>
>
> found measure page wrongly displayed attached the symptom 
> root cause: \{{row["dq.type"].toLowerCase()}} in 
> measure.component.html causing infinite loading if  row["dq.type"] is null
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-174) Fix deploy document

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-174.
---

> Fix deploy document
> ---
>
> Key: GRIFFIN-174
> URL: https://issues.apache.org/jira/browse/GRIFFIN-174
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Affects Versions: 0.2.0-incubating
>Reporter: Kwang-in (Dennis) JUNG
>Assignee: Lionel Liu
>Priority: Trivial
> Fix For: 1.0.0-incubating
>
>
> Hello.
> While following up deploy guide, I found some wrong links/file name, so 
> updated PR for fix.
> [https://github.com/apache/incubator-griffin/pull/340]
>  
>  * Wrong link of SQL table
>  * Wrong environment name
> Maybe my fix can be wrong, so could anyone check this?
> Thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-170) [Measure] Refactor read steps to take the read responsibility instead of data connector and streaming data cache client

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-170.
---
Resolution: Fixed

> [Measure] Refactor read steps to take the read responsibility instead of data 
> connector and streaming data cache client
> ---
>
> Key: GRIFFIN-170
> URL: https://issues.apache.org/jira/browse/GRIFFIN-170
> Project: Griffin (Incubating)
>  Issue Type: Task
>Affects Versions: 1.0.0-incubating
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 1.0.0-incubating
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> At current, griffin reads data actually via data connector (in batch mode) 
> and streaming data cache client (in streaming mode), it should be refactored 
> to be able to read data via read steps, to make the conception much easier 
> for users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-166) [UI] fix Create measure page layout

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-166.
---

> [UI] fix Create measure page layout 
> 
>
> Key: GRIFFIN-166
> URL: https://issues.apache.org/jira/browse/GRIFFIN-166
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
> Attachments: Screen Shot 2018-05-17 at 4.28.03 PM.png
>
>
> current layout is attached 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-164) Make 'Regular expression detection count' available in UI

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-164.
---

> Make 'Regular expression detection count' available in UI
> -
>
> Key: GRIFFIN-164
> URL: https://issues.apache.org/jira/browse/GRIFFIN-164
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Affects Versions: 0.1.6-incubating
>Reporter: Enrico D'Urso
>Priority: Minor
> Fix For: 1.0.0-incubating
>
>
> Hi,
> I have been playing for one month now with Griffin.
> Given my experience, some companies (included the one am working for as a 
> consultant) prefer doing stuff using UI.
> Personally, I find very useful the following feature:
>  
>  * Regular expression detection count
> which is, I have a column which should contain just numbers so I want to 
> check if my ETL process, wrongly, has populated my table with non-numeric 
> values.
> I have been able to run such a job creating my self the right config.json, in 
> particular, using spark-sql as dialect:
> {code:java}
> select count(*) from src where account_id rlike [^0-9]  
> {code}
> I saw that in pr.component.ts there is a commented line of code:
> {code:java}
> // {"id":10,"itemName":"Regular Expression Detection Count","category": 
> "Advanced Statistics"}
> {code}
> which I think is what I am talking about.
> Also, I can read:
> {code:java}
> // case 'Regular Expression Detection Count': // return 
> 'count(source.`'+col.name+'`) where source.`'+col.name+'` LIKE ';
> {code}
> which should be the griffin-dsl dialect, even if, probably, the regex should 
> be added just after LIKE.
> Then, once that the above griffin-dsl statement is available in the backend, 
> ProfilingRulePlanTrans class
> should map that into 'rlike' Spark-sql clause.
> Am not sure where (and if) ProfilingRulePlanTrans should be modified as 
> preGroupbyClause should contains everything, but I do not have enough 
> knowledge about it.
>  
> Please judge yourself the priority of such a feature, which knowing well the 
> code, should not be too hard to make.
> Thanks,
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-163) [Measure] Merge 0.2.0 for spark 2 into master branch

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-163.
---

> [Measure] Merge 0.2.0 for spark 2 into master branch
> 
>
> Key: GRIFFIN-163
> URL: https://issues.apache.org/jira/browse/GRIFFIN-163
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 1.0.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-159) [Measure] Refactor generation of measure metrics

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-159.
---

> [Measure] Refactor generation of measure metrics
> 
>
> Key: GRIFFIN-159
> URL: https://issues.apache.org/jira/browse/GRIFFIN-159
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 1.0.0-incubating
>
>
> measure metrics is generated as json at current, we're planning to enhance 
> the metrics generation.
>  # abstract metrics data generation interface.
>  # implementation of current metric data generation.
>  # implementation of advanced metric data generation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-158) Functional test of generic scheduler

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-158.
---

> Functional test of generic scheduler
> 
>
> Key: GRIFFIN-158
> URL: https://issues.apache.org/jira/browse/GRIFFIN-158
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Kevin Yao
>Assignee: Kevin Yao
>Priority: Major
> Fix For: 1.0.0-incubating
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Functional test of generic schedule for both streaming and batch jobs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-153) [UI] Can not render accuracy chart on UI

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-153.
---

> [UI] Can not render accuracy chart on UI
> 
>
> Key: GRIFFIN-153
> URL: https://issues.apache.org/jira/browse/GRIFFIN-153
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Lionel Liu
>Assignee: Yuqin Xuan
>Priority: Major
> Fix For: 1.0.0-incubating
>
>
> On UI, no matter the dashboard page or side bar, it can not render the 
> accuracy chart in this situation:
> I created an accuracy measure, and created two jobs for this measure, one is 
> daily, and another is hourly.
> After some hours, the hourly job has triggered several times, but the daily 
> job hasn't started yet.
> I created the daily job earlier than the hourly job, in this situation, in 
> the dashboard page or side bar, I can not see any chart of my accuracy 
> measure. But it should be able to render the hourly job chart, because the 
> metric data has been calculated.
>  
> The problem seems like: 
> When ui get the metrics data like this:
>  
> {code:java}
> {  
>   "accu":[  
> {  
>   "name":"accu_daily",
>   "type":"accuracy",
>   "metricValues":[]
> },
> {  
>   "name":"accu_hourly",
>   "type":"accuracy",
>   "metricValues":[  
> {  
>   "name":"accu_hourly",
>   "tmst":152470080,
>   "value":{  
> "total":125000,
> "miss":485,
> "matched":124515
>   }
> }
>   ]
> }
>   ]
> }{code}
> It should be able to render a chart named "accu_hourly" under the measure 
> "accu", but it could not render anything.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-155) [UI] change how the charts are dispalyed on my dashboard view

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-155.
---

> [UI] change how the charts are dispalyed on my dashboard view
> -
>
> Key: GRIFFIN-155
> URL: https://issues.apache.org/jira/browse/GRIFFIN-155
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Yuqin Xuan
>Assignee: Yuqin Xuan
>Priority: Major
>
> The charts are grouped by job name instead of measure name on my dashboard 
> view



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-154) [UI] fix health display bug

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-154.
---

> [UI] fix health display bug
> ---
>
> Key: GRIFFIN-154
> URL: https://issues.apache.org/jira/browse/GRIFFIN-154
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Yuqin Xuan
>Assignee: Yuqin Xuan
>Priority: Major
> Fix For: 0.2.0-incubating
>
>
> Health module is displayed with job name rather than measure name.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-156) [Doc] update the document of metric fetch on wiki

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-156.
---

> [Doc] update the document of metric fetch on wiki
> -
>
> Key: GRIFFIN-156
> URL: https://issues.apache.org/jira/browse/GRIFFIN-156
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Yuqin Xuan
>Priority: Major
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> https://cwiki.apache.org/confluence/display/GRIFFIN/4.+Metric+schema



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-151) [Measure] Upgrade measure module calculation engine to spark 2

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-151.
---

> [Measure] Upgrade measure module calculation engine to spark 2
> --
>
> Key: GRIFFIN-151
> URL: https://issues.apache.org/jira/browse/GRIFFIN-151
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Affects Versions: 0.2.0-incubating
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 0.2.0-incubating
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> We need to upgrade the dependency from spark 1 to spark 2, to support spark 2 
> environment, and ignore the license issue of JSON library.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-146) [Service] Prepare and test job state and action service

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-146.
---

> [Service] Prepare and test job state and action service
> ---
>
> Key: GRIFFIN-146
> URL: https://issues.apache.org/jira/browse/GRIFFIN-146
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Yuqin Xuan
>Priority: Major
>
> cherry pick and test , push to master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-142) [UI] submit button should be grayed out before form is finished

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-142.
---
Resolution: Fixed

merged.

> [UI] submit button should be grayed out before form is finished
> ---
>
> Key: GRIFFIN-142
> URL: https://issues.apache.org/jira/browse/GRIFFIN-142
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-137) [Griffin] As griffin measure, we need to refresh the code, to be align with data quality concept

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-137.
---
Resolution: Fixed

> [Griffin] As griffin measure, we need to refresh the code, to be align with 
> data quality concept
> 
>
> Key: GRIFFIN-137
> URL: https://issues.apache.org/jira/browse/GRIFFIN-137
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> At current, accuracy in griffin measures source with target, but in data 
> quality concept, accuracy measures target with source, they are inverse. We 
> need to change the concept in griffin, to be align with the data quality 
> concept, to reduce the misunderstanding and confuse.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-138) [Doc] Modify readme.md, highlight docker guide for common users

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-138.
---

> [Doc] Modify readme.md, highlight docker guide for common users
> ---
>
> Key: GRIFFIN-138
> URL: https://issues.apache.org/jira/browse/GRIFFIN-138
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, our dev env has a steep learning curve, we need to shallow our 
> learning curve by packaging all dependencies as a  docker image for most 
> developers, and for advanced developers, they can setup local dev env by 
> following tedious step-by-step installing scripts.
> Please reflect this In Readme.md.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-136) [Griffin] As griffin user, I want to use mysql or hibernate with griffin open source version, to deploy it in my environment

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-136.
---

> [Griffin] As griffin user, I want to use mysql or hibernate with griffin open 
> source version, to deploy it in my environment
> 
>
> Key: GRIFFIN-136
> URL: https://issues.apache.org/jira/browse/GRIFFIN-136
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Kevin Yao
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Due to some license issue, griffin open source version choose postgresql and 
> eclipselink jpa as default solution, but for some users, they want to use 
> mysql or hibernate, we need to provide a document to support these use cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-135) [Measure] Support Completeness measurement as a new feature

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-135.
---

> [Measure] Support Completeness measurement as a new feature
> ---
>
> Key: GRIFFIN-135
> URL: https://issues.apache.org/jira/browse/GRIFFIN-135
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 1.0.0-incubating
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Completeness measures the absence of blank (null or empty string) values or 
> the presence of non-blank values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-131) [Doc][Site] Use case documents

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-131.
---
Resolution: Fixed

> [Doc][Site] Use case documents
> --
>
> Key: GRIFFIN-131
> URL: https://issues.apache.org/jira/browse/GRIFFIN-131
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Write some documents of use cases, onboard on site



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-130) [Site] Modify griffin site template

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-130.
---

> [Site] Modify griffin site template
> ---
>
> Key: GRIFFIN-130
> URL: https://issues.apache.org/jira/browse/GRIFFIN-130
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Juan Li
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Change the template of griffin site



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-132) [Community] Update the status file on site

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-132.
---

> [Community] Update the status file on site
> --
>
> Key: GRIFFIN-132
> URL: https://issues.apache.org/jira/browse/GRIFFIN-132
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>        Assignee: William Guo
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Update the project status on site: 
> http://incubator.apache.org/projects/griffin.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-133) [UI] Add missing records download link on UI

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-133.
---

> [UI] Add missing records download link on UI
> 
>
> Key: GRIFFIN-133
> URL: https://issues.apache.org/jira/browse/GRIFFIN-133
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>        Assignee: William Guo
>Priority: Major
> Attachments: Screen Shot 2018-07-24 at 3.31.12 PM.png
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When user watch accuracy chart, provide missing records download link



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-126) [Site] Add a security link in griffin site

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-126.
---

Done, check community section.

> [Site] Add a security link in griffin site
> --
>
> Key: GRIFFIN-126
> URL: https://issues.apache.org/jira/browse/GRIFFIN-126
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Juan Li
>Priority: Major
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Add a security link ([http://www.apache.org/security/]) in griffin site.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-127) [Doc][Site] Announce "document contribution" description in "How to contribute" on site

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-127.
---

> [Doc][Site] Announce "document contribution" description in "How to 
> contribute" on site
> ---
>
> Key: GRIFFIN-127
> URL: https://issues.apache.org/jira/browse/GRIFFIN-127
> Project: Griffin (Incubating)
>  Issue Type: Task
>    Reporter: Lionel Liu
>Assignee: William Guo
>Priority: Major
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> on site document, announce "document contribution" description in "How to 
> contribute"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-124) [Service] Support automatic db table creation for quartz

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-124.
---
Resolution: Fixed

PR Merged

> [Service] Support automatic db table creation for quartz
> 
>
> Key: GRIFFIN-124
> URL: https://issues.apache.org/jira/browse/GRIFFIN-124
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Minyi Ding
>Assignee: Minyi Ding
>Priority: Major
>
> If we can support automatic db table creation, there will be no need to 
> manually create tables for quartz.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-122) [Service] We need to support all livy parameters when submit job.

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-122.
---
Resolution: Fixed
  Assignee: William Guo  (was: Qiyuan Zhou)

> [Service] We need to support all livy parameters when submit job.
> -
>
> Key: GRIFFIN-122
> URL: https://issues.apache.org/jira/browse/GRIFFIN-122
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Lionel Liu
>        Assignee: William Guo
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Support reading livy config file as a map, to support any other extend 
> parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-121) [UI] Add publish measure create process on UI

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-121.
---

> [UI] Add publish measure create process on UI
> -
>
> Key: GRIFFIN-121
> URL: https://issues.apache.org/jira/browse/GRIFFIN-121
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Lionel Liu
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> We need to support publish measure create process on UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-116) [Measure][UT] Enhance UT of data and process part in measure module

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-116.
---
Resolution: Fixed

> [Measure][UT] Enhance UT of data and process part in measure module
> ---
>
> Key: GRIFFIN-116
> URL: https://issues.apache.org/jira/browse/GRIFFIN-116
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> In measure module, data and process part need to enhance UT.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-114) [Service][UT] Enhance unit test of config, info, util in service module

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-114.
---
Resolution: Fixed

> [Service][UT] Enhance unit test of config, info, util in service module
> ---
>
> Key: GRIFFIN-114
> URL: https://issues.apache.org/jira/browse/GRIFFIN-114
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: wanyin
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> In service module, UT of config, info and util need to be enhanced.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-115) [Measure][UT] Enhance UT of rule part in measure module

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-115.
---
Resolution: Fixed

> [Measure][UT] Enhance UT of rule part in measure module
> ---
>
> Key: GRIFFIN-115
> URL: https://issues.apache.org/jira/browse/GRIFFIN-115
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>        Assignee: William Guo
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> In measure module, rule part needs to enhance UT, to cover more conditions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-117) [Service] Provide api for users to download missing records in accuracy

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-117.
---

> [Service] Provide api for users to download missing records in accuracy
> ---
>
> Key: GRIFFIN-117
> URL: https://issues.apache.org/jira/browse/GRIFFIN-117
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Lionel Liu
>Assignee: Zhao, Vincent
>Priority: Major
> Fix For: 1.0.0-incubating
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> In accuracy, users want to download the missing records, we need to provide 
> such api for them.
>  # user can get the missing records list of each metric point.
>  # user can get the url of the specific missing records.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-113) [Service][UT] Enhance unit test of measure, job, metric in service module

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-113.
---
Resolution: Fixed
  Assignee: William Guo  (was: Qiyuan Zhou)

> [Service][UT] Enhance unit test of measure, job, metric in service module
> -
>
> Key: GRIFFIN-113
> URL: https://issues.apache.org/jira/browse/GRIFFIN-113
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>        Assignee: William Guo
>Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> In service module, UT of measure, job and metric need to be enhanced.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-112) [Service] Hive metastore service enhancement

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-112.
---
Resolution: Fixed

> [Service] Hive metastore service enhancement
> 
>
> Key: GRIFFIN-112
> URL: https://issues.apache.org/jira/browse/GRIFFIN-112
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Zhao, Vincent
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> In service module, hive metastore service works in some confusing 
> information, e.g.: when get tables by db name not exist, returns empty array, 
> but it should be better to return an error msg.
>  
> DoD:
>  # enhance the hive metastore service logic.
>  # ut
>  # e2e test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-111) [UI] add progress indicator in every http request

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-111.
---

> [UI] add progress indicator in every http request
> -
>
> Key: GRIFFIN-111
> URL: https://issues.apache.org/jira/browse/GRIFFIN-111
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
> Fix For: 1.0.0-incubating
>
>
> now only some pages have progress icon in table level



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-110) [Service] add createDataAsset for Kafka Type for streaming job service

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-110.
---
Resolution: Fixed

same issue, griffin don't create topic, it just fetch topic from kafka registry.

> [Service] add createDataAsset for Kafka Type for streaming job service
> --
>
> Key: GRIFFIN-110
> URL: https://issues.apache.org/jira/browse/GRIFFIN-110
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-109) [Service] add createDataAsset for HDFS Type service

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-109.
---
Resolution: Won't Fix

Griffin just fetch metadata from metadata store, griffin don't create data 
asset.

> [Service] add createDataAsset for HDFS Type service
> ---
>
> Key: GRIFFIN-109
> URL: https://issues.apache.org/jira/browse/GRIFFIN-109
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-108) Prepare a guidance for committer for license issue in apache project.

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-108.
---

griffin contributors need to follow this guide to verify our release.

> Prepare a guidance for committer for license issue in apache project.
> -
>
> Key: GRIFFIN-108
> URL: https://issues.apache.org/jira/browse/GRIFFIN-108
> Project: Griffin (Incubating)
>  Issue Type: Task
>        Reporter: William Guo
>        Assignee: William Guo
>Priority: Major
>
> USER STORY:
> many committers don't know much about Apache license and don't know how to 
> verify license in voting process.
> Accept criteria:
>  #  explain license principle in Apache project
>  #  the standard process for verifying license for our griffin project.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-107) [UI] - DataAsset register including HDFS and Kafka

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-107.
---
Resolution: Fixed

> [UI] - DataAsset register including HDFS and Kafka
> --
>
> Key: GRIFFIN-107
> URL: https://issues.apache.org/jira/browse/GRIFFIN-107
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Juan Li
>Assignee: Juan Li
>Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-106) [UI] Need to modify description of login page

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-106.
---

since it is done, will close this ticket.

> [UI] Need to modify description of login page
> -
>
> Key: GRIFFIN-106
> URL: https://issues.apache.org/jira/browse/GRIFFIN-106
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Lionel Liu
>Assignee: Yuqin Xuan
>Priority: Major
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Check the description of login page, hide the specific words.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-104) [UI] publish measure create process

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-104.
---
Resolution: Fixed

> [UI] publish measure create process
> ---
>
> Key: GRIFFIN-104
> URL: https://issues.apache.org/jira/browse/GRIFFIN-104
> Project: Griffin (Incubating)
>  Issue Type: New Feature
>Reporter: Lionel Liu
>Assignee: Yuqin Xuan
>Priority: Major
>  Labels: SP_5
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-102) [Service] Fix bug of fetching metrics of different jobs with the same name

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-102.
---
Resolution: Fixed

> [Service] Fix bug of fetching metrics of different jobs with the same name
> --
>
> Key: GRIFFIN-102
> URL: https://issues.apache.org/jira/browse/GRIFFIN-102
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Lionel Liu
>Assignee: He Wang
>Priority: Major
>  Labels: SP_5
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> When fetch metrics from ES, if one job has the same name with another job 
> which was deleted, the metrics would be union and confuse.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-103) [Griffin] Release of incubating-0.2.0

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-103.
---

> [Griffin] Release of incubating-0.2.0
> -
>
> Key: GRIFFIN-103
> URL: https://issues.apache.org/jira/browse/GRIFFIN-103
> Project: Griffin (Incubating)
>  Issue Type: Task
>Reporter: Lionel Liu
>Assignee: Lionel Liu
>Priority: Major
> Fix For: 0.2.0-incubating
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-99) Griffin - Docker - API server doesn't start as MySQL doesn't start on the docker container.

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-99?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-99.
--

> Griffin - Docker - API server doesn't start as MySQL doesn't start on the 
> docker container.
> ---
>
> Key: GRIFFIN-99
> URL: https://issues.apache.org/jira/browse/GRIFFIN-99
> Project: Griffin (Incubating)
>  Issue Type: Bug
>  Components: completeness-batch
>Affects Versions: 0.2.0-incubating
> Environment: Mac OSX
>Reporter: Akash Sharma S
>Assignee: Lionel Liu
>Priority: Critical
>  Labels: SP_4
> Fix For: 0.2.0-incubating
>
>
> * I downloaded the docker images following the steps in the below page:
>  ** 
> [https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/griffin-docker-guide.md]
>  * I'm bringing up the docker container using the docker compose file for 
> batch mode.
>  
> Observation:
>  * I'm not able to access any of the APIs via the URL: :38080
>  * Upon further investigation, I found out after connecting to the docker 
> container that MySQL isn't running inside the container, hence the API server 
> isn't coming up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-100) Improve griffin logo

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-100.
---

> Improve griffin logo
> 
>
> Key: GRIFFIN-100
> URL: https://issues.apache.org/jira/browse/GRIFFIN-100
> Project: Griffin (Incubating)
>  Issue Type: Improvement
>  Components: accuracy-batch
>Affects Versions: 0.2.0-incubating
>    Reporter: William Guo
>Assignee: William Guo
>Priority: Minor
> Attachments: griffin_logo_draft.png
>
>
> HI Guys,
> Anyone know who contributed the logo for Apache Griffin podling?
> Want to see if we could make some changes to incorporate the name Griffin
> itself in addition to Apache Incubator name.
>  
> BY henry



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (GRIFFIN-101) [Service] Fix bug of duplicate submission of measure and job create

[William Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GRIFFIN-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

William Guo closed GRIFFIN-101.
---

> [Service] Fix bug of duplicate submission of measure and job create
> ---
>
> Key: GRIFFIN-101
> URL: https://issues.apache.org/jira/browse/GRIFFIN-101
> Project: Griffin (Incubating)
>  Issue Type: Bug
>Reporter: Lionel Liu
>Assignee: Kevin Yao
>Priority: Major
>  Labels: SP_4
> Fix For: 0.2.0-incubating
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> When create measure or job, the front end might submit twice at one time, 
> service should prevent such situation and handle it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)