Re: [VOTE] Apache Helix 0.8.3 Release

[Harry Zhang]

+1

Get Outlook for iOS


From: Hunter Lee 
Sent: Monday, November 26, 2018 23:55
To: dev@helix.apache.org
Subject: Re: [VOTE] Apache Helix 0.8.3 Release

+1

On Mon, Nov 26, 2018 at 6:09 PM Lei Xia  wrote:

> +1
>
> On Mon, Nov 26, 2018 at 5:48 PM Xue Junkai  wrote:
>
> > Hi,
> >
> >
> > This is to call for a vote on releasing the following candidate as Apache
> > Helix 0.8.3. This is the 15th release of Helix as an Apache project, as
> > well as the 11th release as a top-level Apache project.
> >
> >
> > Apache Helix is a generic cluster management framework that makes it easy
> > to build partitioned and replicated, fault-tolerant and scalable
> > distributed systems.
> >
> >
> > Release notes:
> >
> > *https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelix.apache.org%2F0.8.3-docs%2Freleasenotes%2Frelease-0.8.3.htmldata=02%7C01%7Chrzhang%40linkedin.com%7C04671ce111f94036491908d6543daaae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636789021182256119sdata=gcfP0edzVAW8G7snAZFBXG8aVy0kPRIsOhud3t2Xd%2BA%3Dreserved=0
> > *
> >
> >
> > Release artifacts:
> >
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachehelix-1021data=02%7C01%7Chrzhang%40linkedin.com%7C04671ce111f94036491908d6543daaae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636789021182256119sdata=yt0PcObfdJNHKuU%2BlmxaPLYRWSZwT2etUpAFVHkdV9Y%3Dreserved=0
> >
> >
> > Distribution:
> >
> > * binaries:
> >
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fhelix%2F0.8.3%5D%2Fbinaries%2Fdata=02%7C01%7Chrzhang%40linkedin.com%7C04671ce111f94036491908d6543daaae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636789021182256119sdata=kIvLsIu%2FV0nvkderLCl1ksNOs7KXtSuLoPuM7XRYOjY%3Dreserved=0
> >
> > * sources:
> >
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fhelix%2F0.8.3%2Fsrc%2Fdata=02%7C01%7Chrzhang%40linkedin.com%7C04671ce111f94036491908d6543daaae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636789021182256119sdata=iTUB9K41aOEA5WPMYiu6x850Fq4xFmPw90JX96KFyQ0%3Dreserved=0
> >
> >
> > The 0.8.3 release tag:
> >
> >
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit-wip-us.apache.org%2Frepos%2Fasf%3Fp%3Dhelix.git%3Ba%3Dtag%3Bh%3Drefs%2Ftags%2Fhelix-0.8.3data=02%7C01%7Chrzhang%40linkedin.com%7C04671ce111f94036491908d6543daaae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636789021182266128sdata=nUucSsFf%2FzBanuRN%2BIGS4lJ3yMHpeq1ESSdHs06moEQ%3Dreserved=0
> >
> >
> > KEYS file available here:
> >
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fhelix%2FKEYSdata=02%7C01%7Chrzhang%40linkedin.com%7C04671ce111f94036491908d6543daaae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636789021182266128sdata=OfEaDbDAdIVaJ3TtQxbn1TzKL%2FYfnhVu9VDnwteXosM%3Dreserved=0
> >
> >
> > Please vote on the release. The vote will be open for at least 72 hours.
> >
> >
> > [+1] -- "YES, release"
> >
> > [0] -- "No opinion"
> >
> > [-1] -- "NO, do not release"
> >
> >
> > Thanks,
> >
> > The Apache Helix Team
> >
>

[jira] [Created] (HELIX-785) Report helix latency instead of user latency during top state handoff

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-785:
-

 Summary: Report helix latency instead of user latency during top 
state handoff
 Key: HELIX-785
 URL: https://issues.apache.org/jira/browse/HELIX-785
 Project: Apache Helix
  Issue Type: Task
Reporter: Harry Zhang
Assignee: Harry Zhang


Currently we are reporting top state handoff user latency, but we should report 
Helix latency instead. user should have their way of monitoring their own state 
transitions.

AC:
1. Implement reporting Helix latency for top state handoff and test it



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-781) Implement Helix cluster view aggregator

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-781:
-

 Summary: Implement Helix cluster view aggregator
 Key: HELIX-781
 URL: https://issues.apache.org/jira/browse/HELIX-781
 Project: Apache Helix
  Issue Type: Task
Reporter: Harry Zhang
Assignee: Harry Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-780) Support get/add rest api for workflow/job/task user content

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-780:
-

 Summary: Support get/add rest api for workflow/job/task user 
content
 Key: HELIX-780
 URL: https://issues.apache.org/jira/browse/HELIX-780
 Project: Apache Helix
  Issue Type: Task
Reporter: Harry Zhang
Assignee: Harry Zhang


Need to support get/add rest api for workflow/job/task user content

AC:
 * finish implementation
 * test code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-779) Maintenance rebalancer should not clear preference list in ideal state

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-779:
-

 Summary: Maintenance rebalancer should not clear preference list 
in ideal state
 Key: HELIX-779
 URL: https://issues.apache.org/jira/browse/HELIX-779
 Project: Apache Helix
  Issue Type: Bug
  Components: helix-core
Reporter: Harry Zhang
Assignee: Harry Zhang


Setting list fields to empty map will prevent newly added and initially 
rebalanced resources during maintenance mode from getting re-balanced after 
cluster exists maintenance mode.
The right thing to do is to clear every preference list.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-775) Task driver should support add/get task framework user content

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-775:
-

 Summary: Task driver should support add/get task framework user 
content
 Key: HELIX-775
 URL: https://issues.apache.org/jira/browse/HELIX-775
 Project: Apache Helix
  Issue Type: Task
Reporter: Harry Zhang
Assignee: Harry Zhang


Task driver should support add/get task framework user content at 
workflow/job/task levels

 

AC:
 * finish implementation
 * add tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-773) Support getLastScheduledTaskTimestamp information in workflow rest api

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-773:
-

 Summary: Support getLastScheduledTaskTimestamp information in 
workflow rest api
 Key: HELIX-773
 URL: https://issues.apache.org/jira/browse/HELIX-773
 Project: Apache Helix
  Issue Type: Bug
Reporter: Harry Zhang
Assignee: Harry Zhang


Support getLastScheduledTaskTimestamp information in workflow rest api



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-771) More detailed top state handoff metrics

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-771:
-

 Summary: More detailed top state handoff metrics
 Key: HELIX-771
 URL: https://issues.apache.org/jira/browse/HELIX-771
 Project: Apache Helix
  Issue Type: Bug
  Components: helix-core
Reporter: Harry Zhang
Assignee: Harry Zhang


To define top state handoff SLA, we need some more detailed data:
 * graceful top state handoff (i.e. disable instance / resource / etc, both 
Helix and e2e latency)
 * abrupt top state handoff (i.e. node crash)

AC:
 - prepare metrics, test, code complete



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-753) record top state handoff finished in single cluster data cache refresh

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-753:
-

 Summary: record top state handoff finished in single cluster data 
cache refresh
 Key: HELIX-753
 URL: https://issues.apache.org/jira/browse/HELIX-753
 Project: Apache Helix
  Issue Type: Bug
Reporter: Harry Zhang
Assignee: Harry Zhang


Currently we are calculating top state handoff duration by doing the following:
 - record missing top state when we see a top state missing
 - record top state come back when we see it come back
 - report top state handoff duration

This is perfectly fine for non-P2P state transitions as the entire top state 
handoff process will always finish for >= 2 pipeline runs. However, for P2P 
enabled clusters, top state handoff are quick, and if it is quicker than 
cluster data refresh stage latency, we will lose a lot of short top state 
handoffs, which make the number miserable on ingraph.

We need to revise top state handoff metrics implementation so we don't lose 
data point statistically (i.e. we are losing all short handoffs now).

AC:
 - revise impl so we catch those short top state hand-offs
 - write new tests to catch the fix if needed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HELIX-750) Implement Helix View Aggregator Service

[Harry Zhang (JIRA)]

Harry Zhang created HELIX-750:
-

 Summary: Implement Helix View Aggregator Service
 Key: HELIX-750
 URL: https://issues.apache.org/jira/browse/HELIX-750
 Project: Apache Helix
  Issue Type: Task
Reporter: Harry Zhang
Assignee: Harry Zhang


I have sent out the design for Helix view aggregation service to serve cross 
data center information in a more convenient way 
([https://github.com/apache/helix/pull/266)]

This ticket is to implement the service based on approved design.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)