Re: [VOTE] Apache Helix 0.6.7 Release

[Kanak Biscuitwala]

+1



From: kishore g 
Sent: Sunday, January 22, 2017 7:03 PM
To: dev@helix.apache.org
Subject: Re: [VOTE] Apache Helix 0.6.7 Release

+1

On Jan 22, 2017 6:30 PM, "Olivier Lamy"  wrote:

> +1
>
> On 20 January 2017 at 02:13, Lei Xia  wrote:
>
> > Hi,
> >
> > This is to call for a vote on releasing the following candidate as
> > Apache Helix 0.6.7. This is the tenth release of Helix as an Apache
> > project, as well as the 6th release as a top-level Apache project.
> >
> > Apache Helix is a generic cluster management framework that makes it
> > easy to build partitioned and replicated, fault-tolerant and scalable
> > distributed systems.
> >
> > Release notes:http://helix.apache.org/0.6.7-docs/releasenotes/
> > release-0.6.7.html
> >
> > Release artifacts:https://repository.apache.org/content/
> > repositories/orgapachehelix-1008
> >
> > Distribution:
> > * binaries:https://dist.apache.org/repos/dist/dev/helix/0.6.7/binaries/
> > * sources:https://dist.apache.org/repos/dist/dev/helix/0.6.7/src/
> >
> > The 0.6.7 release
> > tag:https://git-wip-us.apache.org/repos/asf?p=helix.git;a=
> > tag;h=refs/tags/helix-0.6.7
> >
> > KEYS file available here:https://dist.apache.org/
> repos/dist/dev/helix/KEYS
> >
> > Please vote on the release. The vote will be open for at least 72 hours.
> >
> > [+1] -- "YES, release"
> > [0] -- "No opinion"
> > [-1] -- "NO, do not release"
> >
> > Thanks,
> > The Apache Helix Team
> >
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>

Re: [VOTE] Apache Helix 0.6.6 Release

[Kanak Biscuitwala]

+1



From: Greg Brandt 
Sent: Sunday, November 6, 2016 8:53 PM
To: dev@helix.apache.org
Subject: Re: [VOTE] Apache Helix 0.6.6 Release

+1

On Sun, Nov 6, 2016 at 7:59 PM, kishore g  wrote:

> +1.
>
> On Nov 6, 2016 7:00 PM, "Olivier Lamy"  wrote:
>
> > +1
> >
> > On 3 November 2016 at 09:19, Lei Xia  wrote:
> >
> > > Hi,
> > >
> > > This is to call for a vote on releasing the following candidate as
> > > Apache Helix 0.6.6. This is the 9th release of Helix as an Apache
> > > project, as well as the 5th release as a top-level Apache project.
> > >
> > > Apache Helix is a generic cluster management framework that makes it
> > > easy to build partitioned and replicated, fault-tolerant and scalable
> > > distributed systems.
> > >
> > > Release notes:http://helix.apache.org/0.6.6-docs/releasenotes/
> > > release-0.6.6.html#
> > >
> > > Release artifacts:https://repository.apache.org/content/
> > > repositories/orgapachehelix-1007
> > >
> > > Distribution:
> > > * binaries:https://dist.apache.org/repos/dist/dev/helix/0.6.
> 6/binaries/
> > > * sources:https://dist.apache.org/repos/dist/dev/helix/0.6.6/src/
> > >
> > > The [VERSION] release
> > > tag:https://git-wip-us.apache.org/repos/asf?p=helix.git;a=
> > > tag;h=refs/tags/helix-0.6.6
> > >
> > > KEYS file available here:https://dist.apache.org/
> > repos/dist/dev/helix/KEYS
> > >
> > > Please vote on the release. The vote will be open for at least 72
> hours.
> > >
> > > [+1] -- "YES, release"
> > > [0] -- "No opinion"
> > > [-1] -- "NO, do not release"
> > >
> > > Thanks,
> > > The Apache Helix Team
> > >
> >
> >
> >
> > --
> > Olivier Lamy
> > http://twitter.com/olamy | http://linkedin.com/in/olamy
Olivier Lamy (@olamy) | Twitter
twitter.com
The latest Tweets from Olivier Lamy (@olamy). Speaking English with a strong 
French accent! #OpenSource #Apache #jetty #jenkins #ultrarunning. Living 
#downunder @Melbourne. Father of four && husband of one!. Melbourne, Australia



> >
>

[ANNOUNCE] New committer: Lei Xia

[Kanak Biscuitwala]

The Project Management Committee (PMC) for Apache Helix has asked Joe Bloggs to 
become a committer and we are pleased  to announce that he has accepted.

Being a committer enables easier contribution to the project since there is no 
need to go via the patch submission process. This should enable better 
productivity.   

RE: calling ZKHelixLock from state machine transition

[Kanak Biscuitwala]

Hi,

ZkHelixLock is a thin wrapper around the ZooKeeper WriteLock recipe (which was 
last changed over 5 years ago). Though we haven't extensively tested it in 
production, but we haven't seen it fail to return as described.

Do you know if ZKHelixLock._listener.lockAcquired() is ever called?

Feel free to examine the code here: 
https://github.com/apache/helix/blob/master/helix-core/src/main/java/org/apache/helix/lock/zk/ZKHelixLock.java

> From: neutronsh...@gmail.com
> Date: Mon, 9 May 2016 14:26:43 -0700
> Subject: calling ZKHelixLock from state machine transition
> To: dev@helix.apache.org
> 
> Hi Helix team,
> 
> We observed an issue at state machine transition handle:
> 
> // statemodel.java:
> 
> public void offlineToSlave(Message message, NotificationContext context) {
> 
>   // do work to start a local shard
> 
>   // we want to save the new shard info to resource config
> 
> 
>   ZKHelixLock zklock = new ZKHelixLock(clusterId, resource, zkclient);
>   try {
> zklock.lock();// ==> will be blocked here
> 
> ZNRecord record = zkclient.readData(scope.getZkPath(), true);
> update record fields;
> zkclient.writeData(scope.getZkPath(), record);
>   } finally {
> zklock.unlock();
>   }
> }
> 
> After several invocation of this method,  zklock.lock() method doesn't
> return (so the lock is not acquired).  State machine threads become
> blocked.
> 
> At zk path "/LOCKS/RESOURCE_resource"  I see several znodes
> there, representing outstanding lock requests.
> 
> Are there any special care we should be aware of about zk lock ?  Thanks.
> 
> 
> -neutronsharc
  

RE: calling ZKHelixLock from state machine transition

[Kanak Biscuitwala]

Hi,

ZkHelixLock is a thin wrapper around the ZooKeeper WriteLock recipe (which was 
last changed over 5 years ago). Though we haven't extensively tested it in 
production, but we haven't seen it fail to return as described.

Do you know if ZKHelixLock._listener.lockAcquired() is ever called?

Feel free to examine the code here: 
https://github.com/apache/helix/blob/master/helix-core/src/main/java/org/apache/helix/lock/zk/ZKHelixLock.java

> From: neutronsh...@gmail.com
> Date: Mon, 9 May 2016 14:26:43 -0700
> Subject: calling ZKHelixLock from state machine transition
> To: dev@helix.apache.org
> 
> Hi Helix team,
> 
> We observed an issue at state machine transition handle:
> 
> // statemodel.java:
> 
> public void offlineToSlave(Message message, NotificationContext context) {
> 
>   // do work to start a local shard
> 
>   // we want to save the new shard info to resource config
> 
> 
>   ZKHelixLock zklock = new ZKHelixLock(clusterId, resource, zkclient);
>   try {
> zklock.lock();// ==> will be blocked here
> 
> ZNRecord record = zkclient.readData(scope.getZkPath(), true);
> update record fields;
> zkclient.writeData(scope.getZkPath(), record);
>   } finally {
> zklock.unlock();
>   }
> }
> 
> After several invocation of this method,  zklock.lock() method doesn't
> return (so the lock is not acquired).  State machine threads become
> blocked.
> 
> At zk path "/LOCKS/RESOURCE_resource"  I see several znodes
> there, representing outstanding lock requests.
> 
> Are there any special care we should be aware of about zk lock ?  Thanks.
> 
> 
> -neutronsharc
  

Re: Review Request 37704: Adds a Maven archetype to create OnlineOffline application

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37704/#review96132
---

Ship it!


Minor comments, but looks good otherwise!


helix-archetype/README.md (line 31)
https://reviews.apache.org/r/37704/#comment151386

Update references to point to the Apache repo.



helix-archetype/README.md (line 95)
https://reviews.apache.org/r/37704/#comment151387

Maybe also include some cleanup instructions?



helix-archetype/src/main/resources/archetype-resources/pom.xml (line 23)
https://reviews.apache.org/r/37704/#comment151388

whitespace



helix-archetype/src/test/conf/testng.xml (line 24)
https://reviews.apache.org/r/37704/#comment151389

a couple whitespace issues in this file


- Kanak Biscuitwala


On Aug. 23, 2015, 5:17 p.m., Greg Brandt wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/37704/
 ---
 
 (Updated Aug. 23, 2015, 5:17 p.m.)
 
 
 Review request for helix.
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 Generates a simple OnlineOffline application via the command:
 
 ```
 mvn archetype:generate \
   -DarchetypeGroupId=org.apache.helix \
   -DarchetypeArtifactId=helix-archetype \
   -DarchetypeVersion=0.7.2-SNAPSHOT \
   -DgroupId=com.example \
   -DartifactId=my-app \
   -Dname=MyApp \
   -DinteractiveMode=false
 ```
 
 
 Diffs
 -
 
   helix-archetype/.gitignore PRE-CREATION 
   helix-archetype/README.md PRE-CREATION 
   helix-archetype/pom.xml PRE-CREATION 
   helix-archetype/src/main/resources/META-INF/maven/archetype-metadata.xml 
 PRE-CREATION 
   helix-archetype/src/main/resources/archetype-resources/pom.xml PRE-CREATION 
   
 helix-archetype/src/main/resources/archetype-resources/src/main/java/__name__Main.java
  PRE-CREATION 
   
 helix-archetype/src/main/resources/archetype-resources/src/main/java/participant/__name__Participant.java
  PRE-CREATION 
   
 helix-archetype/src/main/resources/archetype-resources/src/main/java/participant/__name__StateTransitionHandler.java
  PRE-CREATION 
   
 helix-archetype/src/main/resources/archetype-resources/src/main/java/participant/__name__StateTransitionHandlerFactory.java
  PRE-CREATION 
   
 helix-archetype/src/main/resources/archetype-resources/src/main/java/spectator/__name__Spectator.java
  PRE-CREATION 
   
 helix-archetype/src/main/resources/archetype-resources/src/main/resources/log4j.xml
  PRE-CREATION 
   helix-archetype/src/test/conf/testng.xml PRE-CREATION 
   pom.xml 4faf806 
 
 Diff: https://reviews.apache.org/r/37704/diff/
 
 
 Testing
 ---
 
 mvn test -pl helix-archetype # passes
 mvn mvn -Prat -DskipTests -pl helix-archetype # passes
 
 
 Thanks,
 
 Greg Brandt
 

[jira] [Created] (HELIX-604) Update website with recent changes

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-604:
---

 Summary: Update website with recent changes
 Key: HELIX-604
 URL: https://issues.apache.org/jira/browse/HELIX-604
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


Complete the following:
- Add Greg Brandt as a committer
- Update committer list with correct titles
- Fix broken CI link
- Download links for old releases should use archive.apache.org



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 36243: [HELIX-604] Update website with committers, download links

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36243/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-604


Repository: helix-git


Description
---

commit 3cb08dbbaebc7b864a0de516eaae10da72e1a424
Author: Kanak Biscuitwala kana...@hotmail.com
Date:   Mon Jul 6 21:48:28 2015 -0700

[HELIX-604] Update website with committers, download links

:100644 100644 72fa105... 4faf806... M  pom.xml
:100644 100644 15805d0... 13b915e... M  
website/0.6.1-incubating/src/site/xdoc/download.xml.vm
:100644 100644 750dd4d... 6356a72... M  
website/0.6.2-incubating/src/site/xdoc/download.xml.vm
:100644 100644 c69dbbd... 404f2df... M  
website/0.6.3/src/site/xdoc/download.xml.vm
:100644 100644 931387e... d992e74... M  
website/0.6.4/src/site/xdoc/download.xml.vm
:100644 100644 17fbb43... c1dbc87... M  
website/0.6.5/src/site/xdoc/download.xml.vm
:100644 100644 768ed27... c37480c... M  
website/0.7.0-incubating/src/site/xdoc/download.xml.vm
:100644 100644 60715d1... 9355009... M  website/src/site/site.xml


Diffs
-

  pom.xml 72fa105 
  website/0.6.1-incubating/src/site/xdoc/download.xml.vm 15805d0 
  website/0.6.2-incubating/src/site/xdoc/download.xml.vm 750dd4d 
  website/0.6.3/src/site/xdoc/download.xml.vm c69dbbd 
  website/0.6.4/src/site/xdoc/download.xml.vm 931387e 
  website/0.6.5/src/site/xdoc/download.xml.vm 17fbb43 
  website/0.7.0-incubating/src/site/xdoc/download.xml.vm 768ed27 
  website/src/site/site.xml 60715d1 

Diff: https://reviews.apache.org/r/36243/diff/


Testing
---

The site is live at helix.apache.org


Thanks,

Kanak Biscuitwala

[jira] [Commented] (HELIX-604) Update website with recent changes

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616175#comment-14616175
 ] 

Kanak Biscuitwala commented on HELIX-604:
-

https://reviews.apache.org/r/36243/

 Update website with recent changes
 --

 Key: HELIX-604
 URL: https://issues.apache.org/jira/browse/HELIX-604
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala

 Complete the following:
 - Add Greg Brandt as a committer
 - Update committer list with correct titles
 - Fix broken CI link
 - Download links for old releases should use archive.apache.org



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

New Committer: Greg Brandt

[Kanak Biscuitwala]

The Project Management Committee (PMC) for Apache Helix has asked Greg Brandt 
to become a committer and we are pleased to announce that he has accepted.

Greg has made a number of contributions to the project, including two top-level 
modules: helix-ipc and helix-ui.

Being a committer enables easier contribution to the project since there is no 
need to go via the patch submission process. This should enable better 
productivity.   

RE: helix-ui dependency

[Kanak Biscuitwala]

Taking a quick look, the only dependency that is LGPL and isn't dual-license 
with Apache, Eclipse, or similar is org.beanshell.bsh. Is this dependency 
critical? If not, we can just add a Maven exclusion.


 From: brandt.g...@gmail.com 
 Date: Sun, 12 Apr 2015 19:40:44 -0700 
 Subject: Re: helix-ui dependency 
 To: dev@helix.apache.org 
 
 I've attached licenses.xml result of running mvn 
 license:download-licenses -f helix-ui/pom.xml 
 
 There are some with LGPL, so helix-ui is not in accordance with 
 https://www.apache.org/legal/resolved.html#category-x 
 
 It may make sense to move helix-ui to contributors/helix-ui. 
 
 -Greg 
 
 On Sun, Apr 12, 2015 at 6:57 PM, kishore g 
 g.kish...@gmail.commailto:g.kish...@gmail.com wrote: 
 Helix-ui module has the following dependencies (close to 100). 
 
 Are the licenses are compatible. Also do we need all these jars, can we 
 eclude the ones we don't need? 
 
 validation-api-1.1.0.Final.jar 
 javax.servlet-api-3.1.0.jar 
 javax.ws.rs-api-2.0.1.jar 
 javax.annotation-api-1.2.jar 
 dropwizard-core-0.8.0.jar 
 dropwizard-util-0.8.0.jar 
 jackson-annotations-2.5.0.jar 
 guava-18.0.jar 
 jsr305-3.0.0.jar 
 joda-time-2.7.jar 
 dropwizard-jackson-0.8.0.jar 
 jackson-core-2.5.1.jar 
 jackson-databind-2.5.1.jar 
 jackson-datatype-jdk7-2.5.1.jar 
 jackson-datatype-guava-2.5.1.jar 
 jackson-module-afterburner-2.5.1.jar 
 jackson-datatype-joda-2.5.1.jar 
 slf4j-api-1.7.10.jar 
 logback-classic-1.1.2.jar 
 logback-core-1.1.2.jar 
 dropwizard-validation-0.8.0.jar 
 hibernate-validator-5.1.3.Final.jar 
 jboss-logging-3.1.3.GA.jar 
 classmate-1.0.0.jar 
 javax.el-3.0.0.jar 
 dropwizard-configuration-0.8.0.jar 
 jackson-dataformat-yaml-2.5.1.jar 
 snakeyaml-1.12.jar 
 commons-lang3-3.3.2.jar 
 dropwizard-logging-0.8.0.jar 
 metrics-logback-3.1.0.jar 
 metrics-core-3.1.0.jar 
 jul-to-slf4j-1.7.10.jar 
 log4j-over-slf4j-1.7.10.jar 
 jcl-over-slf4j-1.7.10.jar 
 jetty-util-9.2.9.v20150224.jar 
 dropwizard-metrics-0.8.0.jar 
 dropwizard-lifecycle-0.8.0.jar 
 jetty-server-9.2.9.v20150224.jar 
 jetty-http-9.2.9.v20150224.jar 
 jetty-io-9.2.9.v20150224.jar 
 dropwizard-jersey-0.8.0.jar 
 jersey-server-2.16.jar 
 jersey-common-2.16.jar 
 jersey-guava-2.16.jar 
 hk2-api-2.4.0-b09.jar 
 hk2-utils-2.4.0-b09.jar 
 aopalliance-repackaged-2.4.0-b09.jar 
 javax.inject-2.4.0-b09.jar 
 hk2-locator-2.4.0-b09.jar 
 javassist-3.18.1-GA.jar 
 osgi-resource-locator-1.0.1.jar 
 jersey-client-2.16.jar 
 jersey-media-jaxb-2.16.jar 
 jersey-metainf-services-2.16.jar 
 metrics-jersey2-3.1.0.jar 
 metrics-annotation-3.1.0.jar 
 jackson-jaxrs-json-provider-2.5.1.jar 
 jackson-jaxrs-base-2.5.1.jar 
 jackson-module-jaxb-annotations-2.5.1.jar 
 jersey-container-servlet-2.16.jar 
 jersey-container-servlet-core-2.16.jar 
 jetty-webapp-9.2.9.v20150224.jar 
 jetty-xml-9.2.9.v20150224.jar 
 jetty-servlet-9.2.9.v20150224.jar 
 jetty-security-9.2.9.v20150224.jar 
 jetty-continuation-9.2.9.v20150224.jar 
 dropwizard-servlets-0.8.0.jar 
 dropwizard-jetty-0.8.0.jar 
 metrics-jetty9-3.1.0.jar 
 jetty-servlets-9.2.9.v20150224.jar 
 metrics-jvm-3.1.0.jar 
 metrics-servlets-3.1.0.jar 
 metrics-healthchecks-3.1.0.jar 
 metrics-json-3.1.0.jar 
 argparse4j-0.4.4.jar 
 jetty-setuid-java-1.0.2.jar 
 dropwizard-assets-0.8.0.jar 
 dropwizard-views-freemarker-0.8.0.jar 
 dropwizard-views-0.8.0.jar 
 freemarker-2.3.21.jar 
 zookeeper-3.3.4.jar 
 jline-0.9.94.jar 
 jackson-core-asl-1.8.5.jar 
 jackson-mapper-asl-1.8.5.jar 
 commons-io-2.2.jar 
 commons-cli-1.2.jar 
 zkclient-0.1.jar 
 commons-math-2.1.jar 
 commons-codec-1.6.jar 
 testng-6.0.1.jar 
 junit-4.11.jar 
 hamcrest-core-1.3.jar 
 bsh-2.0b4.jar 
 jcommander-1.12.jar 
 
  

Review Request 32603: [HELIX-554] Fix TestEntropyFreeNodeBounce

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32603/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-554


Repository: helix-git


Description
---

commit 036ce2118921d2ee39c30dbf64a40001481c7133
Author: Kanak Biscuitwala kana...@hotmail.com
Date:   Sat Mar 28 22:11:32 2015 -0700

[HELIX-554] Fix TestEntropyFreeNodeBounce

:100644 100644 00522b8... 1588072... M  
helix-core/src/test/java/org/apache/helix/integration/TestEntropyFreeNodeBounce.java


Diffs
-

  
helix-core/src/test/java/org/apache/helix/integration/TestEntropyFreeNodeBounce.java
 00522b8 

Diff: https://reviews.apache.org/r/32603/diff/


Testing
---


Thanks,

Kanak Biscuitwala

RE: [VOTE] Apache Helix 0.6.5 Release [RC2]

[Kanak Biscuitwala]

+1



 Date: Thu, 19 Mar 2015 08:39:39 -0700
 Subject: Re: [VOTE] Apache Helix 0.6.5 Release [RC2]
 From: nehzgn...@gmail.com
 To: dev@helix.apache.org

 +1
 On Mar 19, 2015 12:49 AM, kishore g g.kish...@gmail.com wrote:

 Hi,

 This is to call for a vote on releasing the following candidate as Apache
 Helix 0.6.5. This is the 8th release of Helix as an Apache project, as well
 as the 4th release as a top-level Apache project.

 Apache Helix is a generic cluster management framework that makes it easy
 to build partitioned and replicated, fault-tolerant and scalable
 distributed systems.

 Release notes:
 http://helix.apache.org/releasenotes/release-0.6.5.html

 Release artifacts:
 https://repository.apache.org/content/repositories/orgapachehelix-1005

 Distribution:
 * binaries:
 https://dist.apache.org/repos/dist/dev/helix/0.6.5/binaries/
 * sources:
 https://dist.apache.org/repos/dist/dev/helix/0.6.5/src/

 The 0.6.5 release tag:

 https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tag;h=refs/tags/helix-0.6.5

 KEYS file available here:
 https://dist.apache.org/repos/dist/dev/helix/KEYS

 Please vote on the release. The vote will be open for at least 72 hours.

 [+1] -- YES, release
 [0] -- No opinion
 [-1] -- NO, do not release

 Thanks,
 The Apache Helix Team

  

RE: [VOTE] Apache Helix 0.6.5 Release

[Kanak Biscuitwala]

I see that the ivy has been updated in the code, but the release has not been 
updated. I'm not sure if that's allowed.


 Date: Tue, 17 Mar 2015 15:00:13 -0700
 Subject: Re: [VOTE] Apache Helix 0.6.5 Release
 From: nehzgn...@gmail.com
 To: dev@helix.apache.org

 I've fixed the ivy issue as well as updated the release notes:
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314020version=12328652

 Release Notes - Apache Helix - Version 0.6.5

 ** Bug
 * [HELIX-512] - add back HelixManager#getHealthReportCollector()
 interface to 0.6.x
 * [HELIX-514] - ZkBaseDataAccessor#set() should throw
 BadVersionException instead of return false in case of version mismatch
 * [HELIX-518] - Add integration tests to ensure helix tasks work as
 expected during master failover
 * [HELIX-519] - Add integration tests to ensure that kill-switch for
 Helix tasks work as expected
 * [HELIX-521] - Should not start
 GenericHelixController#ClusterEventProcessor in types other than CONTROLLER
 and CONTROLLER_PARTICIPANT
 * [HELIX-537] - org.apache.helix.task.TaskStateModel should have a
 shutdown method.
 * [HELIX-541] - Possible livelock in Helix controller
 * [HELIX-547] - AutoRebalancer may not converge in some rare situation
 * [HELIX-549] - Discarding Throwable exceptions makes threads
 unkillable.
 * [HELIX-550] - ZKHelixManager does not shutdown GenericHelixController
 threads.
 * [HELIX-552] - StateModelFactory#_stateModelMap should use both
 resourceName and partitionKey to map a state model
 * [HELIX-555] - ClusterStateVerifier leaks ZkClients.
 * [HELIX-559] - Helix web admin performance issues
 * [HELIX-562] - TaskRebalancer doesn't honor MaxAttemptsPerTask when
 FailureThreshold is larger than 0
 * [HELIX-563] - Throw more meaningful exceptions when
 AutoRebalanceStrategy#computePartitionAssignment inputs are invalid
 * [HELIX-572] - External view is recreated every time for bucketized
 resource
 * [HELIX-574] - fix bucketize resource bug in current state carryover
 * [HELIX-575] - Should not send FINALIZED callback when a bucketized
 resource is removed
 * [HELIX-579] - fix ivy files issue

 ** Improvement
 * [HELIX-524] - add getProgress() to Task interface
 * [HELIX-573] - Add support to compress/uncompress data on ZK
 * [HELIX-576] - Make StateModelFactory change backward compatible

 ** New Feature
 * [HELIX-546] - REST Admin APIs needed for helix job queue management
 * [HELIX-581] - Support deleting job from a job queue

 ** Task
 * [HELIX-539] - Add ivy file for helix-agent

 ** Test
 * [HELIX-580] - Fix test: TestBatchMessage#testSubMsgExecutionFail

 Thanks,
 Jason


 On Mon, Mar 16, 2015 at 11:22 PM, kishore g g.kish...@gmail.com wrote:

 Jason is working on the release notes. Does any one know if we have to re
 cut the release because of the ivy file?

 thanks,
 Kishore G

 On Mon, Mar 16, 2015 at 10:30 PM, Kanak Biscuitwala kana...@hotmail.com
 wrote:

 A couple things:


 1. Release notes are missing. You can just link to this page if you want
 to update the site later, but it's currently not up-to-date:

 https://issues.apache.org/jira/browse/HELIX/fixforversion/12328652/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel


 2. Rat is failing because of this file:
 helix-agent/helix-agent-0.6.5-SNAPSHOT.ivy


 
 Date: Mon, 16 Mar 2015 10:19:27 -0700
 Subject: Re: [VOTE] Apache Helix 0.6.5 Release
 From: nehzgn...@gmail.com
 To: dev@helix.apache.org

 +1

 On Mon, Mar 16, 2015 at 12:15 AM, kishore g g.kish...@gmail.com
 wrote:

 Hi,

 This is to call for a vote on releasing the following candidate as
 Apache
 Helix 0.6.5. This is the 8th release of Helix as an Apache project, as
 well
 as the 4th release as a top-level Apache project.

 Apache Helix is a generic cluster management framework that makes it
 easy
 to build partitioned and replicated, fault-tolerant and scalable
 distributed systems.

 Release notes:
 http://helix.apache.org/releasenotes/release-0.6.5.html

 Release artifacts:

 https://repository.apache.org/content/repositories/orgapachehelix-1004

 Distribution:
 * binaries:
 https://dist.apache.org/repos/dist/dev/helix/0.6.5/binaries/
 * sources:
 https://dist.apache.org/repos/dist/dev/helix/0.6.5/src/

 The 0.6.5 release tag:



 https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tag;h=refs/tags/helix-0.6.5

 KEYS file available here:
 https://dist.apache.org/repos/dist/dev/helix/KEYS

 Please vote on the release. The vote will be open for at least 72
 hours.

 [+1] -- YES, release
 [0] -- No opinion
 [-1] -- NO, do not release

 Thanks,
 The Apache Helix Team




  

RE: [VOTE] Apache Helix 0.6.5 Release

[Kanak Biscuitwala]

A couple things:


1. Release notes are missing. You can just link to this page if you want to 
update the site later, but it's currently not up-to-date: 
https://issues.apache.org/jira/browse/HELIX/fixforversion/12328652/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel


2. Rat is failing because of this file: 
helix-agent/helix-agent-0.6.5-SNAPSHOT.ivy



 Date: Mon, 16 Mar 2015 10:19:27 -0700
 Subject: Re: [VOTE] Apache Helix 0.6.5 Release
 From: nehzgn...@gmail.com
 To: dev@helix.apache.org

 +1

 On Mon, Mar 16, 2015 at 12:15 AM, kishore g g.kish...@gmail.com wrote:

 Hi,

 This is to call for a vote on releasing the following candidate as Apache
 Helix 0.6.5. This is the 8th release of Helix as an Apache project, as well
 as the 4th release as a top-level Apache project.

 Apache Helix is a generic cluster management framework that makes it easy
 to build partitioned and replicated, fault-tolerant and scalable
 distributed systems.

 Release notes:
 http://helix.apache.org/releasenotes/release-0.6.5.html

 Release artifacts:
 https://repository.apache.org/content/repositories/orgapachehelix-1004

 Distribution:
 * binaries:
 https://dist.apache.org/repos/dist/dev/helix/0.6.5/binaries/
 * sources:
 https://dist.apache.org/repos/dist/dev/helix/0.6.5/src/

 The 0.6.5 release tag:

 https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tag;h=refs/tags/helix-0.6.5

 KEYS file available here:
 https://dist.apache.org/repos/dist/dev/helix/KEYS

 Please vote on the release. The vote will be open for at least 72 hours.

 [+1] -- YES, release
 [0] -- No opinion
 [-1] -- NO, do not release

 Thanks,
 The Apache Helix Team

  

Re: Review Request 31836: [HELIX-573] Add support to automatically compress/uncompress data in Zookeeper

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31836/#review76077
---

Ship it!



helix-core/src/main/java/org/apache/helix/controller/stages/ExternalViewComputeStage.java
https://reviews.apache.org/r/31836/#comment123472

whitespace



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
https://reviews.apache.org/r/31836/#comment123474

indentation



helix-core/src/main/java/org/apache/helix/util/GZipCompressionUtil.java
https://reviews.apache.org/r/31836/#comment123473

License should come before imports.



helix-core/src/main/java/org/apache/helix/util/GZipCompressionUtil.java
https://reviews.apache.org/r/31836/#comment123475

whitespace



helix-core/src/test/java/org/apache/helix/integration/TestEnableCompression.java
https://reviews.apache.org/r/31836/#comment123476

whitespace



helix-core/src/test/java/org/apache/helix/integration/TestEnableCompression.java
https://reviews.apache.org/r/31836/#comment123477

whitespace


- Kanak Biscuitwala


On March 11, 2015, 8:49 a.m., Kishore Gopalakrishna wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/31836/
 ---
 
 (Updated March 11, 2015, 8:49 a.m.)
 
 
 Review request for helix.
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit 1ef44c2e9a132df3513a51e3a8dac658236a2263
 Author: Kishore Gopalakrishna g.kish...@gmail.com
 Date:   Sun Mar 8 16:40:29 2015 -0700
 
 [HELIX-573] Add support to automatically compress/uncompress data in 
 Zookeeper
 
 :100644 100644 4419fdd... 1f34529... M  
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java
 :100644 100644 2d7cb3c... 26d7e2b... M  
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
 :00 100644 000... 90c1e8e... A  
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordSerializer.java
 :100644 100644 e4b0b25... 95064f8... M  
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordStreamingSerializer.java
 :100755 100755 e869f25... 99ef81c... M  hpost-review.sh
 
 
 Diffs
 -
 
   
 helix-core/src/main/java/org/apache/helix/controller/stages/ExternalViewComputeStage.java
  903840c 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java 
 4419fdd 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
  2d7cb3c 
   helix-core/src/main/java/org/apache/helix/util/GZipCompressionUtil.java 
 PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/integration/TestEnableCompression.java
  PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordSerializer.java
  PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordStreamingSerializer.java
  e4b0b25 
 
 Diff: https://reviews.apache.org/r/31836/diff/
 
 
 Testing
 ---
 
 Added basic test for compress/uncompress
 
 
 Thanks,
 
 Kishore Gopalakrishna
 

RE: Cutting a release 0.6.5 tonight

[Kanak Biscuitwala]

The first two are concerning. I don't think the scheduler message test failures 
should block releases.


 From: l...@linkedin.com.INVALID
 To: dev@helix.apache.org
 CC: u...@helix.apache.org
 Subject: RE: Cutting a release 0.6.5 tonight
 Date: Wed, 11 Mar 2015 21:25:15 +

 The test failed are:

 org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates
 org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource
 org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3
 org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4
 org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints
 org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue

 I got these from my local build too. (mvn clean install package on 
 helix-0.6.x)



 Thanks
 Lei

 --

 Lei Xia
 Software Engineer
 Data Infrastructure/Distributed Data Systems/Nuage
 LinkedIn

 l...@linkedin.com
 www.linkedin.com/in/lxia1

 
 From: kishore g [g.kish...@gmail.com]
 Sent: Wednesday, March 11, 2015 2:03 PM
 To: dev@helix.apache.org
 Cc: u...@helix.apache.org
 Subject: Re: Cutting a release 0.6.5 tonight

 Hi Lei,

 Can you point to the failures?

 thanks,
 Kishore G

 On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia l...@linkedin.com.invalid wrote:

 Hi, Kishore

 I saw there are regression test failures from last two recent commits
 on 0.6.x branch, running from both local box and Linkedin's hudson jobs.
 Are we going to fix them before the release?


 Thanks
 Lei

 --

 Lei Xia
 Software Engineer
 Data Infrastructure/Distributed Data Systems/Nuage
 LinkedIn

 l...@linkedin.com
 www.linkedin.com/in/lxia1

 
 From: kishore g [g.kish...@gmail.com]
 Sent: Wednesday, March 11, 2015 12:04 PM
 To: u...@helix.apache.org
 Cc: dev@helix.apache.org
 Subject: Re: Cutting a release 0.6.5 tonight

 Will write a test case for migration and document the steps
 On Mar 11, 2015 11:37 AM, Varun Sharma va...@pinterest.com wrote:

 What would be the migration path from non-compressed buckets to
 compressed
 non bucket resources ? It seems even the CURRENTSTATES are being bucketed
 in this case, I thought that was not expected with bucketing. Does the
 controller read these current states appropriately ? To migrate, it seems
 that we would need to also rewrite the CURRENT STATES ?

 On Wed, Mar 11, 2015 at 10:02 AM, kishore g g.kish...@gmail.com wrote:

 Hi,

 I will work with Jason to cut a 0.6.5 release tonight.

 The new thing I added is to enableCompression while storing data in
 Zookeeper, this allows us to go up to 100k partitions per resource
 without
 having to use bucketing feature. We also fixed few bugs with bucketed
 resource just in case some one needs it.

 The property store api needs some changes, I plan to get it in today.

 Let me know if you need any other changes to be included. Are there any
 changes that went into 0.7.x branch that we need to merge it back in to
 0.6.x ?

 thanks,
 Kishore G








  

Re: Review Request 31975: [HELIX-573] Add support to automatically compress/uncompress data in Zookeeper

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31975/#review76208
---

Ship it!


Ship It!

- Kanak Biscuitwala


On March 11, 2015, 9:32 p.m., Kishore Gopalakrishna wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/31975/
 ---
 
 (Updated March 11, 2015, 9:32 p.m.)
 
 
 Review request for helix.
 
 
 Bugs: HELIX-573
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit d704af4423360b382fc5f895b85ad689fa003b92
 Author: Kishore Gopalakrishna g.kish...@gmail.com
 Date:   Wed Mar 11 21:30:31 2015 -0700
 
 [HELIX-573] Add support to automatically compress/uncompress data in 
 Zookeeper
 
 :100644 100644 80529df... d0bdbd9... M
 helix-core/src/main/java/org/apache/helix/controller/stages/ExternalViewComputeStage.java
 :100644 100644 1552e53... 45dbd58... M
 helix-core/src/test/java/org/apache/helix/integration/TestEnableCompression.java
 
 
 Diffs
 -
 
   
 helix-core/src/main/java/org/apache/helix/controller/stages/ExternalViewComputeStage.java
  80529df 
   
 helix-core/src/test/java/org/apache/helix/integration/TestEnableCompression.java
  1552e53 
 
 Diff: https://reviews.apache.org/r/31975/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Kishore Gopalakrishna
 

Re: Review Request 31970: [HELIX-574] fix bucketize resource bug in current state carryover

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31970/#review76202
---

Ship it!


Ship it! (assuming tests pass)

- Kanak Biscuitwala


On March 11, 2015, 7:28 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/31970/
 ---
 
 (Updated March 11, 2015, 7:28 p.m.)
 
 
 Review request for helix, Kanak Biscuitwala and Kishore Gopalakrishna.
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-574] fix bucketize resource bug in current state carryover
 
 One logic change is previous we don't allow drop a disabled resource. Now we 
 change to drop a resource regardless it's disabled or not.
 
 
 Diffs
 -
 
   
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/CustomRebalancer.java
  0f3fbc4 
   
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/util/ConstraintBasedAssignment.java
  bab357b 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ParticipantManagerHelper.java
  1bee2fe 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixDataAccessor.java 
 169c993 
   
 helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
  207a318 
 
 Diff: https://reviews.apache.org/r/31970/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Zhen Zhang
 

RE: Use compression to store data in ZK

[Kanak Biscuitwala]

I like this idea, but we would still need to support bucketizing either way 
because we cannot guarantee that the compressed version will be compact enough 
for every use case.

What types of compression schemes are you planning to support?


 Date: Sat, 7 Mar 2015 22:30:15 -0800
 Subject: Use compression to store data in ZK
 From: g.kish...@gmail.com
 To: dev@helix.apache.org

 Hi,

 Currently we have bucketing as one of the options when the number of
 partitions are large. We have couple of bugs with the handling of
 bucketized resources (one of them is fatal).

 One of the reasons to split the znode is because we use JSON to store the
 data in ZNode. While JSON is good for debugging, its space inefficient.

 A better option before going to bucketing is to support compression of
 Ideal state, current state and External View. This also gives good
 performance.

 I plan to add this support and make it configurable. Feedback/suggestions

 thanks,
 Kishore G
  

Re: Review Request 31832: [HELIX-572] Fixing External View update logic for bucketized resource

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31832/#review75644
---

Ship it!



helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
https://reviews.apache.org/r/31832/#comment122854

Remove indent


- Kanak Biscuitwala


On March 8, 2015, 12:19 a.m., Kishore Gopalakrishna wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/31832/
 ---
 
 (Updated March 8, 2015, 12:19 a.m.)
 
 
 Review request for helix.
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit 1cd1327f45d5bda9e6ee8d371353e43a65cae743
 Author: Kishore Gopalakrishna g.kish...@gmail.com
 Date:   Sat Mar 7 23:45:45 2015 -0800
 
 [HELIX-572] Fixing External View update logic for bucketized resource
 
 :100644 100644 38c1417... 358971d... M  
 helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixDataAccessor.java
 :100644 100644 7234658... f83d14e... M  
 helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.jav
 
 
 Diffs
 -
 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixDataAccessor.java 
 38c1417 
   
 helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
  7234658 
 
 Diff: https://reviews.apache.org/r/31832/diff/
 
 
 Testing
 ---
 
 Added the check for version of external view in the TestBucketizedResource 
 integration test case
 
 
 Thanks,
 
 Kishore Gopalakrishna
 

Re: Review Request 31836: [HELIX-573] Add support to automatically compress/uncompress data in Zookeeper

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31836/#review75661
---



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java
https://reviews.apache.org/r/31836/#comment122869

As currently written, compression will be default because valueOf returns 
true if the string is null. I'm not sure if you want that, or at the very 
least, it should be made explicit, i.e.:

```
if (record.getBooleanField(enableCompression, defaultValue) {
```



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java
https://reviews.apache.org/r/31836/#comment122870

Lines 91 and 92 can be combined into 1 line.



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java
https://reviews.apache.org/r/31836/#comment122868

This log message won't be very useful if the bytes are compressed.



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
https://reviews.apache.org/r/31836/#comment122876

Same comment as above. Use getBooleanField. Also, consider putting this 
code in a common area since it's copy-pasted from the other serializer class.



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
https://reviews.apache.org/r/31836/#comment122877

Same comment about code duplication.



helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
https://reviews.apache.org/r/31836/#comment122878

This method too.


- Kanak Biscuitwala


On March 8, 2015, 4:48 p.m., Kishore Gopalakrishna wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/31836/
 ---
 
 (Updated March 8, 2015, 4:48 p.m.)
 
 
 Review request for helix.
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit 1ef44c2e9a132df3513a51e3a8dac658236a2263
 Author: Kishore Gopalakrishna g.kish...@gmail.com
 Date:   Sun Mar 8 16:40:29 2015 -0700
 
 [HELIX-573] Add support to automatically compress/uncompress data in 
 Zookeeper
 
 :100644 100644 4419fdd... 1f34529... M  
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java
 :100644 100644 2d7cb3c... 26d7e2b... M  
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
 :00 100644 000... 90c1e8e... A  
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordSerializer.java
 :100644 100644 e4b0b25... 95064f8... M  
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordStreamingSerializer.java
 :100755 100755 e869f25... 99ef81c... M  hpost-review.sh
 
 
 Diffs
 -
 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixDataAccessor.java 
 169c993 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordSerializer.java 
 4419fdd 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZNRecordStreamingSerializer.java
  2d7cb3c 
   
 helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
  207a318 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordSerializer.java
  PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZNRecordStreamingSerializer.java
  e4b0b25 
   hpost-review.sh 82cbcf9 
 
 Diff: https://reviews.apache.org/r/31836/diff/
 
 
 Testing
 ---
 
 Added basic test for compress/uncompress
 
 
 Thanks,
 
 Kishore Gopalakrishna
 

RE: 0.7.2 release

[Kanak Biscuitwala]

+1 for a release

Unfortunately I don't have a computer capable of running all the tests (and am 
thus unable to do a release), but I can help with the website and javadocs.

From: zzh...@linkedin.com
To: u...@helix.apache.org; dev@helix.apache.org
Subject: RE: 0.7.2 release
Date: Wed, 25 Feb 2015 01:23:02 +







I can cherry pick all the fixes.




From: kishore g [g.kish...@gmail.com]

Sent: Tuesday, February 24, 2015 5:08 PM

To: dev@helix.apache.org

Cc: u...@helix.apache.org

Subject: Re: 0.7.2 release






how do we know the things that we have pushed to 0.6.x but not to 0.7?









On Tue, Feb 24, 2015 at 4:48 PM, Zhen Zhang 
zzh...@linkedin.com.invalid wrote:


+1



As far as I can think of, here are a few things we need to complete before 
releasing 0.7.2

- merge fixes that have been done on 0.6.x but haven't been ported to master yet

- fix 1 test failure

- fix any other critical bugs we have in 0.7



- Jason





From: Greg Brandt [brandt.g...@gmail.com]

Sent: Tuesday, February 24, 2015 4:05 PM

To: u...@helix.apache.org

Cc: dev@helix.apache.org

Subject: Re: 0.7.2 release





Bump.



What needs to be done for this?



-Greg



On Feb 18, 2015 9:02 PM, kishore g 
g.kish...@gmail.commailto:g.kish...@gmail.com wrote:



+1

The recent bug we found is critical and would love to see that in the release.



Thanks

Kishore G





On Feb 18, 2015 1:17 PM, Greg Brandt 
brandt.g...@gmail.commailto:brandt.g...@gmail.com wrote:

Hey guys,



Would it be possible to get an 0.7.2 release?



I have a use case that uses helix-ipc, for which there were a few critical

patches after 0.7.1 release.



Thanks,

-Greg














  

RE: integrating with riemann

[Kanak Biscuitwala]

Hi,


We tried creating a reference implementation a while back, where we could 
launch the server using Java code, and have it join as a participant in a Helix 
cluster. Here's the code, but it hasn't been heavily tested in production:


https://github.com/apache/helix/tree/helix-monitoring/helix-monitor-server/src/main/java/org/apache/helix/monitoring/riemann


Kanak


 From: tomer...@gmail.com
 Date: Sun, 28 Dec 2014 09:10:38 +
 Subject: integrating with riemann
 To: dev@helix.apache.org

 Hi

 I saw this interesting
 https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=34837670
 (helix +
 riemann exploration).

 You mentinoed you have an issue with riemann single point of failure. Did
 you have any thoughts on how to resolve this?

 I'm asking because I thought of implementing riemann state persistence to
 cassandra.

 Thanks
  

Re: Review Request 28974: [HELIX-559] Fix Helix web admin performance issues

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28974/#review64879
---



pom.xml
https://reviews.apache.org/r/28974/#comment107679

If I recall correctly, restlet 2.2.1 contains the ctrl+c CPU spike bug


- Kanak Biscuitwala


On Dec. 11, 2014, 4:07 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28974/
 ---
 
 (Updated Dec. 11, 2014, 4:07 p.m.)
 
 
 Review request for helix, Kishore Gopalakrishna and Shi Lu.
 
 
 Bugs: 559
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-559] Fix Helix web admin performance issues
 
 Helix web admin has a couple of performance issues:
 
 1) Use restlet default server which is slow
 Need to switch to use jetty
 
 2) Unnecessary json deserialization/serialization
 For reading idealStates from helix web admin, we read as ZNRecord, serialize 
 ZNRecord to byte arrays, and return the result. It's not necessary to do the 
 der/ser which costs lots of CPU cycles. Instead, we can read the raw data as 
 byte arrays and return it directly.
 
 
 Diffs
 -
 
   helix-admin-webapp/pom.xml 4e312d1 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/HelixAdminWebApp.java
  991886c 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/ClusterRepresentationUtil.java
  5e458c4 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/ExternalViewResource.java
  6ec28dc 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/IdealStateResource.java
  0081922 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/InstanceResource.java
  e62fe5e 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/ResourceGroupResource.java
  6dc721d 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/ResourceGroupsResource.java
  ad4e934 
   
 helix-admin-webapp/src/main/java/org/apache/helix/webapp/resources/ResourceUtil.java
  f066dfc 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/resources/TestJobQueuesResource.java
  6c0e0e1 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/resources/TestJsonParameters.java
  383ac21 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZKUtil.java c61dccd 
   pom.xml 92c1d7d 
 
 Diff: https://reviews.apache.org/r/28974/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Zhen Zhang
 

Review Request 28215: [HELIX-550] Shutdown GenericHelixController on disconnect (port to master)

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28215/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-550


Repository: helix-git


Description
---

This is a port to master of this PR: 
https://github.com/apache/helix/pull/11/files#diff-866b65f4aa4b4753224ff615eb2efc1eR533

commit bfb4a3d34228f5c3806b1eee9e98f401386e66a9
Author: Kanak Biscuitwala kana...@hotmail.com
Date:   Tue Nov 18 21:23:29 2014 -0800

[HELIX-550] Shutdown GenericHelixController on disconnect

:100644 100644 aef636e... 113cace... M  
helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
:100644 100644 295b69c... fafe604... M  
helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java


Diffs
-

  
helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
 6fa3d05 
  helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java 
295b69c 

Diff: https://reviews.apache.org/r/28215/diff/


Testing
---


Thanks,

Kanak Biscuitwala

[jira] [Commented] (HELIX-537) org.apache.helix.task.TaskStateModel should have a shutdown method.

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207665#comment-14207665
 ] 

Kanak Biscuitwala commented on HELIX-537:
-

RBs are unnecessary. Code reviews can be done directly on PRs.

 org.apache.helix.task.TaskStateModel should have a shutdown method.
 ---

 Key: HELIX-537
 URL: https://issues.apache.org/jira/browse/HELIX-537
 Project: Apache Helix
  Issue Type: Bug
  Components: helix-core
Affects Versions: 0.6.3
Reporter: Antony T Curtis
Priority: Blocker

 There should be a shutdown method to terminate the Timer and Executor which 
 the org.apache.helix.task.TaskStateModel class creates.
 ie.
 {noformat}
 public boolean shutdown(long timeout, TimeUnit unit)
   throws InterruptedException
 {
   reset();
   _taskExecutor.shutdown();
   _timer.cancel();
   return _taskExecutor.awaitTermination(timeout, unit);
 }
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 27808: [HELIX-543] Avoid moving partitions unnecessarily when auto-rebalancing

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27808/#review60959
---

Ship it!


Looks good, assuming that existing integration tests still pass. Some ones to 
try include TestAutoRebalance, TestFullAutoNodeTagging, 
TestAutoRebalancePartitionLimit, and TestIndependentTaskRebalancer. Only minor 
comments regarding the code.


helix-agent/helix-agent-0.7.2-SNAPSHOT.ivy
https://reviews.apache.org/r/27808/#comment102379

thanks!



helix-core/src/main/java/org/apache/helix/controller/strategy/AutoRebalanceStrategy.java
https://reviews.apache.org/r/27808/#comment102380

active for each participant*



helix-core/src/main/java/org/apache/helix/controller/strategy/AutoRebalanceStrategy.java
https://reviews.apache.org/r/27808/#comment102381

It may be cleaner to use Guava comparator helpers, but this is fine too.

Reference: 
https://code.google.com/p/guava-libraries/wiki/CommonObjectUtilitiesExplained#compare/compareTo


- Kanak Biscuitwala


On Nov. 10, 2014, 5:07 a.m., Tom Widmer wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/27808/
 ---
 
 (Updated Nov. 10, 2014, 5:07 a.m.)
 
 
 Review request for helix and Kanak Biscuitwala.
 
 
 Bugs: HELIX-543
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit ba45d4bc2019c4069fe810c6e61a0696391f207e
 Author: Tom Widmer tom.wid...@camcog.com
 Date:   Mon Nov 10 12:40:33 2014 +
 
 [HELIX-543] Add missing licence header
 
 :100644 100644 f59be07... ef1f57e... M
 helix-agent/helix-agent-0.7.2-SNAPSHOT.ivy
 
 commit 5f63c6d69d594c70e85de31d5ed67f62f348ecb0
 Author: Tom Widmer tom.wid...@camcog.com
 Date:   Mon Nov 10 12:26:40 2014 +
 
 [HELIX-543] Avoid moving partitions unnecessarily when auto-rebalancing
 
 This is done by allocating capacity first to those nodes which already 
 have
 the most partitions.
 
 :100644 100644 09b66c1... 6e0e226... M
 helix-core/src/main/java/org/apache/helix/controller/strategy/AutoRebalanceStrategy.java
 :100644 100644 1322b40... 25c550d... M
 helix-core/src/test/java/org/apache/helix/controller/strategy/TestAutoRebalanceStrategy.java
 
 
 Diffs
 -
 
   helix-agent/helix-agent-0.7.2-SNAPSHOT.ivy f59be07 
   
 helix-core/src/main/java/org/apache/helix/controller/strategy/AutoRebalanceStrategy.java
  09b66c1 
   
 helix-core/src/test/java/org/apache/helix/controller/strategy/TestAutoRebalanceStrategy.java
  1322b40 
 
 Diff: https://reviews.apache.org/r/27808/diff/
 
 
 Testing
 ---
 
 Added new test to check case (failed before, now passes). Ran other 
 re-balancing tests.
 
 
 Thanks,
 
 Tom Widmer
 

[jira] [Commented] (HELIX-536) IdealState should ignore null value for InstanceGroupTag.

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191238#comment-14191238
 ] 

Kanak Biscuitwala commented on HELIX-536:
-

Alternatively, you could make setInstanceGroup(null) mean 
getRecord().getSimpleFields().remove(IdealStateProperty.INSTANCE_GROUP_TAG.name())
 rather than simply ignoring it.

 IdealState should ignore null value for InstanceGroupTag.
 -

 Key: HELIX-536
 URL: https://issues.apache.org/jira/browse/HELIX-536
 Project: Apache Helix
  Issue Type: Bug
Reporter: Xia Lei





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HELIX-518) Add integration tests to ensure helix tasks work as expected during master failover

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155748#comment-14155748
 ] 

Kanak Biscuitwala commented on HELIX-518:
-

Task#cancel should force run() to return a canceled task state. This will set a 
requested state, which the rebalancer will honor and it will send a transition. 
So it should be correct to call Task#cancel.

 Add integration tests to ensure helix tasks work as expected during master 
 failover
 ---

 Key: HELIX-518
 URL: https://issues.apache.org/jira/browse/HELIX-518
 Project: Apache Helix
  Issue Type: Bug
Reporter: Zhen Zhang
Assignee: Zhen Zhang
   Original Estimate: 72h
  Remaining Estimate: 72h

 Integration test to verify that a helix task running on master gets 
 rescheduled to new master during mastership handoff
 Integration test to verify that a helix task running on master gets 
 rescheduled to new master when the current master just dies



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 26212: [HELIX-519] Add integration tests to ensure that kill-switch for Helix tasks work as expected

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26212/#review55051
---



helix-core/src/main/java/org/apache/helix/task/TaskDriver.java
https://reviews.apache.org/r/26212/#comment95390

Why this change?


- Kanak Biscuitwala


On Sept. 30, 2014, 6 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/26212/
 ---
 
 (Updated Sept. 30, 2014, 6 p.m.)
 
 
 Review request for helix, Kanak Biscuitwala and Shi Lu.
 
 
 Bugs: 519
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-519] Add integration tests to ensure that kill-switch for Helix 
 tasks work as expected
 
 
 Diffs
 -
 
   helix-core/src/main/java/org/apache/helix/task/TaskDriver.java 31d785b 
   
 helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancerStopResume.java
  4e0d92a 
   helix-core/src/test/java/org/apache/helix/integration/task/TestUtil.java 
 f599920 
 
 Diff: https://reviews.apache.org/r/26212/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Zhen Zhang
 

[jira] [Commented] (HELIX-522) Support dynamic config by Helix

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150975#comment-14150975
 ] 

Kanak Biscuitwala commented on HELIX-522:
-

I think if we keep it at version management and let someone else decide what 
those versions mean, that is probably sufficient and avoids the slippery slope.

A question I have is, what happens during a bounce of a node? Previously, we 
had a well-defined initial state that the node would start in, and further, a 
node that comes up is assumed to have not been assigned anything yet (and is 
thus in that initial state for all partitions). To maintain this property, we 
would need to define some transition that's like (partition, nothing) -- 
(partition, state, version), which is kind of like a 2D -- 3D transition, and 
that's fine, but it's worth keeping in mind since it doesn't follow the same 
pattern as other transitions.

 Support dynamic config by Helix
 ---

 Key: HELIX-522
 URL: https://issues.apache.org/jira/browse/HELIX-522
 Project: Apache Helix
  Issue Type: New Feature
Reporter: Zhen Zhang

 We are seeing a couple of use cases where an online service may need to roll 
 out a set of new configs in a coordinated way without bouncing the service. 
 Helix can be extended to support this. The basic idea is to generalize 
 ideal-state to support multi-dimensional mappings. Currently Helix uses 
 ideal-state to define the target mapping from a partition to the placement 
 and state of the partition (i.e. partition-(placement, state). The mapping 
 can be extended to support another attribute, say config-version, so it would 
 be partition-(placement, state, config-version). Helix controller is then 
 able to bring the system to the target mappings in a coordinated way. Here is 
 a high level design for this:
 https://cwiki.apache.org/confluence/display/HELIX/Support+Dynamic+Config



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 26045: [HELIX-521] Should not start GenericHelixController#ClusterEventProcessor in types other than CONTROLLER and CONTROLLER_PARTICIPANT

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26045/#review54604
---

Ship it!


Ship It!

- Kanak Biscuitwala


On Sept. 25, 2014, 11:03 a.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/26045/
 ---
 
 (Updated Sept. 25, 2014, 11:03 a.m.)
 
 
 Review request for helix and Shi Lu.
 
 
 Bugs: HELIX-521
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-521] Should not start GenericHelixController#ClusterEventProcessor in 
 types other than CONTROLLER and CONTROLLER_PARTICIPANT
 
 
 Diffs
 -
 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java 
 6c3923a 
 
 Diff: https://reviews.apache.org/r/26045/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Zhen Zhang
 

Re: Review Request 25927: [HELIX-489] Convert an existing example to use new API

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25927/#review54248
---

Ship it!



recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerStateModel.java
https://reviews.apache.org/r/25927/#comment94260

It would be nice to rename this to ConsumerTransitionHandler to make it 
consistent with the name change for the interface.


- Kanak Biscuitwala


On Sept. 22, 2014, 10:03 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/25927/
 ---
 
 (Updated Sept. 22, 2014, 10:03 p.m.)
 
 
 Review request for helix and Shi Lu.
 
 
 Bugs: HELIX-489
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-489] Convert rabbitmq to use new API
 
 
 Diffs
 -
 
   
 recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/Consumer.java
  de56171 
   
 recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerStateModel.java
  b41ccf7 
   
 recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerStateTransitionHandlerFactory.java
  a0d11f8 
   
 recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerThread.java
  ddd466c 
 
 Diff: https://reviews.apache.org/r/25927/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Zhen Zhang
 

[jira] [Commented] (HELIX-518) Add integration tests to ensure helix tasks work as expected during master failover

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139612#comment-14139612
 ] 

Kanak Biscuitwala commented on HELIX-518:
-

Clarification: a task running on the old master will *not* get rescheduled on 
the new master automatically if the old master is still alive. As part of the 

 Add integration tests to ensure helix tasks work as expected during master 
 failover
 ---

 Key: HELIX-518
 URL: https://issues.apache.org/jira/browse/HELIX-518
 Project: Apache Helix
  Issue Type: Bug
Reporter: Zhen Zhang
   Original Estimate: 72h
  Remaining Estimate: 72h

 Integration test to verify that a helix task running on master gets 
 rescheduled to new master during mastership handoff
 Integration test to verify that a helix task running on master gets 
 rescheduled to new master when the current master just dies



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HELIX-518) Add integration tests to ensure helix tasks work as expected during master failover

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139612#comment-14139612
 ] 

Kanak Biscuitwala edited comment on HELIX-518 at 9/18/14 10:18 PM:
---

Clarification: a task running on the old master will *not* get rescheduled on 
the new master automatically if the old master is still alive. As part of the 
master - slave transition, the old master needs to cancel any tasks it is 
currently running. At that point, the task will be rescheduled on the new 
master.


was (Author: kanak):
Clarification: a task running on the old master will *not* get rescheduled on 
the new master automatically if the old master is still alive. As part of the 

 Add integration tests to ensure helix tasks work as expected during master 
 failover
 ---

 Key: HELIX-518
 URL: https://issues.apache.org/jira/browse/HELIX-518
 Project: Apache Helix
  Issue Type: Bug
Reporter: Zhen Zhang
   Original Estimate: 72h
  Remaining Estimate: 72h

 Integration test to verify that a helix task running on master gets 
 rescheduled to new master during mastership handoff
 Integration test to verify that a helix task running on master gets 
 rescheduled to new master when the current master just dies



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

GitHub pull request process for committers

[Kanak Biscuitwala]

Hi,

I've written up a quick guide for how committers can merge pull requests from 
GitHub: https://cwiki.apache.org/confluence/display/HELIX/Merging+Pull+Requests

Let me know if you have any feedback or if anything isn't clear.

Kanak 

RE: GitHub pull request process for committers

[Kanak Biscuitwala]

It's optional, but it helps with aesthetics when you have multi-commit pull 
requests where the commits after the first are just deltas due to code reviews.

But yeah, you don't have to do it if you don't want to.

Kanak


 Date: Thu, 18 Sep 2014 21:18:52 -0700
 Subject: Re: GitHub pull request process for committers
 From: g.kish...@gmail.com
 To: dev@helix.apache.org

 thanks Kanak. qq. why is this required?

 git rebase -i HEAD~N # Replace N with the number of commits in this pull
 request
 # In the interactive rebase, keep pick next to the first commit, and
 change pick to fixup for subsequent commits
 git push --force origin/master




 On Thu, Sep 18, 2014 at 8:49 PM, Kanak Biscuitwala kana...@hotmail.com
 wrote:

 Hi,

 I've written up a quick guide for how committers can merge pull requests
 from GitHub:
 https://cwiki.apache.org/confluence/display/HELIX/Merging+Pull+Requests

 Let me know if you have any feedback or if anything isn't clear.

 Kanak
  

RE: [GitHub] helix pull request: refactored Netty IPC code into separate classe...

[Kanak Biscuitwala]

The steps are straightforward, and I can document them on the wiki:

0. Do a code review on the pull request on GitHub until it meets standards 
(e.g. code style, correctness, etc)

1. Check out helix from apache.

2. Pull changes from remote fork, e.g.

git pull https://github.com/brandtg/helix master

2a. If git creates a merge commit message, append (This closes #[PRNUM]) to 
the end, otherwise, create a new empty commit with at least that text

3. Run tests and rat:

mvn test
mvn -Prat -DskipTests

4. git push origin master


 Date: Tue, 16 Sep 2014 11:41:38 -0700
 Subject: Re: [GitHub] helix pull request: refactored Netty IPC code into 
 separate classe...
 From: g.kish...@gmail.com
 To: dev@helix.apache.org

 Do we have any documented on how to merge this pull request. Will be nice
 to have a script that does some validation?

 thanks,
 Kishore G

 On Tue, Sep 16, 2014 at 9:49 AM, brandtg g...@git.apache.org wrote:

 GitHub user brandtg opened a pull request:

 https://github.com/apache/helix/pull/5

 refactored Netty IPC code into separate classes



 You can merge this pull request into a Git repository by running:

 $ git pull https://github.com/brandtg/helix master

 Alternatively you can review and apply these changes as the patch at:

 https://github.com/apache/helix/pull/5.patch

 To close this pull request, make a commit to your master/trunk branch
 with (at least) the following in the commit message:

 This closes #5

 
 commit 21ecdc7c5ef7f4e2ad7a98239b5d70164ee83505
 Author: Greg Brandt brandt.g...@gmail.com
 Date: 2014-09-16T16:48:14Z

 refactored Netty IPC code into separte classes

 


 ---
 If your project is set up for it, you can reply to this email and have your
 reply appear on GitHub as well. If your project does not have this feature
 enabled and wishes so, or if the feature is enabled but not working, please
 contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
 with INFRA.
 ---

  

[jira] [Created] (HELIX-515) Prepare website for 0.6.4 and 0.7.1 releases

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-515:
---

 Summary: Prepare website for 0.6.4 and 0.7.1 releases
 Key: HELIX-515
 URL: https://issues.apache.org/jira/browse/HELIX-515
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 25615: [HELIX-515] Prepare website for 0.6.4 and 0.7.1 releases

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25615/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-515


Repository: helix-git


Description
---

commit de9197fa9f84caff4f3168200b889d2494c03341
Author: Kanak Biscuitwala kana...@hotmail.com
Date:   Sat Sep 13 19:00:28 2014 -0700

[HELIX-515] Prepare website for 0.6.4 and 0.7.1 releases

:100644 100644 2046121... e02b905... M  pom.xml
:00 100644 000... bc6e098... A  website/0.6.4/pom.xml
:00 100644 000... ada9363... A  
website/0.6.4/src/site/apt/privacy-policy.apt
:00 100644 000... 43aea0f... A  
website/0.6.4/src/site/apt/releasenotes/release-0.6.4.apt
:00 100644 000... fee9fb0... A  
website/0.6.4/src/site/markdown/Building.md
:00 100644 000... ba9d0e7... A  
website/0.6.4/src/site/markdown/Features.md
:00 100644 000... 9582151... A  
website/0.6.4/src/site/markdown/Quickstart.md
:00 100644 000... 1815ae3... A  
website/0.6.4/src/site/markdown/Tutorial.md
:00 100644 000... d6af590... A  website/0.6.4/src/site/markdown/index.md
:00 100644 000... a7bd24d... A  
website/0.6.4/src/site/markdown/recipes/lock_manager.md
:00 100644 000... 97a8e29... A  
website/0.6.4/src/site/markdown/recipes/rabbitmq_consumer_group.md
:00 100644 000... 7014072... A  
website/0.6.4/src/site/markdown/recipes/rsync_replicated_file_store.md
:00 100644 000... 2b87643... A  
website/0.6.4/src/site/markdown/recipes/service_discovery.md
:00 100644 000... 532ac7c... A  
website/0.6.4/src/site/markdown/recipes/task_dag_execution.md
:00 100644 000... 3ae6551... A  
website/0.6.4/src/site/markdown/tutorial_admin.md
:00 100644 000... 235d71d... A  
website/0.6.4/src/site/markdown/tutorial_agent.md
:00 100644 000... d3c5526... A  
website/0.6.4/src/site/markdown/tutorial_controller.md
:00 100644 000... 45bde40... A  
website/0.6.4/src/site/markdown/tutorial_messaging.md
:00 100644 000... cb38e45... A  
website/0.6.4/src/site/markdown/tutorial_participant.md
:00 100644 000... 7e7d86c... A  
website/0.6.4/src/site/markdown/tutorial_propstore.md
:00 100644 000... 2e1a79b... A  
website/0.6.4/src/site/markdown/tutorial_rebalance.md
:00 100644 000... e43cd6b... A  
website/0.6.4/src/site/markdown/tutorial_spectator.md
:00 100644 000... 856b8b3... A  
website/0.6.4/src/site/markdown/tutorial_state.md
:00 100644 000... 16a6f81... A  
website/0.6.4/src/site/markdown/tutorial_throttling.md
:00 100644 000... 2149739... A  
website/0.6.4/src/site/markdown/tutorial_user_def_rebalancer.md
:00 100644 000... 1e4772e... A  
website/0.6.4/src/site/markdown/tutorial_yaml.md
:00 100644 000... d5c7bf3... A  
website/0.6.4/src/site/resources/.htaccess
:00 100644 000... f9a0e30... A  
website/0.6.4/src/site/resources/download.cgi
:00 100644 000... 7eea3a0... A  
website/0.6.4/src/site/resources/images/PFS-Generic.png
:00 100644 000... 0cc55ae... A  
website/0.6.4/src/site/resources/images/RSYNC_BASED_PFS.png
:00 100644 000... 9bfa91c... A  website/0.6.4/src/site/site.xml
:00 100644 000... 931387e... A  
website/0.6.4/src/site/xdoc/download.xml.vm
:00 100644 000... 58f0803... A  website/0.6.4/src/test/conf/testng.xml
:00 100644 000... d88223c... A  website/0.7.1/pom.xml
:00 100644 000... ada9363... A  
website/0.7.1/src/site/apt/privacy-policy.apt
:00 100644 000... c01eb32... A  
website/0.7.1/src/site/apt/releasenotes/release-0.7.1.apt
:00 100644 000... b1d0c91... A  
website/0.7.1/src/site/markdown/Building.md
:00 100644 000... ba9d0e7... A  
website/0.7.1/src/site/markdown/Features.md
:00 100644 000... 94ac600... A  
website/0.7.1/src/site/markdown/Quickstart.md
:00 100644 000... cfe528a... A  
website/0.7.1/src/site/markdown/Tutorial.md
:00 100644 000... 678a1cd... A  website/0.7.1/src/site/markdown/index.md
:00 100644 000... 0b5b0c3... A  
website/0.7.1/src/site/markdown/recipes/lock_manager.md
:00 100644 000... cc69b16... A  
website/0.7.1/src/site/markdown/recipes/rabbitmq_consumer_group.md
:00 100644 000... 26b6b13... A  
website/0.7.1/src/site/markdown/recipes/rsync_replicated_file_store.md
:00 100644 000... 06d9d65... A  
website/0.7.1/src/site/markdown/recipes/service_discovery.md
:00 100644 000... 1fc25ac... A  
website/0.7.1/src/site/markdown/recipes/task_dag_execution.md
:00 100644 000... db8ceeb... A  
website/0.7.1/src/site/markdown/recipes/user_def_rebalancer.md
:00 100644 000... eb4b046... A  
website/0.7.1/src/site/markdown/tutorial_accessors.md
:00 100644 000... 0208c24... A  
website/0.7.1/src/site/markdown

[ANNOUNCE] Apache Helix 0.7.1 (beta) Release

[Kanak Biscuitwala]

The Apache Helix Team is pleased to announce version 0.7.1 (beta) of the Apache 
Helix project.

Apache Helix is a generic cluster management framework that makes it easy
to build partitioned, fault tolerant, and scalable distributed systems.

The full release notes are available here:
http://helix.apache.org/0.7.1-docs/releasenotes/release-0.7.1.html

You can declare a maven dependency to use it:

dependency
  groupIdorg.apache.helix/groupId
  artifactIdhelix-core/artifactId
  version0.7.1/version
/dependency

Or download the release sources:
http://helix.apache.org/0.7.1-docs/download.cgi

Additional info:

Website: http://helix.apache.org/
Helix mailing lists: http://helix.apache.org/mail-lists.html

We hope you will enjoy using the latest release of Apache Helix!

Cheers,
Apache Helix Team 

RE: [RESULT][VOTE] Apache Helix 0.6.4 Release

[Kanak Biscuitwala]

There is some website work I need to do first. Once the download pages are 
ready, I can announce.


 Date: Thu, 11 Sep 2014 17:41:15 -0700
 Subject: Re: [RESULT][VOTE] Apache Helix 0.6.4 Release
 From: g.kish...@gmail.com
 To: dev@helix.apache.org

 Did we send out the announcement?


 On Wed, Sep 3, 2014 at 9:06 PM, Kanak Biscuitwala kana...@hotmail.com
 wrote:

 Thanks for voting on the 0.6.4 release. It has now exceeded 72 hours so I
 am closing the vote.

 Binding +1s:
 Kanak Biscuitwala
 Zhen Zhang
 Shi Lu

 No 0s or -1s.

 The vote has passed, thanks a lot to everyone for voting!

  

[RESULT][VOTE] Apache Helix 0.7.1 (beta) Release

[Kanak Biscuitwala]

Thanks for voting on the 0.7.1 release. It has now exceeded 72 hours so I am 
closing the vote.

Binding +1s:
 Kanak Biscuitwala
 Zhen Zhang
 Shi Lu

No 0s or -1s.

The vote has passed, thanks a lot to everyone for voting!   
  

[RESULT][VOTE] Apache Helix 0.6.4 Release

[Kanak Biscuitwala]

Thanks for voting on the 0.6.4 release. It has now exceeded 72 hours so I am 
closing the vote.

Binding +1s:
 Kanak Biscuitwala
 Zhen Zhang
 Shi Lu

No 0s or -1s.

The vote has passed, thanks a lot to everyone for voting!   
  

RE: Apache Helix Contribution -- newbie

[Kanak Biscuitwala]

Hi Mitesh,

The best way to get started as a Helix developer is to get started as a Helix 
user. We recommend that you go through the documentation and recipes to see how 
it works, and more importantly, why it's useful.

Concepts: http://helix.apache.org/Concepts.htmlArchitecture: 
http://helix.apache.org/Architecture.htmlQuick start: 
http://helix.apache.org/0.6.3-docs/Quickstart.htmlTutorial: 
http://helix.apache.org/0.6.3-docs/Tutorial.htmlRecipes and other 
release-specific links: http://helix.apache.org/0.6.3-docs/index.html
Once you're familiar with the architecture how to use Helix, the next step is 
to work on some issues that are things that newbies can pick up: 
https://issues.apache.org/jira/browse/HELIX-413?jql=project%20%3D%20HELIX%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20in%20(newbie%2C%20starter%2C%20easy%2C%20beginners)
At that point, you will likely be able to pick up other Helix tasks.
If you have any questions, this is the correct alias to email, and we're also 
available via IRC on freenode #apachehelix.
Kanak

 Date: Sat, 30 Aug 2014 21:28:07 +0530
 Subject: Apache Helix Contribution -- newbie
 From: miteshpatha...@gmail.com
 To: dev@helix.apache.org
 
 Hi Team,
 
 I am new to Helix, and I would like to contribute.
 I have gone through the wiki page on How to contribute, but I want to
 start working on the issues that will help me understand Helix better.
 
 Initially I'll be less active, but I am planning to work on this project
 late in the year.
 I'll be grateful if someone can guide me through this phase.
 
 About Me - I am a B.E pass-out who loves Java, I hope its good enough to
 get started.
 
 Thanks  Regards
 Mitesh Pathak
 mailto: miteshpatha...@gmail.com
 website: miteshpathak.com
  

[jira] [Created] (HELIX-511) TestSchedulerMessage#testSchedulerMsg4 is flaky

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-511:
---

 Summary: TestSchedulerMessage#testSchedulerMsg4 is flaky
 Key: HELIX-511
 URL: https://issues.apache.org/jira/browse/HELIX-511
 Project: Apache Helix
  Issue Type: Bug
Reporter: Kanak Biscuitwala


This test is slowing down our releases because it needs to pass twice in order 
to complete the two release steps. It currently has a 33% success rate on my 
box.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[VOTE] Apache Helix 0.7.1 (beta) Release

[Kanak Biscuitwala]

Hi,

This is to call for a vote on releasing the following candidate as Apache Helix 
0.7.1 (beta). This is the 7th release of Helix as an Apache project, as well as 
the 3rd release as a top-level Apache project.

Apache Helix is a generic cluster management framework that makes it easy to 
build partitioned and replicated, fault-tolerant and scalable distributed 
systems.

Release notes:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314020version=12325579

Release artifacts:
https://repository.apache.org/content/repositories/orgapachehelix-1003

Distribution:
* binaries:
https://dist.apache.org/repos/dist/dev/helix/0.7.1/binaries/
* sources:
https://dist.apache.org/repos/dist/dev/helix/0.7.1/src/

The 0.7.1 release tag:
https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tag;h=refs/tags/helix-0.7.1

KEYS file available here:
https://dist.apache.org/repos/dist/dev/helix/KEYS

Please vote on the release. The vote will be open for at least 72 hours.

[+1] -- YES, release
[0] -- No opinion
[-1] -- NO, do not release

Thanks,
The Apache Helix Team 

[jira] [Commented] (HELIX-508) Write a test for monitoring controller lifecycle

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115881#comment-14115881
 ] 

Kanak Biscuitwala commented on HELIX-508:
-

Wiki page: 
https://cwiki.apache.org/confluence/display/HELIX/Controller+Lifecycle+Listener

 Write a test for monitoring controller lifecycle
 

 Key: HELIX-508
 URL: https://issues.apache.org/jira/browse/HELIX-508
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala

 It's important to make sure that controller lifecycles are sufficiently 
 expressible so that controller nodes can do extra work when becoming leaders.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 25210: [HELIX-512] add back HelixManager#getHealthReportCollector() interface to 0.6.x

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25210/#review51936
---

Ship it!


Ship It!

- Kanak Biscuitwala


On Aug. 29, 2014, 4:49 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/25210/
 ---
 
 (Updated Aug. 29, 2014, 4:49 p.m.)
 
 
 Review request for helix and Kanak Biscuitwala.
 
 
 Bugs: HELIX-512
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-512] add back HelixManager#getHealthReportCollector() interface to 
 0.6.x
 
 
 Diffs
 -
 
   helix-core/src/main/java/org/apache/helix/HelixManager.java b4d918b 
   helix-core/src/main/java/org/apache/helix/PropertyKey.java f4e47a0 
   
 helix-core/src/main/java/org/apache/helix/healthcheck/ParticipantHealthReportCollector.java
  e69de29 
   
 helix-core/src/main/java/org/apache/helix/healthcheck/ParticipantHealthReportCollectorImpl.java
  e69de29 
   
 helix-core/src/main/java/org/apache/helix/healthcheck/ParticipantHealthReportTask.java
  e69de29 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java 
 cc24656 
   helix-core/src/main/java/org/apache/helix/model/HealthStat.java e69de29 
   helix-core/src/test/java/org/apache/helix/Mocks.java e6582e0 
   
 helix-core/src/test/java/org/apache/helix/controller/stages/DummyClusterManager.java
  73ba122 
   
 helix-core/src/test/java/org/apache/helix/healthcheck/TestParticipantHealthReportCollectorImpl.java
  e69de29 
   
 helix-core/src/test/java/org/apache/helix/participant/MockZKHelixManager.java 
 11cdd34 
 
 Diff: https://reviews.apache.org/r/25210/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Zhen Zhang
 

[jira] [Created] (HELIX-509) Determine if LICENSE and NOTICE need to be duplicated in all modules

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-509:
---

 Summary: Determine if LICENSE and NOTICE need to be duplicated in 
all modules
 Key: HELIX-509
 URL: https://issues.apache.org/jira/browse/HELIX-509
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala


Right now, the Helix source code has a LICENSE and NOTICE at the top of the 
tree, but also duplicates of these files in submodules (or in the case of 
NOTICE, specific subsets of dependencies relevant to each submodule). We need 
to figure out if we can remove this duplication safely while still complying 
with http://www.apache.org/legal/src-headers.html



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-510) Apache CI is stalling while running tests

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-510:
---

 Summary: Apache CI is stalling while running tests
 Key: HELIX-510
 URL: https://issues.apache.org/jira/browse/HELIX-510
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala


It looks like there might be some sort of resource exhaustion or other issue 
that is preventing CI from proceeding. We should look at trying to reduce the 
number of threads, connections, etc that we have open at any time.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[VOTE] Apache Helix 0.6.4 Release

[Kanak Biscuitwala]

Hi,


This is to call for a vote on releasing the following candidate as Apache Helix 
0.6.4. This is the 6th release of Helix as an Apache project, as well as the 
2nd release as a top-level Apache project.


Apache Helix is a generic cluster management framework that makes it easy to 
build partitioned and replicated, fault-tolerant and scalable distributed 
systems.


Release notes:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314020version=12326649


Release artifacts:
https://repository.apache.org/content/repositories/orgapachehelix-1001


Distribution:

* binaries:
https://dist.apache.org/repos/dist/dev/helix/0.6.4/binaries/

* sources:
https://dist.apache.org/repos/dist/dev/helix/0.6.4/src/


The 0.6.4 release tag:
https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tag;h=refs/tags/helix-0.6.4


KEYS file available here:
https://dist.apache.org/repos/dist/dev/helix/KEYS


Please vote on the release. The vote will be open for at least 72 hours.


[+1] -- YES, release

[0] -- No opinion

[-1] -- NO, do not release


Thanks,

The Apache Helix Team 

[jira] [Created] (HELIX-508) Write a test for monitoring controller lifecycle

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-508:
---

 Summary: Write a test for monitoring controller lifecycle
 Key: HELIX-508
 URL: https://issues.apache.org/jira/browse/HELIX-508
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


It's important to make sure that controller lifecycles are sufficiently 
expressible so that controller nodes can do extra work when becoming leaders.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 25131: [HELIX-508] Write a test for monitoring controller lifecycle

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25131/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-508


Repository: helix-git


Description
---

commit 657335031f42939bedb22733c687d08133bc8d37
Author: Kanak Biscuitwala ka...@apache.org
Date:   Wed Aug 27 16:50:44 2014 -0700

[HELIX-508] Write a test for monitoring controller lifecycle

:00 100644 000... 661b133... A  
helix-core/src/test/java/org/apache/helix/integration/TestControllerChangeCallbacks.java


Diffs
-

  
helix-core/src/test/java/org/apache/helix/integration/TestControllerChangeCallbacks.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/25131/diff/


Testing
---


Thanks,

Kanak Biscuitwala

[jira] [Created] (HELIX-506) Ensure that tasks are not placed on targets with pending transitions

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-506:
---

 Summary: Ensure that tasks are not placed on targets with pending 
transitions
 Key: HELIX-506
 URL: https://issues.apache.org/jira/browse/HELIX-506
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


It may be possible that a task is scheduled/rescheduled somewhere that the 
target is in process of changing. The task rebalancer should hold off placing 
these.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 25068: [HELIX-506] Ensure that tasks are not placed on targets with pending states

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25068/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-506


Repository: helix-git


Description
---

commit f3be50e4b0e1ef4513fb04ece07009b9d7168fb2
Author: Kanak Biscuitwala ka...@apache.org
Date:   Tue Aug 26 11:02:10 2014 -0700

[HELIX-506] Ensure that tasks are not placed on targets with pending states

:100644 100644 4c4fee1... 53d2ee9... M  
helix-core/src/main/java/org/apache/helix/task/FixedTargetTaskRebalancer.java
:100644 100644 8903ae2... f4145c5... M  
helix-core/src/main/java/org/apache/helix/task/GenericTaskRebalancer.java


Diffs
-

  helix-core/src/main/java/org/apache/helix/task/FixedTargetTaskRebalancer.java 
4c4fee1 
  helix-core/src/main/java/org/apache/helix/task/GenericTaskRebalancer.java 
8903ae2 

Diff: https://reviews.apache.org/r/25068/diff/


Testing
---


Thanks,

Kanak Biscuitwala

[jira] [Created] (HELIX-507) Add an API for controller lifecycle callbacks

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-507:
---

 Summary: Add an API for controller lifecycle callbacks
 Key: HELIX-507
 URL: https://issues.apache.org/jira/browse/HELIX-507
 Project: Apache Helix
  Issue Type: New Feature
Reporter: Kanak Biscuitwala


It would be nice to be able to define an API similar to 
StatelessParticipantService where a controller can get callbacks when it 
becomes leader, goes to standby, etc.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

RE: Add performant IPC (Helix actors)

[Kanak Biscuitwala]

PR is available at https://github.com/apache/helix/pull/2 for anyone interested 
in commenting.



From: kana...@hotmail.com
To: u...@helix.apache.org; dev@helix.apache.org
Subject: RE: Add performant IPC (Helix actors)
Date: Tue, 26 Aug 2014 10:37:56 -0700




PR is probably the better way to go here. Greg and Sandeep can probably move 
much faster that way, and it's not much overhead for us.



Date: Tue, 26 Aug 2014 10:17:40 -0700
Subject: Re: Add performant IPC (Helix actors)
From: g.kish...@gmail.com
To: dev@helix.apache.org
CC: u...@helix.apache.org

+1, if there is no additional dependency. But I would like to go through the 
review process. I dont think we can use rb. 

Henry had couple of ideas
-- create a branch and give access to Greg/Sandeep on the branches. Apache 
allows contributors to work on branches. Not sure how to get this.

-- Greg can fork the apache helix github repo and submit a PR and we can 
comment on the PR. Apache has done some cool integration with github, Henry can 
provide more info on this.




On Tue, Aug 26, 2014 at 9:39 AM, Kanak Biscuitwala kana...@hotmail.com wrote:



Are folks OK with the current iteration of this being in the 0.7.1 beta? It 
would be in a separate module, so no additional dependencies are required, and 
it could be a good opportunity to get some more feedback.Date: Fri, 22 Aug 2014 
13:42:31 -0700


Subject: Re: Add performant IPC (Helix actors)

From: g.kish...@gmail.com

To: dev@helix.apache.org

CC: u...@helix.apache.org



Thanks Greg for explanation. I agree Akka is 2 or 3 layers above what Helix-ipc 
is trying to provide. Also there are some nuances in order guarantees with 
Akka. For control messages where reliability is a must and probably more 
important than latency, I think using ZK as the communication channel is the 
right solution. Without pushing state transition message through a consensus 
system like ZK, there will be lot of edge cases where ensuring correctness 
becomes impossible.






















On Fri, Aug 22, 2014 at 1:29 PM, Greg Brandt brandt.g...@gmail.com wrote:



Hey Henry,







So Akka actors actually give exactly the same messaging guarantees as the



current implementation of Helix IPC: at-most-once message delivery, and



ordering per sender-receiver pair. In Helix's case, the latter can be



thought of further as ordering per-partition or resource, but basically is



guaranteed by Netty's Channel implementation for sender-receiver pair (



http://stackoverflow.com/questions/9082101/do-messages-sent-via-nettys-channel-write-preserve-their-order-when-begin-sen






).







What Akka provides that this IPC layer doesn't is the application life



cycle management part. This is why I made the claim that Akka Actors are



analogous to Helix Participants. One writes callbacks for different message



types in the Actor API to manage various control messages in the same way



one writes callbacks in Helix to manage application state transitions.







But more importantly, using Akka instead of getting really down into the



transport layer might hide too much when we want to use it for



performance-critical things like data replication. Continuing with that



example, we likely want to use direct IO and minimize data copies in the



transport layer, and it seems like some of the niceness involved in Akka's



API would get in the way of that.







Something that may be really cool to do is write another Akka transport



implementation on top of this, so you could basically add partition/state



to the Akka API and not have to implement the state machines yourself (as



is shown in some Akka docs). In that scenario, users should probably be



unaware that they were using Helix and just use the same Akka API.







-Greg











On Fri, Aug 22, 2014 at 11:13 AM, Henry Saputra henry.sapu...@gmail.com



wrote:







 Hi Greg,







 Thanks for the reply. Sorry for late for the discussions.







 If the IPC semantic will be used only by Helix internally I agree that



 lower level transport like Netty should be the way to go.







 For control plane layer, such as communications between controller and



 participants, wouldn't Akka actors give benefits of already reliable



 message passing and monitoring without building from ground up?



 Akka actor system has many different routers and allow callback to



 caller to get async response (in this case return back response to



 controller).







 I just saw the similarity of what the new API looks like with what



 Akka actor APIs.







 Thanks,







 - Henry







 On Wed, Aug 20, 2014 at 5:46 PM, Greg Brandt brandt.g...@gmail.com



 wrote:



  Hey Henry,



 



  Initially, I thought the same thing. But after evaluating Akka Actors, I



  think they're a fundamentally different approach to developing



 distributed



  systems as opposed to simply a messaging layer implementation. Let me



  explain

[jira] [Commented] (HELIX-470) Add performant IPC (Helix actors)

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111479#comment-14111479
 ] 

Kanak Biscuitwala commented on HELIX-470:
-

You might need access. Kishore can get you that.

 Add performant IPC (Helix actors)
 -

 Key: HELIX-470
 URL: https://issues.apache.org/jira/browse/HELIX-470
 Project: Apache Helix
  Issue Type: Improvement
  Components: helix-core
Affects Versions: 0.7.1, 0.6.4
Reporter: Greg Brandt

 Helix is missing a high-performance way to exchange messages among resource 
 partitions, with a user-friendly API.
 Currently, the Helix messaging service relies on creating many nodes in 
 ZooKeeper, which can lead to ZooKeeper outages if messages are sent too 
 frequently.
 In order to avoid this, high-performance NIO-based {{HelixActors}} should be 
 implemented (in rough accordance with the actor model). {{HelixActors}} 
 exchange messages asynchronously without waiting for a response, and are 
 partition/state-addressable.
 The API would look something like this:
 {code}
 public interface HelixActorT {
 void send(Partition partition, String state, T message);
 void register(String resource, HelixActorCallbackT callback);
 }
 public interface HelixActorCallbackT {
 void onMessage(Partition partition, State state, T message);
 }
 {code}
 {{#send}} should likely support wildcards for partition number and state, or 
 its method signature might need to be massaged a little bit for more 
 flexibility. But that's the basic idea.
 Nothing is inferred about the format of the messages - the only metadata we 
 need to be able to interpret is (1) partition name and (2) state. The user 
 provides a codec to encode / decode messages, so it's nicer to implement 
 {{HelixActor#send}} and {{HelixActorCallback#onMessage}}.
 {code}
 public interface HelixActorMessageCodecT {
 byte[] encode(T message);
 T decode(byte[] message);
 }
 {code}
 Actors should support somewhere around 100k to 1M messages per second. The 
 Netty framework is a potential implementation candidate, but should be 
 thoroughly evaluated w.r.t. performance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24994: [HELIX-482] Support delayed task reassignment

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24994/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-482


Repository: helix-git


Description
---

commit b87d72f00f3367dc5f0b91d0556d25b310fadabf
Author: Kanak Biscuitwala ka...@apache.org
Date:   Fri Aug 22 13:46:07 2014 -0700

[HELIX-482] Support delayed task reassignment

:100644 100644 1dad5e4... 780db55... M  
helix-core/src/main/java/org/apache/helix/task/JobConfig.java
:100644 100644 f843834... 77885cd... M  
helix-core/src/main/java/org/apache/helix/task/JobContext.java
:100644 100644 b4b2937... 934db92... M  
helix-core/src/main/java/org/apache/helix/task/TaskRebalancer.java
:100644 100644 bc5350a... 32fd5ac... M  
helix-core/src/main/java/org/apache/helix/task/beans/JobBean.java
:100644 100644 9dda550... da6b08b... M  
helix-core/src/test/java/org/apache/helix/integration/task/TestIndependentTaskRebalancer.java


Diffs
-

  helix-core/src/main/java/org/apache/helix/task/JobConfig.java 1dad5e4 
  helix-core/src/main/java/org/apache/helix/task/JobContext.java f843834 
  helix-core/src/main/java/org/apache/helix/task/TaskRebalancer.java b4b2937 
  helix-core/src/main/java/org/apache/helix/task/beans/JobBean.java bc5350a 
  
helix-core/src/test/java/org/apache/helix/integration/task/TestIndependentTaskRebalancer.java
 9dda550 

Diff: https://reviews.apache.org/r/24994/diff/


Testing
---

mvn test + new test


Thanks,

Kanak Biscuitwala

[jira] [Created] (HELIX-504) Controller should avoid resetting watches on removed paths

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-504:
---

 Summary: Controller should avoid resetting watches on removed paths
 Key: HELIX-504
 URL: https://issues.apache.org/jira/browse/HELIX-504
 Project: Apache Helix
  Issue Type: Bug
Reporter: Kanak Biscuitwala


Right now, if a participant has a session change, the old current state will be 
removed, but the controller despite unregistering the listener, still reset the 
watch on the path deleted event. We should avoid resetting this path on delete 
and let the controller explicitly set watches/listeners on new sessions.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

RE: Add performant IPC (Helix actors)

[Kanak Biscuitwala]

Yes, that would probably be best. I think what Greg has is a reasonable v1 for 
helix-ipc, and (feel free to disagree with me) should be flexible enough as a 
base for most future things we want to build, including multi-ZK support.
Kanak

Date: Thu, 21 Aug 2014 14:07:35 -0700
Subject: Re: Add performant IPC (Helix actors)
From: g.kish...@gmail.com
To: u...@helix.apache.org
CC: dev@helix.apache.org

Should we move the ZK cross data center to a separate thread. This is an 
interesting problem and probably needs more detailed discussions.

What do you guys think.



On Wed, Aug 20, 2014 at 7:44 PM, Kanak Biscuitwala kana...@hotmail.com wrote:




Hey Vlad,
In theory, we should be able to plug in a resolver that can resolve ZKs in 
other datacenters. We already resolve different clusters within the same ZK. 
Here is the resolver interface:

https://github.com/brandtg/helix-actors/blob/master/helix-ipc/src/main/java/org/apache/helix/resolver/HelixResolver.java

Right now the ZK implementation of the resolver can return a different Helix 
manager for different clusters, and perhaps it can be adapted to also accept a 
different ZK address from a different DC. Then we can connect to ZK observers. 
However, the message scope probably needs an additional resolver-specific 
metadata field or something that we can use to store the ZK observer addresses 
(and therefore a corresponding field in a message).

Regarding ZK vs DNS, push updates via watchers is an advantage as you pointed 
out. Also since Helix persists state on ZK, you have access to the entire 
cluster metadata in the context of your requests. This can certainly be 
implemented in a DNS-like scheme, though. Perhaps others can chime in on the 
tradeoffs here.

Kanak

From: brandt.g...@gmail.com
Date: Wed, 20 Aug 2014 18:16:55 -0700
Subject: Re: Add performant IPC (Helix actors)

To: u...@helix.apache.org
CC: dev@helix.apache.org

Hey Vlad,

Correct, ZooKeeper would still be used for the address resolution component.
Cross-datacenter hasn't really been thought of yet, but you do make a good 
point in that we should address this early because it usually seems to be an 
afterthought. However, we are planning to allow cross-cluster messaging within 
the same ZooKeeper.



Something that might be interesting to do in the cross-datacenter use case is 
to replicate each datacenter's ZooKeeper logs to each other datacenter in order 
to maintain a read-only view of that ZooKeeper cluster. One could then resolve 
addresses with only connections to local ZooKeeper(s), though the replication 
delay would have to be accounted for during state transitions.



I've got a small project that does generic log change capture, and intend on 
doing ZooKeeper log replication next, actually - have MySQL binary log and HDFS 
namenode edit logs now (https://github.com/brandtg/switchboard). Just an idea.



-Greg

On Wed, Aug 20, 2014 at 6:00 PM, vlad...@gmail.com vlad...@gmail.com wrote:




Hi Greg,
Within Turn, a few of us were looking at Helix as a possible solution for 
service discovery. It seems that the new system would keep the address 
resolution of the old messaging system but replace ZooKeeper with a direct 
connection for high throughput. Incidentally, we have a similar system, so we 
are more interested in the addressing part.



One thing that came up within our discussion was the possibility of accessing 
services in other datacenters. Are there plans to offer inter-datacenter 
discovery capabilities? Would that be based on ZK-observers or some other 
import mechanism (someone would need to connect to the remote ZK, preferably 
not the client).



Also, especially in the across-datacenter case, there are clear parallels to 
using DNS for the service discovery (I think Netflix published a blog post 
where they detailed this as an option). What would we gain and lose by using ZK 
as the building block as opposed to DNS? One thing that comes to mind is that 
ZK-based service discovery in Helix can be configured as a push-from-ZK or a 
poll-by-client service, while DNS entries timeout and must always be polled 
again.




Regards,Vlad

On Wed, Aug 20, 2014 at 5:46 PM, Greg Brandt brandt.g...@gmail.com wrote:




Hey Henry,
Initially, I thought the same thing. But after evaluating Akka Actors, I think 
they're a fundamentally different approach to developing distributed systems as 
opposed to simply a messaging layer implementation. Let me explain...






The primary difference between Akka Actors and Helix is that the former prefers 
loose, decentralized control, whereas the latter prefers strong, centralized 
control. Akka applications are expected to manage state by themselves, whereas 
in Helix, that responsibility is delegated to the controller. And the actual 
user Akka Actor implementations are analogous to Helix Participant 
implementations.






So with that perspective, it would be a bit of a kludge to leverage Akka Actors 
for simple cluster-scope

[jira] [Created] (HELIX-505) Prevent creation of resource names with pipe character

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-505:
---

 Summary: Prevent creation of resource names with pipe character
 Key: HELIX-505
 URL: https://issues.apache.org/jira/browse/HELIX-505
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala


ConfigAccessor uses '|' as a separator when reading partition configs for a 
resource, so resource names with pipes cannot have their configs properly read.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[DISCUSS] Release Helix 0.6.4 and 0.7.1

[Kanak Biscuitwala]

It's been quite some time and we've made many changes since the last Helix 
release. Here is what we could provide with each release:
0.6.4- Bug fixes- Task framework- New metrics
0.7.1 (beta)- Performance improvements- API simplification- Task framework- 
YARN support- IPC
Any thoughts? Is there anything important you would like to see in one or both 
of these releases?
Kanak 

Review Request 24899: [HELIX-503] Bring ivy files in sync with dependencies

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24899/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-503


Repository: helix-git


Description
---

commit 7089354043099ba84c865706889014ee80d897ce
Author: Kanak Biscuitwala ka...@apache.org
Date:   Wed Aug 20 11:04:50 2014 -0700

[HELIX-503] Bring ivy files in sync with dependencies

:100644 100644 cb78677... 7133b1b... M  helix-core/helix-core-0.6.4-SNAPSHOT.ivy


Diffs
-

  helix-core/helix-core-0.6.4-SNAPSHOT.ivy cb78677 

Diff: https://reviews.apache.org/r/24899/diff/


Testing
---


Thanks,

Kanak Biscuitwala

Review Request 24906: [HELIX-502] Start containers with more memory by default

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24906/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-502


Repository: helix-git


Description
---

commit 363c049e3ad874269ca765d9407b4f6623a56e70
Author: Kanak Biscuitwala ka...@apache.org
Date:   Wed Aug 20 13:01:11 2014 -0700

[HELIX-502] Start containers with more memory by default

:100644 100644 f3edb62... 7f49194... M  
helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java
:100644 100644 2be7062... 891013b... M  
helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/AppLauncher.java
:100644 100644 5fba08f... 6458518... M  
helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/AppStatusReportGenerator.java
:100644 100644 ad2f5f4... 4266688... M  
helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/YarnProvisioner.java


Diffs
-

  helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java f3edb62 
  
helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/AppLauncher.java
 2be7062 
  
helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/AppStatusReportGenerator.java
 5fba08f 
  
helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/YarnProvisioner.java
 ad2f5f4 

Diff: https://reviews.apache.org/r/24906/diff/


Testing
---


Thanks,

Kanak Biscuitwala

RE: Add performant IPC (Helix actors)

[Kanak Biscuitwala]

Hey Vlad,
In theory, we should be able to plug in a resolver that can resolve ZKs in 
other datacenters. We already resolve different clusters within the same ZK. 
Here is the resolver interface:
https://github.com/brandtg/helix-actors/blob/master/helix-ipc/src/main/java/org/apache/helix/resolver/HelixResolver.java
Right now the ZK implementation of the resolver can return a different Helix 
manager for different clusters, and perhaps it can be adapted to also accept a 
different ZK address from a different DC. Then we can connect to ZK observers. 
However, the message scope probably needs an additional resolver-specific 
metadata field or something that we can use to store the ZK observer addresses 
(and therefore a corresponding field in a message).
Regarding ZK vs DNS, push updates via watchers is an advantage as you pointed 
out. Also since Helix persists state on ZK, you have access to the entire 
cluster metadata in the context of your requests. This can certainly be 
implemented in a DNS-like scheme, though. Perhaps others can chime in on the 
tradeoffs here.
Kanak

From: brandt.g...@gmail.com
Date: Wed, 20 Aug 2014 18:16:55 -0700
Subject: Re: Add performant IPC (Helix actors)
To: u...@helix.apache.org
CC: dev@helix.apache.org

Hey Vlad,
Correct, ZooKeeper would still be used for the address resolution component.
Cross-datacenter hasn't really been thought of yet, but you do make a good 
point in that we should address this early because it usually seems to be an 
afterthought. However, we are planning to allow cross-cluster messaging within 
the same ZooKeeper.


Something that might be interesting to do in the cross-datacenter use case is 
to replicate each datacenter's ZooKeeper logs to each other datacenter in order 
to maintain a read-only view of that ZooKeeper cluster. One could then resolve 
addresses with only connections to local ZooKeeper(s), though the replication 
delay would have to be accounted for during state transitions.


I've got a small project that does generic log change capture, and intend on 
doing ZooKeeper log replication next, actually - have MySQL binary log and HDFS 
namenode edit logs now (https://github.com/brandtg/switchboard). Just an idea.


-Greg

On Wed, Aug 20, 2014 at 6:00 PM, vlad...@gmail.com vlad...@gmail.com wrote:



Hi Greg,
Within Turn, a few of us were looking at Helix as a possible solution for 
service discovery. It seems that the new system would keep the address 
resolution of the old messaging system but replace ZooKeeper with a direct 
connection for high throughput. Incidentally, we have a similar system, so we 
are more interested in the addressing part.


One thing that came up within our discussion was the possibility of accessing 
services in other datacenters. Are there plans to offer inter-datacenter 
discovery capabilities? Would that be based on ZK-observers or some other 
import mechanism (someone would need to connect to the remote ZK, preferably 
not the client).


Also, especially in the across-datacenter case, there are clear parallels to 
using DNS for the service discovery (I think Netflix published a blog post 
where they detailed this as an option). What would we gain and lose by using ZK 
as the building block as opposed to DNS? One thing that comes to mind is that 
ZK-based service discovery in Helix can be configured as a push-from-ZK or a 
poll-by-client service, while DNS entries timeout and must always be polled 
again.



Regards,Vlad

On Wed, Aug 20, 2014 at 5:46 PM, Greg Brandt brandt.g...@gmail.com wrote:



Hey Henry,
Initially, I thought the same thing. But after evaluating Akka Actors, I think 
they're a fundamentally different approach to developing distributed systems as 
opposed to simply a messaging layer implementation. Let me explain...





The primary difference between Akka Actors and Helix is that the former prefers 
loose, decentralized control, whereas the latter prefers strong, centralized 
control. Akka applications are expected to manage state by themselves, whereas 
in Helix, that responsibility is delegated to the controller. And the actual 
user Akka Actor implementations are analogous to Helix Participant 
implementations.





So with that perspective, it would be a bit of a kludge to leverage Akka Actors 
for simple cluster-scope-addressable message passing. Actually, Helix already 
provides this messaging feature, but it is done via ZooKeeper which is 
dangerous and less-than-performant for a high volume of messages.





The right thing to do seems to be to provide a set of better transport 
implemenations (e.g. Netty, ActiveMQ, Kafka, etc.) for the 
cluster-scope-addressable messaging system, and better preserve the fundamental 
concepts in Helix.





Let me know what you think.
-Greg



On Wed, Aug 20, 2014 at 9:55 AM, Henry Saputra henry.sapu...@gmail.com wrote:


This seems fitting for Akka actor system [1]. Maybe we could just use

it as base?





- Henry



[1] http://akka.io



On 

Review Request 24870: [HELIX-501] Skip website module in build by default

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24870/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-501


Repository: helix-git


Description
---

commit 010fe5c513884ef7299ed0257ffec61d46c7b90f
Author: Kanak Biscuitwala ka...@apache.org
Date:   Tue Aug 19 15:21:53 2014 -0700

[HELIX-501] Skip website module in build by default

:100644 100644 da5d476... 4bd6f2e... M  pom.xml


Diffs
-

  pom.xml da5d476 

Diff: https://reviews.apache.org/r/24870/diff/


Testing
---

Still builds correctly.


Thanks,

Kanak Biscuitwala

Re: Review Request 24807: [HELIX-329] HelixAutoController isn't a very clear name, consider renaming

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24807/#review50914
---

Ship it!



helix-core/src/main/java/org/apache/helix/participant/MultiClusterControllerStateModel.java
https://reviews.apache.org/r/24807/#comment88793

Rename to MultiClusterControllerTransitionHandler



helix-core/src/main/java/org/apache/helix/participant/MultiClusterControllerStateModelFactory.java
https://reviews.apache.org/r/24807/#comment88795

Rename to MultiClusterControllerTransitionHandlerFactory


- Kanak Biscuitwala


On Aug. 18, 2014, 11:07 a.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24807/
 ---
 
 (Updated Aug. 18, 2014, 11:07 a.m.)
 
 
 Review request for helix, Kanak Biscuitwala and Kishore Gopalakrishna.
 
 
 Bugs: HELIX-329
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-329] HelixAutoController isn't a very clear name, consider renaming
 
 
 Diffs
 -
 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestHelixAdminScenariosRest.java
  5aa26d2 
   helix-core/src/main/java/org/apache/helix/HelixConnection.java 4b7487b 
   helix-core/src/main/java/org/apache/helix/HelixManager.java 9e2bd14 
   
 helix-core/src/main/java/org/apache/helix/controller/HelixControllerMain.java 
 ca540c5 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java 
 12a3fe6 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixConnection.java 
 0a9dc94 
   
 helix-core/src/main/java/org/apache/helix/participant/DistClusterControllerStateModel.java
  4579810 
   
 helix-core/src/main/java/org/apache/helix/participant/DistClusterControllerStateModelFactory.java
  5f21e90 
   
 helix-core/src/main/java/org/apache/helix/participant/MultiClusterControllerStateModel.java
  PRE-CREATION 
   
 helix-core/src/main/java/org/apache/helix/participant/MultiClusterControllerStateModelFactory.java
  PRE-CREATION 
   helix-core/src/test/java/org/apache/helix/integration/TestAddClusterV2.java 
 5d7df59 
   
 helix-core/src/test/java/org/apache/helix/integration/TestAddNodeAfterControllerStart.java
  97acc1c 
   
 helix-core/src/test/java/org/apache/helix/integration/TestDistributedCMMain.java
  cf32d73 
   
 helix-core/src/test/java/org/apache/helix/integration/TestMultiClusterController.java
  8945843 
   
 helix-core/src/test/java/org/apache/helix/integration/TestReelectedPipelineCorrectness.java
  e522008 
   
 helix-core/src/test/java/org/apache/helix/integration/manager/TestConsecutiveZkSessionExpiry.java
  1b55af5 
   
 helix-core/src/test/java/org/apache/helix/integration/manager/TestHelixMultiClusterController.java
  51e7e19 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/MockMultiClusterController.java
  f68444c 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZkHelixAutoController.java
  a34dc66 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/TestZkHelixMultiClusterController.java
  PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/monitoring/TestClusterStatusMonitorLifecycle.java
  2b8d5e5 
   
 helix-core/src/test/java/org/apache/helix/participant/TestDistControllerStateModel.java
  116cd72 
   
 helix-core/src/test/java/org/apache/helix/participant/TestDistControllerStateModelFactory.java
  7055500 
   
 helix-core/src/test/java/org/apache/helix/participant/TestMultiClusterControllerStateModel.java
  PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/participant/TestMultiClusterControllerStateModelFactory.java
  PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/24807/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Zhen Zhang
 

Review Request 24811: [HELIX-499] Controller should listen for all config changes

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24811/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-499


Repository: helix-git


Description
---

commit 173f207dc452a3f00d7e6c3ff78176579be06d8d
Author: Kanak Biscuitwala ka...@apache.org
Date:   Mon Aug 18 12:08:44 2014 -0700

[HELIX-499] Controller should listen for all config changes

:100644 100644 421ff60... 2b59b27... M  
helix-core/src/main/java/org/apache/helix/api/Cluster.java
:100644 100644 ddf809a... eac2bf8... M  
helix-core/src/main/java/org/apache/helix/api/accessor/ClusterAccessor.java
:100644 100644 f1c2583... d36b6f5... M  
helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
:100644 100644 ca540c5... b652f35... M  
helix-core/src/main/java/org/apache/helix/controller/HelixControllerMain.java
:100644 100644 877baf2... 53ddd19... M  
helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
:100644 100644 623b874... e2def60... M  
helix-core/src/main/java/org/apache/helix/manager/zk/ControllerManagerHelper.java
:100644 100644 ee420b9... b953e0b... M  
helix-core/src/main/java/org/apache/helix/manager/zk/ZkCallbackHandler.java
:100644 100644 0a9dc94... 38332c5... M  
helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixConnection.java
:100644 100644 0698945... 295b69c... M  
helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java
:100644 100644 8f861cd... af89944... M  
helix-core/src/main/java/org/apache/helix/task/TaskRebalancer.java
:100644 100644 17b610d... cca7d76... M  
helix-core/src/main/java/org/apache/helix/task/TaskUtil.java
:100644 100644 9e8fd85... 190f739... M  
helix-core/src/test/java/org/apache/helix/integration/TestZkCallbackHandlerLeak.java
:100644 00 1393231... 000... D  
helix-core/src/test/java/org/apache/helix/integration/manager/TestZkCallbackHandlerLeak.java


Diffs
-

  helix-core/src/main/java/org/apache/helix/api/Cluster.java 421ff60 
  helix-core/src/main/java/org/apache/helix/api/accessor/ClusterAccessor.java 
ddf809a 
  
helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
 f1c2583 
  helix-core/src/main/java/org/apache/helix/controller/HelixControllerMain.java 
ca540c5 
  
helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
 877baf2 
  
helix-core/src/main/java/org/apache/helix/manager/zk/ControllerManagerHelper.java
 623b874 
  helix-core/src/main/java/org/apache/helix/manager/zk/ZkCallbackHandler.java 
ee420b9 
  helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixConnection.java 
0a9dc94 
  helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java 
0698945 
  helix-core/src/main/java/org/apache/helix/task/TaskRebalancer.java 8f861cd 
  helix-core/src/main/java/org/apache/helix/task/TaskUtil.java 17b610d 
  
helix-core/src/test/java/org/apache/helix/integration/TestZkCallbackHandlerLeak.java
 9e8fd85 
  
helix-core/src/test/java/org/apache/helix/integration/manager/TestZkCallbackHandlerLeak.java
 1393231 

Diff: https://reviews.apache.org/r/24811/diff/


Testing
---

mvn test


Thanks,

Kanak Biscuitwala

Re: Review Request 24811: [HELIX-499] Controller should listen for all config changes

[Kanak Biscuitwala]

 On Aug. 18, 2014, 12:20 p.m., Kishore Gopalakrishna wrote:
  why did we remove _cache and testzkcallbacklead

_cache was added, not removed.

The test was removed because it was code duplication with another test of the 
same name, different package.


 On Aug. 18, 2014, 12:20 p.m., Kishore Gopalakrishna wrote:
  helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java,
   line 221
  https://reviews.apache.org/r/24811/diff/1/?file=662677#file662677line221
 
  Can we create a enum/constant for event names. probably a separate jira

Created a separate jira.


- Kanak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24811/#review50924
---


On Aug. 18, 2014, 12:12 p.m., Kanak Biscuitwala wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24811/
 ---
 
 (Updated Aug. 18, 2014, 12:12 p.m.)
 
 
 Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
 
 
 Bugs: HELIX-499
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit 173f207dc452a3f00d7e6c3ff78176579be06d8d
 Author: Kanak Biscuitwala ka...@apache.org
 Date:   Mon Aug 18 12:08:44 2014 -0700
 
 [HELIX-499] Controller should listen for all config changes
 
 :100644 100644 421ff60... 2b59b27... M
 helix-core/src/main/java/org/apache/helix/api/Cluster.java
 :100644 100644 ddf809a... eac2bf8... M
 helix-core/src/main/java/org/apache/helix/api/accessor/ClusterAccessor.java
 :100644 100644 f1c2583... d36b6f5... M
 helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
 :100644 100644 ca540c5... b652f35... M
 helix-core/src/main/java/org/apache/helix/controller/HelixControllerMain.java
 :100644 100644 877baf2... 53ddd19... M
 helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
 :100644 100644 623b874... e2def60... M
 helix-core/src/main/java/org/apache/helix/manager/zk/ControllerManagerHelper.java
 :100644 100644 ee420b9... b953e0b... M
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkCallbackHandler.java
 :100644 100644 0a9dc94... 38332c5... M
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixConnection.java
 :100644 100644 0698945... 295b69c... M
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java
 :100644 100644 8f861cd... af89944... M
 helix-core/src/main/java/org/apache/helix/task/TaskRebalancer.java
 :100644 100644 17b610d... cca7d76... M
 helix-core/src/main/java/org/apache/helix/task/TaskUtil.java
 :100644 100644 9e8fd85... 190f739... M
 helix-core/src/test/java/org/apache/helix/integration/TestZkCallbackHandlerLeak.java
 :100644 00 1393231... 000... D
 helix-core/src/test/java/org/apache/helix/integration/manager/TestZkCallbackHandlerLeak.java
 
 
 Diffs
 -
 
   helix-core/src/main/java/org/apache/helix/api/Cluster.java 421ff60 
   helix-core/src/main/java/org/apache/helix/api/accessor/ClusterAccessor.java 
 ddf809a 
   
 helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
  f1c2583 
   
 helix-core/src/main/java/org/apache/helix/controller/HelixControllerMain.java 
 ca540c5 
   
 helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
  877baf2 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ControllerManagerHelper.java
  623b874 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkCallbackHandler.java 
 ee420b9 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixConnection.java 
 0a9dc94 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java 
 0698945 
   helix-core/src/main/java/org/apache/helix/task/TaskRebalancer.java 8f861cd 
   helix-core/src/main/java/org/apache/helix/task/TaskUtil.java 17b610d 
   
 helix-core/src/test/java/org/apache/helix/integration/TestZkCallbackHandlerLeak.java
  9e8fd85 
   
 helix-core/src/test/java/org/apache/helix/integration/manager/TestZkCallbackHandlerLeak.java
  1393231 
 
 Diff: https://reviews.apache.org/r/24811/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Kanak Biscuitwala
 

[jira] [Created] (HELIX-500) Controller event names should be constants/enums

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-500:
---

 Summary: Controller event names should be constants/enums
 Key: HELIX-500
 URL: https://issues.apache.org/jira/browse/HELIX-500
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala


Right now controller event names are hardcoded strings. These should be cleaned 
up.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

RE: [GSOC] Current Status

[Kanak Biscuitwala]

I don't have any specific documents in mind, but the build steps video will be 
very useful.


 Date: Fri, 15 Aug 2014 20:33:09 +0530
 Subject: Re: [GSOC] Current Status
 From: ayolajayamah...@gmail.com
 To: dev@helix.apache.org; kana...@hotmail.com

 Hi All,
 I will upload some videos on how to build helix dashboard. And write few
 blog posts. Are there any more documents to be written.
 Thanks.



 On 15 August 2014 06:13, Kanak Biscuitwala kana...@hotmail.com wrote:

 Thanks for making this. It is very useful for understanding and seeing the
 whole flow.

 Date: Thu, 14 Aug 2014 17:59:16 +0530
 Subject: Re: [GSOC] Current Status
 From: ayolajayamah...@gmail.com
 To: kana...@hotmail.com
 CC: dev@helix.apache.org

 Hi All,

 Herewith I'm sending a screencaste of the helix dashboard
 http://youtu.be/teP_Hb7oAgo.
 Thanks.


 On 11 August 2014 23:30, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi All,

 I have completed dashboard with solving issue that came across. Here is
 my
 repo https://github.com/AyolaJayamaha/helix

 Here I am attaching the document.

 ​

 https://drive.google.com/file/d/0ByWGexlJqMwbamFrdEFzQk9XUWM/edit?usp=sharing
 ​

 I'm planing to write few more blog posts on the task, I have written few
 blog posts can be found in here.
 http://ayolajayamaha.blogspot.com



 On 11 August 2014 09:22, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi All,

 I have completed most of the project and I'm commiting to my git repo.
 I
 will share the screen shots in a PDF document. All the problems of the
 project has been already successfully tackled.
 Thanks.



 On 2014 අගෝ 4, at පෙ.ව. 4.35, Ayola Jayamaha 
 ayolajayamah...@gmail.com
 wrote:

 Hi All,

 Helix Cluster Manager, Resources Manager and Instances Manager Tabs in
 the Dashboard are updated. Hope the Layout of the dashboard is alright
 with
 you. Shall upload the Screenshots soon.


 On 3 August 2014 06:38, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi Kishore,

 Thanks for your interest. Any questions or difficulties I 'd be glad
 to
 answer.

 Thanks



 On 2014 අගෝ 2, at ප.ව. 10.24, kishore g g.kish...@gmail.com wrote:

 Hi Ayola

 Thanks for the instructions. I will try out the instructions and give
 feedback.

 Thanks
 Kishore G
 On Aug 2, 2014 9:01 AM, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi All,

 Attached herewith is the Updated User Manual. You can try it with the
 git repo https://github.com/ayolajayamaha by getting a clone of
 helix
 and hawtio.

 Any questions don't hesitate to ask.

 Thanks


 On 1 August 2014 12:19, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi All,

 I was able to resolve the issue with deploying a plugin with Tomcat.

 The solution can be found in my blog post.


 http://ayolajayamaha.blogspot.com/2014/08/deploying-hawtio-plugin-in-apache-tomcat.html

 Thanks.


 On 31 July 2014 22:30, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi All,

 Attached herewith is the User Manual. However there is a small
 issue
 with deploying the plugin(.war file). I'm trying with tomcat,
 fabric8 and
 also communicating with the hawtio community. By next week this
 problem
 will surly be resolved. And I will write on blog the necessary.


 On 27 July 2014 07:03, Ayola Jayamaha ayolajayamah...@gmail.com
 wrote:

 Hi Kanak,

 Ok I will send the user manual to you.

 Sent from my iPhone

 On 2014 ජූලි 26, at පෙ.ව. 11.57, Kanak Biscuitwala 
 kana...@hotmail.com wrote:

 Hi Ayola,

 Could you write a Getting Started guide with step-by-step
 instructions on how to build and deploy the dashboard? If you
 already have
 that, could you send a link?

 Thanks,
 Kanak

 Date: Thu, 24 Jul 2014 22:49:32 +0530
 Subject: Re: [GSOC] Current Status
 From: ayolajayamah...@gmail.com
 To: dev@helix.apache.org; kana...@hotmail.com

 Hi All,

 I was able to resolve the Cross Domain Issue in Helix where it
 arises since
 the Rest API domain differs from the dashboard domain (ports).
 Currently it
 is resolved as to allow all requests from the API. I have
 committed to my
 git repo [1] in which Cluster API is resolved. I will clean the
 code and
 remove any duplicates and resolve the other APIs as well.
 More information will be found in my blog [2]. Also I'm
 following
 AngularJS, Angular Directives and Angular Services. These will
 be
 helpful
 to me to write good maintainable code for hawtio dashboard.

 [1]https://github.com/AyolaJayamaha/helix
 [2]

 http://ayolajayamaha.blogspot.com/2014/07/enable-cors-in-restlet-21.html



 On 23 July 2014 03:25, Kanak Biscuitwala kana...@hotmail.com
 wrote:

 In terms of the CORS pattern, I think it's just pattern
 matching
 the
 origin headers (i.e. Origin with Access-Control-Allow-Origin),
 so if you
 can make it configurable, that would be good.

 If it simplifies your implementation and resolves the XSS
 issues, you can
 co-deploy helix-admin-webapp with your dashboard (along with
 any
 components
 other than helix

Re: Review Request 24716: [HELIX-485] Remove controller leader election duplicate code

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24716/#review50645
---

Ship it!


Ship It!

- Kanak Biscuitwala


On Aug. 14, 2014, 2:14 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24716/
 ---
 
 (Updated Aug. 14, 2014, 2:14 p.m.)
 
 
 Review request for helix, Kanak Biscuitwala and Kishore Gopalakrishna.
 
 
 Bugs: 485
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 [HELIX-485] Remove controller leader election duplicate code
 
 
 Diffs
 -
 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/DistributedLeaderElection.java
  fe1ca4b 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkCallbackHandler.java 
 374a30b 
   helix-core/src/test/java/org/apache/helix/manager/zk/MockController.java 
 f4d2159 
   
 helix-core/src/test/java/org/apache/helix/manager/zk/MockMultiClusterController.java
  a39bd84 
   helix-core/src/test/java/org/apache/helix/manager/zk/MockParticipant.java 
 3936680 
   
 helix-core/src/test/java/org/apache/helix/participant/MockZKHelixManager.java 
 3efbffb 
   
 helix-core/src/test/java/org/apache/helix/participant/TestControllerElection.java
  PRE-CREATION 
   
 helix-core/src/test/java/org/apache/helix/participant/TestDistControllerElection.java
  de510c1 
 
 Diff: https://reviews.apache.org/r/24716/diff/
 
 
 Testing
 ---
 
 mvn test
 
 
 Thanks,
 
 Zhen Zhang
 

[jira] [Created] (HELIX-499) Helix controller should listen for resource config changes

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-499:
---

 Summary: Helix controller should listen for resource config changes
 Key: HELIX-499
 URL: https://issues.apache.org/jira/browse/HELIX-499
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


This may not be 100% backwards compatible, but it's the right thing to do, as 
right now people have to touch ideal states just to trigger a resource config 
change update. This is particularly relevant for the task framework, as we will 
be able to avoid reading the job/workflow configs repeatedly in the controller. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 24332: [HELIX-484] Remove CallbackHandler/ZkCallbackHandler code duplication, [HELIX-486] Remove StateModelFactory/HelixStateModelFactory code duplication

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24332/#review49943
---

Ship it!


Ship It!

- Kanak Biscuitwala


On Aug. 7, 2014, 10:50 a.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24332/
 ---
 
 (Updated Aug. 7, 2014, 10:50 a.m.)
 
 
 Review request for helix, Kanak Biscuitwala and Kishore Gopalakrishna.
 
 
 Bugs: HELIX-484 and HELIX-486
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 Remove CallbackHandler/ZkCallbackHandler code duplication
 Remove StateModelFactory/HelixStateModelFactory code duplication
 
 
 Diffs
 -
 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestResetInstance.java
  a9ecaa0 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestResetResource.java
  a54b0a3 
   helix-agent/src/main/java/org/apache/helix/agent/AgentStateModel.java 
 d227ac3 
   
 helix-agent/src/main/java/org/apache/helix/agent/AgentStateModelFactory.java 
 69d45ae 
   
 helix-core/src/main/java/org/apache/helix/api/StateTransitionHandlerFactory.java
  45f56e5 
   helix-core/src/main/java/org/apache/helix/api/TransitionHandler.java 
 9717340 
   helix-core/src/main/java/org/apache/helix/api/id/StateModelDefId.java 
 7c84f0f 
   
 helix-core/src/main/java/org/apache/helix/controller/HelixControllerMain.java 
 6aa3ab9 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixParticipant.java 
 af50eb7 
   
 helix-core/src/main/java/org/apache/helix/messaging/handling/HelixStateTransitionHandler.java
  1bb6506 
   
 helix-core/src/main/java/org/apache/helix/participant/CustomCodeInvoker.java 
 6c96629 
   
 helix-core/src/main/java/org/apache/helix/participant/DistClusterControllerStateModel.java
  0c2eb7c 
   
 helix-core/src/main/java/org/apache/helix/participant/DistClusterControllerStateModelFactory.java
  a367c81 
   
 helix-core/src/main/java/org/apache/helix/participant/GenericLeaderStandbyModel.java
  3866cf5 
   
 helix-core/src/main/java/org/apache/helix/participant/GenericLeaderStandbyStateModelFactory.java
  51c91cc 
   
 helix-core/src/main/java/org/apache/helix/participant/HelixCustomCodeRunner.java
  2f169cc 
   
 helix-core/src/main/java/org/apache/helix/participant/HelixStateMachineEngine.java
  95afb70 
   
 helix-core/src/main/java/org/apache/helix/participant/StateMachineEngine.java 
 abb7d81 
   
 helix-core/src/main/java/org/apache/helix/participant/statemachine/ScheduledTaskStateModel.java
  ca67d42 
   
 helix-core/src/main/java/org/apache/helix/participant/statemachine/ScheduledTaskStateModelFactory.java
  a205910 
   
 helix-core/src/main/java/org/apache/helix/participant/statemachine/StateModelParser.java
  eddeaa5 
   helix-core/src/main/java/org/apache/helix/task/TaskRunner.java 66abba6 
   helix-core/src/main/java/org/apache/helix/task/TaskStateModel.java a44a8cb 
   helix-core/src/main/java/org/apache/helix/task/TaskStateModelFactory.java 
 2537747 
   helix-core/src/main/java/org/apache/helix/tools/ClusterSetup.java 6599b33 
   helix-core/src/test/java/org/apache/helix/DummyProcessThread.java f51aa1d 
   helix-core/src/test/java/org/apache/helix/Mocks.java 0303f12 
   helix-core/src/test/java/org/apache/helix/TestHelixTaskExecutor.java 
 a3b16e5 
   helix-core/src/test/java/org/apache/helix/TestHelixTaskHandler.java 43b4407 
   helix-core/src/test/java/org/apache/helix/TestHelper.java 879e727 
   
 helix-core/src/test/java/org/apache/helix/integration/TestAddStateModelFactoryAfterConnect.java
  5f37845 
   
 helix-core/src/test/java/org/apache/helix/integration/TestBatchMessageWrapper.java
  6a6837a 
   
 helix-core/src/test/java/org/apache/helix/integration/TestCorrectnessOnConnectivityLoss.java
  abb2a7b 
   
 helix-core/src/test/java/org/apache/helix/integration/TestErrorPartition.java 
 19af9a7 
   
 helix-core/src/test/java/org/apache/helix/integration/TestHelixConnection.java
  3d02ae8 
   
 helix-core/src/test/java/org/apache/helix/integration/TestMessageThrottle2.java
  496a16f 
   
 helix-core/src/test/java/org/apache/helix/integration/TestMessagingService.java
  08954e5 
   
 helix-core/src/test/java/org/apache/helix/integration/TestMultiClusterController.java
  c2f9a5c 
   
 helix-core/src/test/java/org/apache/helix/integration/TestNonOfflineInitState.java
  105633a 
   
 helix-core/src/test/java/org/apache/helix/integration/TestPartitionLevelTransitionConstraint.java
  823a9ce 
   
 helix-core/src/test/java/org/apache/helix/integration/TestPreferenceListAsQueue.java
  06a2b56 
   
 helix-core/src/test/java/org/apache/helix/integration/TestResetInstance.java 
 5804744 
   
 helix-core/src/test/java/org/apache/helix/integration/TestResetPartitionState.java
  4855b3d 
   
 helix-core

Review Request 24473: [HELIX-498] Update configurations and static files to reflect TLP

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24473/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-498


Repository: helix-git


Description
---

commit ecfedff24185001363e0795a4ca461e65bd5ac03
Author: Kanak Biscuitwala ka...@apache.org
Date:   Thu Aug 7 14:48:21 2014 -0700

[HELIX-498] Update configurations and static files to reflect TLP

:100644 00 2001d31... 000... D  DISCLAIMER
:100644 00 a3af24b... 000... D  HelixProposal.mediawiki
:00 100644 000... ca4a5e0... A  NOTICE
:100644 00 2001d31... 000... D  helix-admin-webapp/DISCLAIMER
:100644 100644 e070e15... ca4a5e0... M  helix-admin-webapp/NOTICE
:00 100644 000... b0673af... A  
helix-admin-webapp/helix-admin-webapp-0.7.1-SNAPSHOT.ivy
:100644 00 35c6158... 000... D  
helix-admin-webapp/helix-admin-webapp-0.7.1-incubating-SNAPSHOT.ivy
:100644 100644 b4f38b5... 9f16e98... M  helix-admin-webapp/pom.xml
:100644 00 2001d31... 000... D  helix-agent/DISCLAIMER
:100644 100644 e070e15... 1ee0d24... M  helix-agent/NOTICE
:100644 100644 7d2a0ce... c2b697a... M  helix-agent/pom.xml
:100644 00 2001d31... 000... D  helix-core/DISCLAIMER
:100644 100644 e070e15... 1ee0d24... M  helix-core/NOTICE
:00 100644 000... de104a4... A  helix-core/helix-core-0.7.1-SNAPSHOT.ivy
:100644 00 eb37bed... 000... D  
helix-core/helix-core-0.7.1-incubating-SNAPSHOT.ivy
:100644 100644 8bb9ed1... 7c4e6eb... M  helix-core/pom.xml
:100644 00 2001d31... 000... D  helix-examples/DISCLAIMER
:100644 100644 e070e15... 1ee0d24... M  helix-examples/NOTICE
:100644 100644 c3b319c... 7fa3053... M  helix-examples/pom.xml
:100644 00 2001d31... 000... D  helix-provisioning/DISCLAIMER
:100644 100644 e070e15... 1ee0d24... M  helix-provisioning/NOTICE
:100644 100644 0c02919... f3fa05a... M  helix-provisioning/pom.xml
:100644 100644 dc5cc88... da5d476... M  pom.xml
:100644 100644 f3b2fbe... 4a8b99d... M  
recipes/distributed-lock-manager/README.md
:100644 100644 f9f6385... 9e563b8... M  recipes/distributed-lock-manager/pom.xml
:100644 100644 fe41e9c... 2e165bd... M  
recipes/helloworld-provisioning-yarn/pom.xml
:100755 100755 761c97e... f8679de... M  
recipes/helloworld-provisioning-yarn/src/main/resources/hello_world_app_spec.yaml
:100644 100644 c4d560f... 8f8c12f... M  recipes/jobrunner-yarn/pom.xml
:100755 100755 83c7edf... 6793ebe... M  
recipes/jobrunner-yarn/src/main/resources/job_runner_app_spec.yaml
:100644 100644 3fcaf42... 577acb3... M  recipes/pom.xml
:100644 100644 0eac7f4... dbb7c21... M  
recipes/rabbitmq-consumer-group/README.md
:100644 100644 9b749bb... ae9032a... M  recipes/rabbitmq-consumer-group/pom.xml
:100644 100644 f8a74a0... fc488f7... M  
recipes/rsync-replicated-file-system/README.md
:100644 100644 7d27f1f... 9e1864b... M  
recipes/rsync-replicated-file-system/pom.xml
:100644 100644 8e06ead... e7a52bd... M  recipes/service-discovery/README.md
:100644 100644 a876614... dd472de... M  recipes/service-discovery/pom.xml
:100644 100644 f0474e4... 14e6153... M  recipes/task-execution/README.md
:100644 100644 27464c9... a5130de... M  recipes/task-execution/pom.xml
:100644 100644 3dca51c... 82ca593... M  
recipes/user-defined-rebalancer/README.md
:100644 100644 8eba035... 56c1765... M  recipes/user-defined-rebalancer/pom.xml
:100644 100644 96529d1... c77f704... M  website/0.6.1-incubating/pom.xml
:100644 100644 785147b... 840c8c2... M  website/0.6.2-incubating/pom.xml
:100644 100644 2499562... 1a95ebc... M  website/0.6.3/pom.xml
:100644 100644 ae6a9f5... 2cd0753... M  website/0.7.0-incubating/pom.xml
:100644 100644 e74a377... 9b28d8f... M  website/pom.xml
:100644 100644 cdd2951... 2fb4f88... M  website/trunk/pom.xml


Diffs
-

  DISCLAIMER 2001d31 
  HelixProposal.mediawiki a3af24b 
  NOTICE PRE-CREATION 
  helix-admin-webapp/DISCLAIMER 2001d31 
  helix-admin-webapp/NOTICE e070e15 
  helix-admin-webapp/helix-admin-webapp-0.7.1-SNAPSHOT.ivy PRE-CREATION 
  helix-admin-webapp/helix-admin-webapp-0.7.1-incubating-SNAPSHOT.ivy 35c6158 
  helix-admin-webapp/pom.xml b4f38b5 
  helix-agent/DISCLAIMER 2001d31 
  helix-agent/NOTICE e070e15 
  helix-agent/pom.xml 7d2a0ce 
  helix-core/DISCLAIMER 2001d31 
  helix-core/NOTICE e070e15 
  helix-core/helix-core-0.7.1-SNAPSHOT.ivy PRE-CREATION 
  helix-core/helix-core-0.7.1-incubating-SNAPSHOT.ivy eb37bed 
  helix-core/pom.xml 8bb9ed1 
  helix-examples/DISCLAIMER 2001d31 
  helix-examples/NOTICE e070e15 
  helix-examples/pom.xml c3b319c 
  helix-provisioning/DISCLAIMER 2001d31 
  helix-provisioning/NOTICE e070e15 
  helix-provisioning/pom.xml 0c02919 
  pom.xml dc5cc88 
  recipes/distributed-lock-manager/README.md f3b2fbe 
  recipes/distributed-lock-manager/pom.xml f9f6385 
  recipes/helloworld-provisioning-yarn/pom.xml fe41e9c 
  
recipes/helloworld

Re: Review Request 24473: [HELIX-498] Update configurations and static files to reflect TLP

[Kanak Biscuitwala]

 On Aug. 7, 2014, 5:28 p.m., Kishore Gopalakrishna wrote:
  helix-admin-webapp/DISCLAIMER why did we remove these DISCLAIMER files.

They're only for incubator projects.


 On Aug. 7, 2014, 5:28 p.m., Kishore Gopalakrishna wrote:
  helix-admin-webapp/NOTICE, line 31
  https://reviews.apache.org/r/24473/diff/1/?file=655459#file655459line31
 
  did we remove dependency on zkclient

No, the diff is in the other direction; zkclient was added to the notice


 On Aug. 7, 2014, 5:28 p.m., Kishore Gopalakrishna wrote:
  recipes/jobrunner-yarn/src/main/resources/job_runner_app_spec.yaml, line 24
  https://reviews.apache.org/r/24473/diff/1/?file=655483#file655483line24
 
  location points your workspace kbiscuit? u can fix this with a separate 
  jira

The other recipe points to kgopalak... we need to make this pluggable somehow.


- Kanak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24473/#review49983
---


On Aug. 7, 2014, 2:50 p.m., Kanak Biscuitwala wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24473/
 ---
 
 (Updated Aug. 7, 2014, 2:50 p.m.)
 
 
 Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
 
 
 Bugs: HELIX-498
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit ecfedff24185001363e0795a4ca461e65bd5ac03
 Author: Kanak Biscuitwala ka...@apache.org
 Date:   Thu Aug 7 14:48:21 2014 -0700
 
 [HELIX-498] Update configurations and static files to reflect TLP
 
 :100644 00 2001d31... 000... DDISCLAIMER
 :100644 00 a3af24b... 000... DHelixProposal.mediawiki
 :00 100644 000... ca4a5e0... ANOTICE
 :100644 00 2001d31... 000... Dhelix-admin-webapp/DISCLAIMER
 :100644 100644 e070e15... ca4a5e0... Mhelix-admin-webapp/NOTICE
 :00 100644 000... b0673af... A
 helix-admin-webapp/helix-admin-webapp-0.7.1-SNAPSHOT.ivy
 :100644 00 35c6158... 000... D
 helix-admin-webapp/helix-admin-webapp-0.7.1-incubating-SNAPSHOT.ivy
 :100644 100644 b4f38b5... 9f16e98... Mhelix-admin-webapp/pom.xml
 :100644 00 2001d31... 000... Dhelix-agent/DISCLAIMER
 :100644 100644 e070e15... 1ee0d24... Mhelix-agent/NOTICE
 :100644 100644 7d2a0ce... c2b697a... Mhelix-agent/pom.xml
 :100644 00 2001d31... 000... Dhelix-core/DISCLAIMER
 :100644 100644 e070e15... 1ee0d24... Mhelix-core/NOTICE
 :00 100644 000... de104a4... A
 helix-core/helix-core-0.7.1-SNAPSHOT.ivy
 :100644 00 eb37bed... 000... D
 helix-core/helix-core-0.7.1-incubating-SNAPSHOT.ivy
 :100644 100644 8bb9ed1... 7c4e6eb... Mhelix-core/pom.xml
 :100644 00 2001d31... 000... Dhelix-examples/DISCLAIMER
 :100644 100644 e070e15... 1ee0d24... Mhelix-examples/NOTICE
 :100644 100644 c3b319c... 7fa3053... Mhelix-examples/pom.xml
 :100644 00 2001d31... 000... Dhelix-provisioning/DISCLAIMER
 :100644 100644 e070e15... 1ee0d24... Mhelix-provisioning/NOTICE
 :100644 100644 0c02919... f3fa05a... Mhelix-provisioning/pom.xml
 :100644 100644 dc5cc88... da5d476... Mpom.xml
 :100644 100644 f3b2fbe... 4a8b99d... M
 recipes/distributed-lock-manager/README.md
 :100644 100644 f9f6385... 9e563b8... M
 recipes/distributed-lock-manager/pom.xml
 :100644 100644 fe41e9c... 2e165bd... M
 recipes/helloworld-provisioning-yarn/pom.xml
 :100755 100755 761c97e... f8679de... M
 recipes/helloworld-provisioning-yarn/src/main/resources/hello_world_app_spec.yaml
 :100644 100644 c4d560f... 8f8c12f... Mrecipes/jobrunner-yarn/pom.xml
 :100755 100755 83c7edf... 6793ebe... M
 recipes/jobrunner-yarn/src/main/resources/job_runner_app_spec.yaml
 :100644 100644 3fcaf42... 577acb3... Mrecipes/pom.xml
 :100644 100644 0eac7f4... dbb7c21... M
 recipes/rabbitmq-consumer-group/README.md
 :100644 100644 9b749bb... ae9032a... M
 recipes/rabbitmq-consumer-group/pom.xml
 :100644 100644 f8a74a0... fc488f7... M
 recipes/rsync-replicated-file-system/README.md
 :100644 100644 7d27f1f... 9e1864b... M
 recipes/rsync-replicated-file-system/pom.xml
 :100644 100644 8e06ead... e7a52bd... M
 recipes/service-discovery/README.md
 :100644 100644 a876614... dd472de... M
 recipes/service-discovery/pom.xml
 :100644 100644 f0474e4... 14e6153... Mrecipes/task-execution/README.md
 :100644 100644 27464c9... a5130de... Mrecipes/task-execution/pom.xml
 :100644 100644 3dca51c... 82ca593... M
 recipes/user-defined-rebalancer/README.md
 :100644 100644 8eba035... 56c1765... M
 recipes/user-defined-rebalancer

[jira] [Created] (HELIX-497) Support named queues of jobs

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-497:
---

 Summary: Support named queues of jobs
 Key: HELIX-497
 URL: https://issues.apache.org/jira/browse/HELIX-497
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala


This is effectively a way to dynamically grow workflow DAGs. Workflow is a 
non-essential concept, and we should be able to support alternative 
organizations of jobs. Here, we just need a way to persist a queue, add work to 
it, and process that work. Then the cleanup should be at job level rather than 
workflow level. Some other things that are useful are returning the current 
queue, and (optionally) removing from the queue.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HELIX-342) Set info field in current-state only if it's not empty

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086525#comment-14086525
 ] 

Kanak Biscuitwala commented on HELIX-342:
-

This may not be a big deal because it's just an added simple field to a delta 
that is already updated with ZK on every state transition. So the added latency 
is just the CPU time of setting that simple field. We can leave this open, but 
if we actually want to do this, we would need to separate set to null from 
actually removing the info field.

 Set info field in current-state only if it's not empty
 --

 Key: HELIX-342
 URL: https://issues.apache.org/jira/browse/HELIX-342
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Zhen Zhang
Assignee: Zhen Zhang

 task framework uses an additional info field in the current-state. the 
 default value is an empty string. we may not set it if it is empty.
 In HelixStateTransitionHandler#postHandleMessage()
 _currentStateDelta.setInfo(partitionId, taskResult.getInfo());
 this is a perf concern instead of correctness



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24335: [HELIX-422] Simplify single job creation

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24335/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-422


Repository: helix-git


Description
---

commit 5bc77ac195e91292ceb2fb3584dce7a98175a623
Author: Kanak Biscuitwala ka...@apache.org
Date:   Tue Aug 5 13:10:52 2014 -0700

[HELIX-422] Simplify single job creation

This both augments the Java API to specify fully-built job configs, as
well as reduce the confusion between the job configs and the command
config maps they contain.

:100644 100644 3f9ab41... 1dad5e4... M  
helix-core/src/main/java/org/apache/helix/task/JobConfig.java
:100644 100644 1aa75d6... e1a96a8... M  
helix-core/src/main/java/org/apache/helix/task/TaskUtil.java
:100644 100644 84680d3... 77a5ba7... M  
helix-core/src/main/java/org/apache/helix/task/Workflow.java
:100644 100644 b5856b1... b7f20d1... M  
helix-core/src/test/java/org/apache/helix/integration/task/TestIndependentTaskRebalancer.java
:100644 100644 208480c... 2ce76c1... M  
helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancer.java
:100644 100644 97b8c7e... 4e0d92a... M  
helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancerStopResume.java
:100644 100644 921a5f9... 318673f... M  
helix-core/src/test/java/org/apache/helix/integration/task/WorkflowGenerator.java


Diffs
-

  helix-core/src/main/java/org/apache/helix/task/JobConfig.java 3f9ab41 
  helix-core/src/main/java/org/apache/helix/task/TaskUtil.java 1aa75d6 
  helix-core/src/main/java/org/apache/helix/task/Workflow.java 84680d3 
  
helix-core/src/test/java/org/apache/helix/integration/task/TestIndependentTaskRebalancer.java
 b5856b1 
  
helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancer.java
 208480c 
  
helix-core/src/test/java/org/apache/helix/integration/task/TestTaskRebalancerStopResume.java
 97b8c7e 
  
helix-core/src/test/java/org/apache/helix/integration/task/WorkflowGenerator.java
 921a5f9 

Diff: https://reviews.apache.org/r/24335/diff/


Testing
---


Thanks,

Kanak Biscuitwala

Review Request 24266: [HELIX-495] Make TestPreferenceListAsQueue non-flaky

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24266/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-495


Repository: helix-git


Description
---

commit 8ba16a8e26103ebddc04ab57cbac81b5226d6788
Author: Kanak Biscuitwala ka...@apache.org
Date:   Mon Aug 4 14:43:05 2014 -0700

[HELIX-495] Make TestPreferenceListAsQueue non-flaky

:100644 100644 06a2b56... f9c2d47... M  
helix-core/src/test/java/org/apache/helix/integration/TestPreferenceListAsQueue.java


Diffs
-

  
helix-core/src/test/java/org/apache/helix/integration/TestPreferenceListAsQueue.java
 06a2b56 

Diff: https://reviews.apache.org/r/24266/diff/


Testing
---


Thanks,

Kanak Biscuitwala

[jira] [Created] (HELIX-492) Task should be its own rebalance mode

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-492:
---

 Summary: Task should be its own rebalance mode
 Key: HELIX-492
 URL: https://issues.apache.org/jira/browse/HELIX-492
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


Right now the task framework creates a rebalancer with mode USER_DEFINED, where 
since it's built in, it's not really user-defined, but rather task-specific. 
There should be a new mode called TASK or similar.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 24193: [HELIX-492] Task should have its own rebalance mode

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24193/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-492


Repository: helix-git


Description
---

commit 2bdc82cf6a258dad3b364c5edab32f10238409ce
Author: Kanak Biscuitwala ka...@apache.org
Date:   Fri Aug 1 13:00:26 2014 -0700

[HELIX-492] Task should have its own rebalance mode

:100644 100644 706fd41... 9a9767e... M  
helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
:100644 100644 aa2d617... 89fdab7... M  
helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java
:100644 100644 a209cd9... 46a8f22... M  
helix-core/src/main/java/org/apache/helix/model/IdealState.java
:100644 100644 e4871b5... ab504cd... M  
helix-core/src/main/java/org/apache/helix/task/TaskDriver.java


Diffs
-

  
helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
 706fd41 
  helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java 
aa2d617 
  helix-core/src/main/java/org/apache/helix/model/IdealState.java a209cd9 
  helix-core/src/main/java/org/apache/helix/task/TaskDriver.java e4871b5 

Diff: https://reviews.apache.org/r/24193/diff/


Testing
---

Existing tests pass.


Thanks,

Kanak Biscuitwala

Re: Review Request 24190: [HELIX-376] Remove HelixConnection/HelixManager duplicate code

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24190/#review49377
---



helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestHelixAdminScenariosRest.java
https://reviews.apache.org/r/24190/#comment86387

MockMultiClusterController is better



helix-core/src/main/java/org/apache/helix/HelixAdministrator.java
https://reviews.apache.org/r/24190/#comment86388

Should this have getters/creators for HelixAdmin and ClusterAccessor?

Alternatively, maybe make a default HelixRole implementation for the 
administrator adaptor.



helix-core/src/main/java/org/apache/helix/HelixSpectator.java
https://reviews.apache.org/r/24190/#comment86389

This should either support adding listeners, or just use a default 
HelixRole implementation.



helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java
https://reviews.apache.org/r/24190/#comment86390

It's unsafe to call HelixConnection#connect in a constructor.



helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java
https://reviews.apache.org/r/24190/#comment86392

This may be a good opportunity to remove getConfigAccessor since we 
deprecated it in an older version.



helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java
https://reviews.apache.org/r/24190/#comment86393

Remove TODO



helix-core/src/main/java/org/apache/helix/messaging/handling/HelixTaskExecutor.java
https://reviews.apache.org/r/24190/#comment86396

log instead of printing, make message more descriptive



helix-core/src/test/java/org/apache/helix/integration/IntegrationTest.java
https://reviews.apache.org/r/24190/#comment86397

This class needs a more descriptive name.



recipes/jobrunner-yarn/src/main/java/org/apache/helix/provisioning/yarn/example/JobRunnerMain.java
https://reviews.apache.org/r/24190/#comment86402

Remove TODO


- Kanak Biscuitwala


On Aug. 1, 2014, 12:03 p.m., Zhen Zhang wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24190/
 ---
 
 (Updated Aug. 1, 2014, 12:03 p.m.)
 
 
 Review request for helix, Kanak Biscuitwala and Kishore Gopalakrishna.
 
 
 Bugs: HELIX-376
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 Remove HelixConnection/HelixManager duplicate code
 
 
 Diffs
 -
 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestHelixAdminScenariosRest.java
  66065c3 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestResetInstance.java
  b89a067 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestResetPartitionState.java
  8cd6f42 
   
 helix-admin-webapp/src/test/java/org/apache/helix/webapp/TestResetResource.java
  464edc4 
   helix-agent/src/test/java/org/apache/helix/agent/TestHelixAgent.java 
 cbf0582 
   helix-core/src/main/java/org/apache/helix/HelixAdministrator.java 
 PRE-CREATION 
   helix-core/src/main/java/org/apache/helix/HelixConnection.java ff5f458 
   helix-core/src/main/java/org/apache/helix/HelixRole.java ffcb700 
   helix-core/src/main/java/org/apache/helix/HelixService.java 40e9bae 
   helix-core/src/main/java/org/apache/helix/HelixSpectator.java PRE-CREATION 
   helix-core/src/main/java/org/apache/helix/api/id/AdministratorId.java 
 PRE-CREATION 
   
 helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
  6f34953 
   
 helix-core/src/main/java/org/apache/helix/controller/stages/TaskAssignmentStage.java
  9d6228e 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/HelixConnectionAdaptor.java
  ef17715 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java 
 f95f6ee 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkCallbackHandler.java 
 5961fe3 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixAdministrator.java
  PRE-CREATION 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixAutoController.java
  1d4b225 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixConnection.java 
 bec6f5c 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixController.java 
 f9529b7 
   
 helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixParticipant.java 
 d3ee8d1 
   helix-core/src/main/java/org/apache/helix/manager/zk/ZkHelixSpectator.java 
 PRE-CREATION 
   
 helix-core/src/main/java/org/apache/helix/messaging/handling/HelixTaskExecutor.java
  0ac3b58 
   helix-core/src/test/java/org/apache/helix/TestHelper.java 8328a15 
   helix-core/src/test/java/org/apache/helix/ZkTestHelper.java 444c069 
   helix-core/src/test/java/org/apache/helix/api/TestNewStages.java b5d218d 
   
 helix-core/src/test/java/org/apache/helix/controller/stages/TestRebalancePipeline.java

[jira] [Created] (HELIX-493) REST API should have a removeResourceProperty on IdealState

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-493:
---

 Summary: REST API should have a removeResourceProperty on 
IdealState
 Key: HELIX-493
 URL: https://issues.apache.org/jira/browse/HELIX-493
 Project: Apache Helix
  Issue Type: Bug
Reporter: Kanak Biscuitwala


The REST API has an addResourceProperty for adding tags and other things, but 
does not have the same for removal. This should be added.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 24121: [HELIX-374] Rebalancer config should be fully user-specified

[Kanak Biscuitwala]

 On July 30, 2014, 3:54 p.m., Kishore Gopalakrishna wrote:
  helix-core/src/main/java/org/apache/helix/model/builder/IdealStateBuilder.java,
   line 164
  https://reviews.apache.org/r/24121/diff/1/?file=646299#file646299line164
 
  can we always set the nodeGroup probably have a default name?

That would have to be a pretty broad change to stay compatible. Considering 
this is just a builder, I think checking for null as unset is acceptable.


 On July 30, 2014, 3:54 p.m., Kishore Gopalakrishna wrote:
  helix-core/src/test/java/org/apache/helix/api/TestNewStages.java, line 29
  https://reviews.apache.org/r/24121/diff/1/?file=646301#file646301line29
 
  Need to rename this class and move it to a different package

Will rename and move to where the other pipeline stage tests are.


- Kanak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24121/#review49176
---


On July 30, 2014, 2:40 p.m., Kanak Biscuitwala wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/24121/
 ---
 
 (Updated July 30, 2014, 2:40 p.m.)
 
 
 Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
 
 
 Bugs: HELIX-374
 
 
 Repository: helix-git
 
 
 Description
 ---
 
 commit 1caffd725814ffe479ade78727ff8ade6bdf54c2
 Author: Kanak Biscuitwala ka...@apache.org
 Date:   Wed Jul 30 14:37:02 2014 -0700
 
 [HELIX-374] Rebalancer config should be fully user-specified
 
 :100644 100644 6b92275... ddf809a... M
 helix-core/src/main/java/org/apache/helix/api/accessor/ClusterAccessor.java
 :100644 00 d6aa10e... 000... D
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/BasicRebalancerConfig.java
 :100644 00 a44b230... 000... D
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/CustomRebalancerConfig.java
 :100644 00 16bb4cb... 000... D
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/FullAutoRebalancerConfig.java
 :100644 00 934a9c2... 000... D
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/PartitionedRebalancerConfig.java
 :100644 100644 3f8c9d1... b725c9e... M
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/RebalancerConfig.java
 :100644 100644 d6ddb50... 19970c4... M
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/RebalancerConfigHolder.java
 :100644 00 3118b2a... 000... D
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/ReplicatedRebalancerConfig.java
 :100644 00 60e30f4... 000... D
 helix-core/src/main/java/org/apache/helix/controller/rebalancer/config/SemiAutoRebalancerConfig.java
 :100644 100644 6f34953... f6f5d61... M
 helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
 :100644 100644 deabb56... 810a675... M
 helix-core/src/main/java/org/apache/helix/controller/stages/ExternalViewComputeStage.java
 :100644 100644 61da673... 4ba249f... M
 helix-core/src/main/java/org/apache/helix/controller/stages/MessageGenerationStage.java
 :100644 100644 2408e29... 0e4a694... M
 helix-core/src/main/java/org/apache/helix/controller/stages/MessageSelectionStage.java
 :100644 100644 1036b35... ccd8ae6... M
 helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java
 :100644 100644 88ec610... b0347ef... M
 helix-core/src/main/java/org/apache/helix/model/IdealState.java
 :100644 100644 a6707e9... cc4c786... M
 helix-core/src/main/java/org/apache/helix/model/builder/IdealStateBuilder.java
 :100644 100644 8249b4c... 0d7b0de... M
 helix-core/src/main/java/org/apache/helix/monitoring/mbeans/ClusterStatusMonitor.java
 :100644 100644 b5d218d... 1809f1b... M
 helix-core/src/test/java/org/apache/helix/api/TestNewStages.java
 :100644 00 94deaac... 000... D
 helix-core/src/test/java/org/apache/helix/controller/rebalancer/context/TestSerializeRebalancerContext.java
 :100644 100644 5456884... 0992d2e... M
 helix-core/src/test/java/org/apache/helix/controller/stages/BaseStageTest.java
 :100644 100644 04a25e9... 130937c... M
 helix-core/src/test/java/org/apache/helix/controller/stages/TestResourceComputationStage.java
 :100644 100644 b03a48b... 43915c0... M
 helix-core/src/test/java/org/apache/helix/integration/TestCustomizedIdealStateRebalancer.java
 :100644 100644 3bb17a2... 3d02ae8... M
 helix-core/src/test/java/org/apache/helix/integration/TestHelixConnection.java
 :100644 100644 5ab5f22... bdde8d3... M
 helix-core/src/test/java/org/apache/helix

[jira] [Created] (HELIX-477) Some YARN container start requests fail

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-477:
---

 Summary: Some YARN container start requests fail
 Key: HELIX-477
 URL: https://issues.apache.org/jira/browse/HELIX-477
 Project: Apache Helix
  Issue Type: Bug
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


We need to improve logging when container start fails so that we can identify 
why it happens. We may also consider compensating for the failure to start by 
allocating an additional container.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-478) Validate robustness of Helix-YARN integration

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-478:
---

 Summary: Validate robustness of Helix-YARN integration
 Key: HELIX-478
 URL: https://issues.apache.org/jira/browse/HELIX-478
 Project: Apache Helix
  Issue Type: Task
Reporter: Kanak Biscuitwala


This involves running Helix on a real YARN cluster and evaluating the following:

- All basic provisioner methods: allocate, deallocate, start, stop
- Defining and finalizing an API for an app master and app launcher that Helix 
provide (and how they should be configured)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-481) Evaluate that TargetProvider basic logic holds up in YARN cluster

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-481:
---

 Summary: Evaluate that TargetProvider basic logic holds up in YARN 
cluster
 Key: HELIX-481
 URL: https://issues.apache.org/jira/browse/HELIX-481
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala


If containers fail, we should ensure that the target provider is still capable 
of doing the right thing and requesting for a replacement in the fixed target 
case.

We can test this via shutdown messages and the update container config tools 
that already exist.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-482) Support smarter task failure strategies

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-482:
---

 Summary: Support smarter task failure strategies
 Key: HELIX-482
 URL: https://issues.apache.org/jira/browse/HELIX-482
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala


It would be interesting to support exponential backoff on task failures, and 
even take containers away if the tasks are being backed off.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-485) Remove controller leader election duplicate code

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-485:
---

 Summary: Remove controller leader election duplicate code
 Key: HELIX-485
 URL: https://issues.apache.org/jira/browse/HELIX-485
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-486) Remove StateModelFactory/HelixStateModelFactory code duplication

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-486:
---

 Summary: Remove StateModelFactory/HelixStateModelFactory code 
duplication
 Key: HELIX-486
 URL: https://issues.apache.org/jira/browse/HELIX-486
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HELIX-488) Ensure high test coverage of new APIs

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-488:
---

 Summary: Ensure high test coverage of new APIs
 Key: HELIX-488
 URL: https://issues.apache.org/jira/browse/HELIX-488
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HELIX-389) Unify (Cluster|Resource|Participant)Accessor classes into a single Administrator class

[Kanak Biscuitwala (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070771#comment-14070771
 ] 

Kanak Biscuitwala commented on HELIX-389:
-

Yes the hope is that we can evolve this into a more java-friendly admin class.

 Unify (Cluster|Resource|Participant)Accessor classes into a single 
 Administrator class
 --

 Key: HELIX-389
 URL: https://issues.apache.org/jira/browse/HELIX-389
 Project: Apache Helix
  Issue Type: Sub-task
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala

 It's too confusing to have separate accessors.
 The work tracked by this issue is straightforward: methods that were in 
 ResourceAccessor and ParticipantAccessor will be moved into ClusterAccessor. 
 Some redundant methods may also be deleted. Future work includes updating the 
 configuration and snapshot classes (and associated accessor methods) to be 
 easier to use (and implement for us).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 23515: [HELIX-472] Errors should be cleaned up less frequently

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23515/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-472


Repository: helix-git


Description
---

commit ca3a5bdaefd1f1c5ae3df4c87b00b29300910dbe
Author: Kanak Biscuitwala ka...@apache.org
Date:   Tue Jul 15 11:33:43 2014 -0700

[HELIX-472] Errors should be cleaned up less frequently

:100644 100644 a0190d2... 2f5f773... M  
helix-core/src/main/java/org/apache/helix/monitoring/ZKPathDataDumpTask.java


Diffs
-

  helix-core/src/main/java/org/apache/helix/monitoring/ZKPathDataDumpTask.java 
a0190d2 

Diff: https://reviews.apache.org/r/23515/diff/


Testing
---


Thanks,

Kanak Biscuitwala

Re: Review Request 23515: [HELIX-472] Errors should be cleaned up less frequently

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23515/
---

(Updated July 15, 2014, 2:09 p.m.)


Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Changes
---

Refactored dump task to improve readability, allow separate arguments for 
status update threshold, error threshold, and max leaf node threshold. Added a 
test to verify max leaf node threshold works.


Bugs: HELIX-472


Repository: helix-git


Description
---

commit ca3a5bdaefd1f1c5ae3df4c87b00b29300910dbe
Author: Kanak Biscuitwala ka...@apache.org
Date:   Tue Jul 15 11:33:43 2014 -0700

[HELIX-472] Errors should be cleaned up less frequently

:100644 100644 a0190d2... 2f5f773... M  
helix-core/src/main/java/org/apache/helix/monitoring/ZKPathDataDumpTask.java


Diffs (updated)
-

  helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java 
d92cc7d 
  helix-core/src/main/java/org/apache/helix/monitoring/ZKPathDataDumpTask.java 
a0190d2 
  
helix-core/src/test/java/org/apache/helix/integration/TestSchedulerMessage.java 
281b306 
  
helix-core/src/test/java/org/apache/helix/monitoring/TestZKPathDataDumpTask.java
 a3d8ae3 

Diff: https://reviews.apache.org/r/23515/diff/


Testing
---


Thanks,

Kanak Biscuitwala

[jira] [Created] (HELIX-473) TestLocalContainerProvider is slow and flaky

[Kanak Biscuitwala (JIRA)]

Kanak Biscuitwala created HELIX-473:
---

 Summary: TestLocalContainerProvider is slow and flaky
 Key: HELIX-473
 URL: https://issues.apache.org/jira/browse/HELIX-473
 Project: Apache Helix
  Issue Type: Bug
Reporter: Kanak Biscuitwala
Assignee: Kanak Biscuitwala


This test starts and stops 10 participants, and the wait threshold isn't long 
enough. Realistically, we don't even need to have that many participants to 
verify correctness.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 23526: [HELIX-473] Make TestLocalContainerProvider less flaky

[Kanak Biscuitwala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23526/
---

Review request for helix, Zhen Zhang and Kishore Gopalakrishna.


Bugs: HELIX-473


Repository: helix-git


Description
---

commit 3c9697c4b8fa7748a8ffe2d798bc3effd72f0740
Author: Kanak Biscuitwala ka...@apache.org
Date:   Tue Jul 15 14:15:23 2014 -0700

[HELIX-473] Make TestLocalContainerProvider less flaky

:100644 100644 f27ce79... 111c921... M  
helix-core/src/test/java/org/apache/helix/integration/TestLocalContainerProvider.java


Diffs
-

  
helix-core/src/test/java/org/apache/helix/integration/TestLocalContainerProvider.java
 f27ce79 

Diff: https://reviews.apache.org/r/23526/diff/


Testing
---


Thanks,

Kanak Biscuitwala