[GitHub] helix pull request #82: Add Test for Batch Message ThreadPool

[dasahcc]

GitHub user dasahcc opened a pull request:

https://github.com/apache/helix/pull/82

Add Test for Batch Message ThreadPool

Add Test for Batch Message ThreadPool. For easy testing, change the private 
type to default type.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dasahcc/helix helix-0.6.x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/helix/pull/82.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #82


commit cb806303f7bc5203451ed583625dbb68cc6569b8
Author: Junkai Xue 
Date:   2017-04-04T00:25:33Z

Add Test for Batch Message ThreadPool




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HELIX-640) Support job level expiration.

[Weihan Kong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HELIX-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954192#comment-15954192
 ] 

Weihan Kong commented on HELIX-640:
---

[~dasahcc] Could you assign this ticket to me? I'll try to merge it. Thanks!

> Support job level expiration.
> -
>
> Key: HELIX-640
> URL: https://issues.apache.org/jira/browse/HELIX-640
> Project: Apache Helix
>  Issue Type: New Feature
>  Components: helix-core
>Affects Versions: 0.6.6
>Reporter: Junkai Xue
>Assignee: Junkai Xue
>
> If job queue not being killed, then the jobs in the final stage (COMPLETED, 
> FAILED) will not be cleaned. As the temporary solution, we provided an API to 
> clean up job queue.
> So this ticket will track the feature support for controller clean up the 
> finished jobs with expiry set up.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [ANNOUNCE] New committer: Junkai Xue

[Xue Junkai]

Thanks! Very glad to join the team!

Best,

Junkai

On Mon, Apr 3, 2017 at 11:01 AM, kishore g  wrote:

> Yay!.  Welcome to the club.
>
> On Mon, Apr 3, 2017 at 10:45 AM, Lei Xia  wrote:
>
> > The Project Management Committee (PMC) for Apache Helix has asked Junkai
> > Xue
> > to become a committer and we are pleased to announce that he has
> accepted.
> >
> > Being a committer enables easier contribution to the project since there
> is
> > no need to go via the patch submission process. This should enable better
> > productivity.
> >
> >
> > Helix Team
> >
>



-- 
Junkai Xue

Re: [GitHub] helix issue #81: Creating a separate threadpool to handle batchMessages

[Xue Junkai]

Sure! Will add that.

On Mon, Apr 3, 2017 at 12:21 PM, kishore g  wrote:

> +1 on adding an api to enable/disable this at a cluster level.
>
> On Mon, Apr 3, 2017 at 12:18 PM, dasahcc  wrote:
>
> > Github user dasahcc commented on the issue:
> >
> > https://github.com/apache/helix/pull/81
> >
> > Looks good to me! I will do the following things for corresponding
> > change:
> > 1. Will add a test for this.
> > 2. Will provide an API in HelixManager for enabling batch message and
> > support cluster/resource level batch message enabling.
> >
> >
> > ---
> > If your project is set up for it, you can reply to this email and have
> your
> > reply appear on GitHub as well. If your project does not have this
> feature
> > enabled and wishes so, or if the feature is enabled but not working,
> please
> > contact infrastructure at infrastruct...@apache.org or file a JIRA
> ticket
> > with INFRA.
> > ---
> >
>



-- 
Junkai Xue

[GitHub] helix pull request #81: Creating a separate threadpool to handle batchMessag...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/helix/pull/81


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [GitHub] helix issue #81: Creating a separate threadpool to handle batchMessages

[kishore g]

+1 on adding an api to enable/disable this at a cluster level.

On Mon, Apr 3, 2017 at 12:18 PM, dasahcc  wrote:

> Github user dasahcc commented on the issue:
>
> https://github.com/apache/helix/pull/81
>
> Looks good to me! I will do the following things for corresponding
> change:
> 1. Will add a test for this.
> 2. Will provide an API in HelixManager for enabling batch message and
> support cluster/resource level batch message enabling.
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---
>

[GitHub] helix issue #81: Creating a separate threadpool to handle batchMessages

[dasahcc]

Github user dasahcc commented on the issue:

https://github.com/apache/helix/pull/81
  
Looks good to me! I will do the following things for corresponding change:
1. Will add a test for this.
2. Will provide an API in HelixManager for enabling batch message and 
support cluster/resource level batch message enabling.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [ANNOUNCE] New committer: Junkai Xue

[kishore g]

Yay!.  Welcome to the club.

On Mon, Apr 3, 2017 at 10:45 AM, Lei Xia  wrote:

> The Project Management Committee (PMC) for Apache Helix has asked Junkai
> Xue
> to become a committer and we are pleased to announce that he has accepted.
>
> Being a committer enables easier contribution to the project since there is
> no need to go via the patch submission process. This should enable better
> productivity.
>
>
> Helix Team
>

[ANNOUNCE] New committer: Junkai Xue

[Lei Xia]

The Project Management Committee (PMC) for Apache Helix has asked Junkai Xue
to become a committer and we are pleased to announce that he has accepted.

Being a committer enables easier contribution to the project since there is
no need to go via the patch submission process. This should enable better
productivity.


Helix Team

helix - Build # 1351 - Failure

[Apache Jenkins Server]

The Apache Jenkins build system has built helix (build #1351)

Status: Failure

Check console output at https://builds.apache.org/job/helix/1351/ to view the 
results.

[GitHub] helix pull request #81: Creating a separate threadpool to handle batchMessag...

[kishoreg]

GitHub user kishoreg opened a pull request:

https://github.com/apache/helix/pull/81

Creating a separate threadpool to handle batchMessages

batchMessages were using the same threadpool as sub tasks. This works as 
long the thread pool size is greater than the number of messages with 
batchMessageMode=true. If not, there is a deadlock. Consider the following 
scenario -

- 100 messages with batchMessage=true come in, each with 10 submessages
- let's say threadpool size is 40 (this is current default)
- 40 message will get scheduled first. Once the execution of the thread 
starts, each thread now tries to execute the 10 sub-messages but in the current 
code we use the same threadpool to execute batch message and sub message. This 
means the sub-messages never get scheduled and everything gets blocked.

The solution was to use a separate threadpool for state transition messages 
here batchMessage=true.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kishoreg/helix batch-message-bug-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/helix/pull/81.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #81


commit 70a585aca1302aff767a91c59040ad9c94439323
Author: kishoreg 
Date:   2017-04-03T07:10:20Z

Creating a separate threadpool to handle batchMessages




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---