[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695392#comment-16695392
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12777600/ZOOKEEPER-2260.patch
  against trunk revision 1507f67a06175155003722297daeb60bc912af1d.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3717//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>Priority: Major
>  Labels: api, features
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Michael K. Edwards (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695381#comment-16695381
 ] 

Michael K. Edwards commented on ZOOKEEPER-2260:
---

This is a high-value feature.  Can it be revived for the 3.5.5 release?

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>Priority: Major
>  Labels: api, features
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471056#comment-16471056
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12777600/ZOOKEEPER-2260.patch
  against trunk revision 2fa315b7d0ed65828479fcdcc9e76ca8552fba4a.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3669//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>Priority: Major
>  Labels: api, features
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Dan Benediktson]

What's happening with this PR? We're pretty interested in the feature and
would love to see it adopted into the OSS tree, and it looks like it has
already had review feedback addressed and been signed off by Jenkins.

On Tue, Apr 11, 2017 at 2:16 PM, Hadoop QA (JIRA)  wrote:

>
> [ https://issues.apache.org/jira/browse/ZOOKEEPER-2260?
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&
> focusedCommentId=15964970#comment-15964970 ]
>
> Hadoop QA commented on ZOOKEEPER-2260:
> --
>
> +1 overall.  GitHub Pull Request  Build
>
>
> +1 @author.  The patch does not contain any @author tags.
>
> +1 tests included.  The patch appears to include 15 new or modified
> tests.
>
> +1 javadoc.  The javadoc tool did not generate any warning messages.
>
> +1 javac.  The applied patch does not increase the total number of
> javac compiler warnings.
>
> +1 findbugs.  The patch does not introduce any new Findbugs (version
> 3.0.1) warnings.
>
> +1 release audit.  The applied patch does not increase the total
> number of release audit warnings.
>
> +1 core tests.  The patch passed core unit tests.
>
> +1 contrib tests.  The patch passed contrib unit tests.
>
> Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-
> build/535//testReport/
> Findbugs warnings: https://builds.apache.org/job/
> PreCommit-ZOOKEEPER-github-pr-build/535//artifact/trunk/
> build/test/findbugs/newPatchFindbugsWarnings.html
> Console output: https://builds.apache.org/job/
> PreCommit-ZOOKEEPER-github-pr-build/535//console
>
> This message is automatically generated.
>
> > Paginated getChildren call
> > --
> >
> > Key: ZOOKEEPER-2260
> > URL: https://issues.apache.org/
> jira/browse/ZOOKEEPER-2260
> > Project: ZooKeeper
> >  Issue Type: New Feature
> >Affects Versions: 3.4.6, 3.5.0
> >Reporter: Marco P.
> >Assignee: Marco P.
> >  Labels: api, features
> > Fix For: 3.5.4, 3.6.0
> >
> > Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
> >
> >
> > Add pagination support to the getChildren() call, allowing clients to
> iterate over children N at the time.
> > Motivations for this include:
> >   - Getting out of a situation where so many children were created that
> listing them exceeded the network buffer sizes (making it impossible to
> recover by deleting)[1]
> >  - More efficient traversal of nodes with large number of children [2]
> > I do have a patch (for 3.4.6) we've been using successfully for a while,
> but I suspect much more work is needed for this to be accepted.
> > [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> > [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.15#6346)
>

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964970#comment-15964970
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

+1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 15 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/535//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/535//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/535//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964920#comment-15964920
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 15 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 1 new Findbugs (version 3.0.1) 
warnings.

-1 release audit.  The applied patch generated 2 release audit warnings 
(more than the trunk's current 0 warnings).

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/534//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/534//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/534//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/534//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Jordan Zimmerman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964903#comment-15964903
 ] 

Jordan Zimmerman commented on ZOOKEEPER-2260:
-

BTW - if this PR is accepted into ZooKeeper, we at Apache Curator would really 
appreciate a corresponding PR if you have the time.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Marco P. (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964901#comment-15964901
 ] 

Marco P. commented on ZOOKEEPER-2260:
-

I've just added some code to the existing PR 
(https://github.com/apache/zookeeper/pull/50).
Pending comments from last Feb were addressed and I modified the wire protocol 
to handle multiple children with the same creation time (i.e. resulting from 
multi() create), the previous implementation was not handling this at all. Also 
added more tests.

[~rakeshr][~fpj][~esteban][~jmhsieh][~rgs] thanks for your patience with this.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Rakesh R (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869254#comment-15869254
 ] 

Rakesh R commented on ZOOKEEPER-2260:
-

FYI, I've triggered 
[build_338|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/338/]
 to get the report of https://github.com/apache/zookeeper/pull/50 recent 
changes.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Rakesh R (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869027#comment-15869027
 ] 

Rakesh R commented on ZOOKEEPER-2260:
-

bq. Jenkins test this please
Thanks [~mprime] for your time and continuous effort in making this feature. I 
could see Jenkins PreCommit build is failing due to {{Unsupported major.minor 
version 52.0}}. I've filed INFRA-13526 jira to resolve this. Will trigger the 
Precommit build once the INFRA problem is resolved.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868895#comment-15868895
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2260:
---

Github user mprimi commented on the issue:

https://github.com/apache/zookeeper/pull/50
  
Jenkins test this please


> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Marco P. (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864283#comment-15864283
 ] 

Marco P. commented on ZOOKEEPER-2260:
-

This is not forgotten [~esteban] [~fpj], please stand by for updates, hopefully 
soon.


> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Flavio Junqueira (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15844081#comment-15844081
 ] 

Flavio Junqueira commented on ZOOKEEPER-2260:
-

I really like this feature! Please submit a pull request, though, see the 
how-to-contribute page:

https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843449#comment-15843449
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12777600/ZOOKEEPER-2260.patch
  against trunk revision 8771ffdaacb87126a485ae740558f6a288ab980b.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

-1 javac.  The patch appears to cause tar ant target to fail.

-1 findbugs.  The patch appears to cause Findbugs (version 3.0.1) to fail.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3572//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3572//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Esteban Gutierrez (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843444#comment-15843444
 ] 

Esteban Gutierrez commented on ZOOKEEPER-2260:
--

Hey [~mprime] are you still working on this? I've run into this issue a couple 
of times while reading 10s of thousands of children and it would be really 
helpful to have some pagination when the client can be resource limited. 
Thanks! 


> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342865#comment-15342865
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12777600/ZOOKEEPER-2260.patch
  against trunk revision 1748630.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

-1 javac.  The patch appears to cause tar ant target to fail.

-1 findbugs.  The patch appears to cause Findbugs (version 2.0.3) to fail.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3196//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3196//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.6.0, 3.5.3
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176976#comment-15176976
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12777600/ZOOKEEPER-2260.patch
  against trunk revision 1733348.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 1 new Findbugs (version 2.0.3) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3058//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3058//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3058//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>Assignee: Marco P.
>  Labels: api, features
> Fix For: 3.5.2, 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Rakesh R (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140697#comment-15140697
 ] 

Rakesh R commented on ZOOKEEPER-2260:
-

Thanks [~mprime] for taking care this. I've added few more comments in the PR. 
Please take a look at it.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>  Labels: api, features
> Fix For: 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Marco P. (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122061#comment-15122061
 ] 

Marco P. commented on ZOOKEEPER-2260:
-

Anyone up for review?

Rakesh already did a couple of passes so shouldn't be too rough.


> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Marco P.
>  Labels: api, features
> Fix For: 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Rakesh R (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086934#comment-15086934
 ] 

Rakesh R commented on ZOOKEEPER-2260:
-

Good work [~mprime]

 I've added comments in the PR, please take a look at it.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 3.6.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Rakesh R (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082505#comment-15082505
 ] 

Rakesh R commented on ZOOKEEPER-2260:
-

OK, its an interesting feature. I will help in reviews. Mostly will give the 
feedback by today or tomorrow (IST).

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Ashish Singhi (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082461#comment-15082461
 ] 

Ashish Singhi commented on ZOOKEEPER-2260:
--

Can we please get reviews done here, this jira will be a very useful 
contribution.
Thanks.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081528#comment-15081528
 ] 

Jonathan Hsieh commented on ZOOKEEPER-2260:
---

Looks like this could be a solution to an issue I filed a long time ago -- 
ZOOKEEPER-1162.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056777#comment-15056777
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2260:
---

GitHub user mprimi opened a pull request:

https://github.com/apache/zookeeper/pull/50

ZOOKEEPER-2260: Paginated getChildren call

Motivation
 - Allow listing children that normally would exceed the buffer limit
 - Get stat for children in the same call as the list

Modifications
- Implemetation of DataTree#getPaginatedChildren
- Added the call to the zk client
- Handle the call in the FinalRequestProcessor
- Added PathWithStat, GetChildrenPaginatedRequest and 
GetChildrenPaginatedResponse to the protos
- Added tests for pagination and watch

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mprimi/zookeeper ZOOKEEPER-2260

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/50.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #50


commit cfc4e2f9ea8582456e8de120b874301cfd7196b9
Author: Marco Primi 
Date:   2015-12-14T21:41:28Z

ZOOKEEPER-2260: Paginated getChildren call

Motivation
Being able to get children with pagination.
Two main benefits:
 - Allow listing children that normally would exceed the buffer limit
 - Get stat for children in the same call as the list

Modifications
- Implemetation of DataTree#getPaginatedChildren
- Added the call to the zk client
- Handle the call in the FinalRequestProcessor
- Added PathWithStat, GetChildrenPaginatedRequest and 
GetChildrenPaginatedResponse to the protos
- Added tests for pagination and watch




> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Marco P. (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056785#comment-15056785
 ] 

Marco P. commented on ZOOKEEPER-2260:
-

[~rgs] yes, I would still love to see this merged. Any feedback on how to make 
it happen is very welcome!

Here's a PR against trunk: https://github.com/apache/zookeeper/pull/50 


> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Raul Gutierrez Segales (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050162#comment-15050162
 ] 

Raul Gutierrez Segales commented on ZOOKEEPER-2260:
---

[~mprime]: sorry for the delay in reviewing this - are you still working on 
this? Could you put the patch either in reviews.apache.org or on a branch on 
github (maybe better/easier) so that I can add some inline comments? Thanks!

 

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047772#comment-15047772
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12755497/ZOOKEEPER-2260.patch
  against trunk revision 1718511.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 2.0.3) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2988//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2988//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2988//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Jordan Zimmerman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741691#comment-14741691
 ] 

Jordan Zimmerman commented on ZOOKEEPER-2260:
-

Are there async versions of the methods?

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741686#comment-14741686
 ] 

Hadoop QA commented on ZOOKEEPER-2260:
--

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12755497/ZOOKEEPER-2260.patch
  against trunk revision 1702378.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 8 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 2.0.3) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2873//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2873//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2873//console

This message is automatically generated.

> Paginated getChildren call
> --
>
> Key: ZOOKEEPER-2260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.4.5, 3.4.6, 3.5.0, 4.0.0
>Reporter: Marco P.
>Priority: Minor
>  Labels: api, features
> Fix For: 4.0.0
>
> Attachments: ZOOKEEPER-2260.patch
>
>
> Add pagination support to the getChildren() call, allowing clients to iterate 
> over children N at the time.
> Motivations for this include:
>   - Getting out of a situation where so many children were created that 
> listing them exceeded the network buffer sizes (making it impossible to 
> recover by deleting)[1]
>  - More efficient traversal of nodes with large number of children [2]
> I do have a patch (for 3.4.6) we've been using successfully for a while, but 
> I suspect much more work is needed for this to be accepted. 
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
> [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Jordan Zimmerman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14718037#comment-14718037
 ] 

Jordan Zimmerman commented on ZOOKEEPER-2260:
-

In any event, I'd love to see this feature. Are you going to post your patch? 
If not, I'd like to take a shot at it.

 Paginated getChildren call
 --

 Key: ZOOKEEPER-2260
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
 Project: ZooKeeper
  Issue Type: New Feature
Reporter: Marco P.
Priority: Minor
  Labels: features

 Add pagination support to the getChildren() call, allowing clients to iterate 
 over children N at the time.
 Motivations for this include:
   - Getting out of a situation where so many children were created that 
 listing them exceeded the network buffer sizes (making it impossible to 
 recover by deleting)[1]
  - More efficient traversal of nodes with large number of children [2]
 I do have a patch (for 3.4.6) we've been using successfully for a while, but 
 I suspect much more work is needed for this to be accepted. 
 [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
 [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Jordan Zimmerman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717999#comment-14717999
 ] 

Jordan Zimmerman commented on ZOOKEEPER-2260:
-

Multiple sorts might be needed. Sorting by sequence number, for example. Or is 
that the same as zxid?

 Paginated getChildren call
 --

 Key: ZOOKEEPER-2260
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
 Project: ZooKeeper
  Issue Type: New Feature
Reporter: Marco P.
Priority: Minor
  Labels: features

 Add pagination support to the getChildren() call, allowing clients to iterate 
 over children N at the time.
 Motivations for this include:
   - Getting out of a situation where so many children were created that 
 listing them exceeded the network buffer sizes (making it impossible to 
 recover by deleting)[1]
  - More efficient traversal of nodes with large number of children [2]
 I do have a patch (for 3.4.6) we've been using successfully for a while, but 
 I suspect much more work is needed for this to be accepted. 
 [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
 [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Jordan Zimmerman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717835#comment-14717835
 ] 

Jordan Zimmerman commented on ZOOKEEPER-2260:
-

In order for this to be useful I think the method would have to also return the 
version of the parent node so that multiple calls could validate that the 
version number hasn't changed. Otherwise, how do yo correlate 2 calls to the 
method?

 Paginated getChildren call
 --

 Key: ZOOKEEPER-2260
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
 Project: ZooKeeper
  Issue Type: New Feature
Reporter: Marco P.
Priority: Minor
  Labels: features

 Add pagination support to the getChildren() call, allowing clients to iterate 
 over children N at the time.
 Motivations for this include:
   - Getting out of a situation where so many children were created that 
 listing them exceeded the network buffer sizes (making it impossible to 
 recover by deleting)[1]
  - More efficient traversal of nodes with large number of children [2]
 I do have a patch (for 3.4.6) we've been using successfully for a while, but 
 I suspect much more work is needed for this to be accepted. 
 [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
 [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call

[Marco P. (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717844#comment-14717844
 ] 

Marco P. commented on ZOOKEEPER-2260:
-

The version we're using returns children sorted by ZxId, so if children can be 
created and deleted while pagination is in progress.
The newly created will show up in the last page (due to creation order). 




 Paginated getChildren call
 --

 Key: ZOOKEEPER-2260
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260
 Project: ZooKeeper
  Issue Type: New Feature
Reporter: Marco P.
Priority: Minor
  Labels: features

 Add pagination support to the getChildren() call, allowing clients to iterate 
 over children N at the time.
 Motivations for this include:
   - Getting out of a situation where so many children were created that 
 listing them exceeded the network buffer sizes (making it impossible to 
 recover by deleting)[1]
  - More efficient traversal of nodes with large number of children [2]
 I do have a patch (for 3.4.6) we've been using successfully for a while, but 
 I suspect much more work is needed for this to be accepted. 
 [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
 [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)