Release 4.4.0
Hi folks: I'd like to come up a list of major items for 4.4.0. Then we should start planning the release for 4.4.0 Here are the major items that I have in my mind: - last-add-confirmed long poll (BOOKKEEPER-670) - delay ensemble change (BOOKKEEPER-759) - bunch of bookie storage improvements (compaction, caching) that added in 4.4.0 - ssl authentication support (BOOKKEEPER-575) Please feel free to add if I missing anything. - Sijie
[jira] [Commented] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679245#comment-14679245
]
Ted Yu commented on BOOKKEEPER-863:
---
I cannot assign this to myself.
Please add me as a contributor.
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Attachments: bookkeeper-863.patch, bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679253#comment-14679253
]
Sijie Guo commented on BOOKKEEPER-863:
--
[~tedyu] I added you as the contributor and assigned to you. did you compile
before generating the patch? It seems that the patch couldn't compile
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch, bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Release 4.4.0
How about Enhance client API (BOOKKEEPER-860)? On Sun, Aug 9, 2015 at 10:42 AM, Sijie Guo si...@apache.org wrote: Hi folks: I'd like to come up a list of major items for 4.4.0. Then we should start planning the release for 4.4.0 Here are the major items that I have in my mind: - last-add-confirmed long poll (BOOKKEEPER-670) - delay ensemble change (BOOKKEEPER-759) - bunch of bookie storage improvements (compaction, caching) that added in 4.4.0 - ssl authentication support (BOOKKEEPER-575) Please feel free to add if I missing anything. - Sijie -- Jvrao --- First they ignore you, then they laugh at you, then they fight you, then you win. - Mahatma Gandhi
[jira] [Updated] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sijie Guo updated BOOKKEEPER-863:
-
Assignee: Ted Yu
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch, bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679230#comment-14679230
]
Sijie Guo commented on BOOKKEEPER-863:
--
[~tedyu] the fix looks good. could you generate the patch with --no-prefix? so
the patch could be applied by Jenkins
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Attachments: bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated BOOKKEEPER-863:
--
Attachment: bookkeeper-863.patch
This one passes compilation.
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch, bookkeeper-863.patch,
bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679248#comment-14679248
]
Hadoop QA commented on BOOKKEEPER-863:
--
Testing JIRA BOOKKEEPER-863
Patch
[bookkeeper-863.patch|https://issues.apache.org/jira/secure/attachment/12749478/bookkeeper-863.patch]
downloaded at Sun Aug 9 17:51:15 UTC 2015
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
120
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 FINDBUGS{color}
.{color:green}+1{color} the patch does not seem to introduce new Findbugs
warnings
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/949/
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Attachments: bookkeeper-863.patch, bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated BOOKKEEPER-863:
--
Attachment: (was: bookkeeper-863.patch)
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated BOOKKEEPER-863:
--
Attachment: (was: bookkeeper-863.patch)
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Proposed board report
Hi folks, We have a board report due this month. Here is the draft. Let me know if you have anything to add. I plan to submit this on 8/11. snip BookKeeper is a distributed, reliable, and high performance logging service. The project also includes Hedwig which is a highly scalable Pub/Sub service built on top of ZooKeeper and BookKeeper with strong durability guarantees. = Project Status = Development has continued towards the 4.4.0 release, which is a feature release. It will be including improvements on bookie storage, compaction and ledger recovery, and new features about last-add-confirmed long poll. A new api about ledger handle is on proposal and under discussing. = Releases = Our last release was 4.3.1, released on 2015-05-27. The next release will be 4.4.0. = Community Status = The last committer added was Matteo Merli (mmerli) on the 6th Jul 2015. We are going to schedule the first user meetup on September. No infrastructure issues. 61 subscribers in dev@bookkeeper.apache.org 77 subscribers in u...@bookkeeper.apache.org 853 issues opened to date, 10 since 2015-05-15 589 issues resolved to date, 8 since 2015-05-15 54 people have reported issues, 7 since 2015-05-15 26 people have contributed patches, 4 since 2015-05-15 /snip
[jira] [Updated] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated BOOKKEEPER-863:
--
Attachment: bookkeeper-863.patch
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Attachments: bookkeeper-863.patch, bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Release 4.4.0
BK-860 is currently marked for 4.4.0, but I agree with Ivan that we shouldn't block the release on it. It doesn't stop you from working on it, though, and if we are able to converge fast enough, then I don't see a reason for not having it in, Jvrao. -Flavio On 09 Aug 2015, at 22:23, Ivan Kelly iv...@apache.org wrote: Enhance client API (BOOKKEEPER-860)? I think this change is hasn't been fleshed out enough for this release. The other changes that Sijie suggested are all about 80-100% done already. -Ivan
[jira] [Commented] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679295#comment-14679295
]
Hadoop QA commented on BOOKKEEPER-863:
--
Testing JIRA BOOKKEEPER-863
Patch
[bookkeeper-863.patch|https://issues.apache.org/jira/secure/attachment/12749481/bookkeeper-863.patch]
downloaded at Sun Aug 9 18:47:16 UTC 2015
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
120
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 FINDBUGS{color}
.{color:green}+1{color} the patch does not seem to introduce new Findbugs
warnings
{color:green}+1 TESTS{color}
.Tests run: 875
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/951/
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Release 4.4.0
This list looks good to me. +1 -Ivan On Sun, Aug 9, 2015 at 7:42 PM Sijie Guo si...@apache.org wrote: Hi folks: I'd like to come up a list of major items for 4.4.0. Then we should start planning the release for 4.4.0 Here are the major items that I have in my mind: - last-add-confirmed long poll (BOOKKEEPER-670) - delay ensemble change (BOOKKEEPER-759) - bunch of bookie storage improvements (compaction, caching) that added in 4.4.0 - ssl authentication support (BOOKKEEPER-575) Please feel free to add if I missing anything. - Sijie
Re: Release 4.4.0
Enhance client API (BOOKKEEPER-860)? I think this change is hasn't been fleshed out enough for this release. The other changes that Sijie suggested are all about 80-100% done already. -Ivan
Re: Proposed board report
+1, thanks for putting this together, Sijie. Small typo: We are going to schedule the first user meetup in September. -Flavio On 09 Aug 2015, at 22:21, Ivan Kelly iv...@apache.org wrote: lgtm +1 On Sun, Aug 9, 2015 at 7:35 PM Sijie Guo si...@apache.org wrote: Hi folks, We have a board report due this month. Here is the draft. Let me know if you have anything to add. I plan to submit this on 8/11. snip BookKeeper is a distributed, reliable, and high performance logging service. The project also includes Hedwig which is a highly scalable Pub/Sub service built on top of ZooKeeper and BookKeeper with strong durability guarantees. = Project Status = Development has continued towards the 4.4.0 release, which is a feature release. It will be including improvements on bookie storage, compaction and ledger recovery, and new features about last-add-confirmed long poll. A new api about ledger handle is on proposal and under discussing. = Releases = Our last release was 4.3.1, released on 2015-05-27. The next release will be 4.4.0. = Community Status = The last committer added was Matteo Merli (mmerli) on the 6th Jul 2015. We are going to schedule the first user meetup on September. No infrastructure issues. 61 subscribers in dev@bookkeeper.apache.org 77 subscribers in u...@bookkeeper.apache.org 853 issues opened to date, 10 since 2015-05-15 589 issues resolved to date, 8 since 2015-05-15 54 people have reported issues, 7 since 2015-05-15 26 people have contributed patches, 4 since 2015-05-15 /snip
Re: Release 4.4.0
Ah. I see. Thanks for the clarification. On Sunday, August 9, 2015, Ivan Kelly iv...@apache.org wrote: This list looks good to me. +1 -Ivan On Sun, Aug 9, 2015 at 7:42 PM Sijie Guo si...@apache.org javascript:; wrote: Hi folks: I'd like to come up a list of major items for 4.4.0. Then we should start planning the release for 4.4.0 Here are the major items that I have in my mind: - last-add-confirmed long poll (BOOKKEEPER-670) - delay ensemble change (BOOKKEEPER-759) - bunch of bookie storage improvements (compaction, caching) that added in 4.4.0 - ssl authentication support (BOOKKEEPER-575) Please feel free to add if I missing anything. - Sijie -- Sent from iPhone
Re: Proposed board report
lgtm +1 On Sun, Aug 9, 2015 at 7:35 PM Sijie Guo si...@apache.org wrote: Hi folks, We have a board report due this month. Here is the draft. Let me know if you have anything to add. I plan to submit this on 8/11. snip BookKeeper is a distributed, reliable, and high performance logging service. The project also includes Hedwig which is a highly scalable Pub/Sub service built on top of ZooKeeper and BookKeeper with strong durability guarantees. = Project Status = Development has continued towards the 4.4.0 release, which is a feature release. It will be including improvements on bookie storage, compaction and ledger recovery, and new features about last-add-confirmed long poll. A new api about ledger handle is on proposal and under discussing. = Releases = Our last release was 4.3.1, released on 2015-05-27. The next release will be 4.4.0. = Community Status = The last committer added was Matteo Merli (mmerli) on the 6th Jul 2015. We are going to schedule the first user meetup on September. No infrastructure issues. 61 subscribers in dev@bookkeeper.apache.org 77 subscribers in u...@bookkeeper.apache.org 853 issues opened to date, 10 since 2015-05-15 589 issues resolved to date, 8 since 2015-05-15 54 people have reported issues, 7 since 2015-05-15 26 people have contributed patches, 4 since 2015-05-15 /snip
[jira] [Commented] (BOOKKEEPER-863) Potential resource leak with unclosed LedgerManager in BookieShell
[
https://issues.apache.org/jira/browse/BOOKKEEPER-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679591#comment-14679591
]
Rakesh R commented on BOOKKEEPER-863:
-
Thanks [~yuzhih...@gmail.com] for the contribution. Just few suggestions:
# Please uninitialize factory. Also, good to catch exceptions in the finally
block like,
{code}
} finally {
try{
m.close();
mFactory.uninitialize();
} catch (IOException ioe) {
LOG.error(Failed to close ledger manager : , ioe);
}
...
...
}
{code}
# Please close the {{ledgerManager#close();}} before {{zk#close();}}. IMHO,
this avoids unwanted ZooKeeper client exception in between closure.
Potential resource leak with unclosed LedgerManager in BookieShell
--
Key: BOOKKEEPER-863
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-863
Project: Bookkeeper
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: bookkeeper-863.patch
There're more than one occurrence.
Here is an example in ListLedgersCmd#runCmd():
{code}
LedgerManagerFactory mFactory =
LedgerManagerFactory.newLedgerManagerFactory(bkConf, zk);
LedgerManager m = mFactory.newLedgerManager();
{code}
m is not closed upon leaving the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-862) Add tracing and stats to OrderedSafeExecutor for debugging slow tasks
[ https://issues.apache.org/jira/browse/BOOKKEEPER-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sijie Guo updated BOOKKEEPER-862: - Attachment: ose_stats-rebase.patch [~l4stewar] Leigh, I cleaned up your patch to remove unnecessary diffs in hedwig modules. Add tracing and stats to OrderedSafeExecutor for debugging slow tasks - Key: BOOKKEEPER-862 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-862 Project: Bookkeeper Issue Type: Improvement Components: bookkeeper-server Reporter: Leigh Stewart Assignee: Leigh Stewart Priority: Minor Fix For: 4.4.0 Attachments: ose_stats-rebase.patch, ose_stats.patch Porting a change form the Twitter branch to improve stats and logging in OrderedSafeExecutor These changes have been helpful for us in debugging latency issues in Bookkeeper server/client Summary of changes is * add a config option for op stats * add stats for task execution time, task pending time * add a config option for logging a warning when an op takes longer than x micros * add toString implementations for submitted tasks so make it easier to track down slow ops * start using Builder for OrderedSafeExecutor * add a very simple test to make sure that the slow op logging path is exercised Most of this came from Sijie originally, with some changes from me -- This message was sent by Atlassian JIRA (v6.3.4#6332)
