[jira] [Commented] (BOOKKEEPER-946) Provide an option to delay auto recovery of lost bookies

2016-11-24 Thread Enrico Olivelli (JIRA) 

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692585#comment-15692585
 ] 

Enrico Olivelli commented on BOOKKEEPER-946:


Thank you [~rithin.shetty]
I'm trying your change on my private build machines, the official BookKeeper QA 
build of the PR went well (the error is not related yo your fix)

> Provide an option to delay auto recovery of lost bookies
> 
>
> Key: BOOKKEEPER-946
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-946
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
> Attachments: 
> org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt, 
> org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt
>
>
> If auto recovery is enabled, and a bookie goes down for upgrade or even if it 
> looses zk connection intermittently, the auditor detects it as a lost bookie 
> and starts under replication detection and the replication workers on other 
> bookie nodes start replicating the under replicated ledgers. All of this 
> stops once the bookie comes up but by then a few ledgers would get 
> replicated. Given the fact that we have multiple copies of data, it is 
> probably not necessary to start the recovery as soon as a bookie goes down. 
> We can probably wait for an hour or so and then start recovery. This should 
> cover cases like planned upgrade, intermittent network connectivity loss, 
> etc. The amount of time to wait can be an option and the default would be to 
> not wait at all(i.e. retain current behavior).
> Of course, if more than one bookie goes down within a short interval, we 
> could decide to start auto recovery without waiting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BOOKKEEPER-975) Jenkins failed while test BookieClientTest.testWriteGaps

2016-11-24 Thread Jia Zhai (JIRA) 
Jia Zhai created BOOKKEEPER-975:
---

 Summary: Jenkins failed while test BookieClientTest.testWriteGaps
 Key: BOOKKEEPER-975
 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-975
 Project: Bookkeeper
  Issue Type: Bug
Reporter: Jia Zhai


Here is error reporting:
```
2016-11-23 09:03:24,919 - ERROR - 
[BookieWriteThread-13645-orderedsafeexecutor-0-0:WriteEntryProcessorV3@125] - 
Unexpected exception while writing 1@1 : 
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:506)
at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:412)
at 
org.apache.bookkeeper.bookie.SortedLedgerStorage.addEntry(SortedLedgerStorage.java:99)
at 
org.apache.bookkeeper.bookie.LedgerDescriptorImpl.addEntry(LedgerDescriptorImpl.java:80)
at 
org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1176)
at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1235)
at 
org.apache.bookkeeper.proto.WriteEntryProcessorV3.getAddResponse(WriteEntryProcessorV3.java:109)
at 
org.apache.bookkeeper.proto.WriteEntryProcessorV3.safeRun(WriteEntryProcessorV3.java:142)
at org.apache.bookkeeper.util.SafeRunnable.run(SafeRunnable.java:31)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-11-23 09:03:24,950 - ERROR - 
[BKClientOrderedSafeExecutor-orderedsafeexecutor-1-0:SafeRunnable@33] - 
Unexpected throwable caught 
java.lang.IndexOutOfBoundsException: Invalid readerIndex: 16 - Maximum is 0
at 
org.jboss.netty.buffer.EmptyChannelBuffer.readerIndex(EmptyChannelBuffer.java:50)
at 
org.apache.bookkeeper.test.BookieClientTest$1.readEntryComplete(BookieClientTest.java:112)
at 
org.apache.bookkeeper.proto.PerChannelBookieClient$ReadCompletion$1.readEntryComplete(PerChannelBookieClient.java:930)
at 
org.apache.bookkeeper.proto.PerChannelBookieClient.handleReadResponse(PerChannelBookieClient.java:868)
at 
org.apache.bookkeeper.proto.PerChannelBookieClient$7.safeRun(PerChannelBookieClient.java:793)
at org.apache.bookkeeper.util.SafeRunnable.run(SafeRunnable.java:31)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-11-23 09:03:25,088 - WARN  - 
[GarbageCollectorThread:ScanAndCompareGarbageCollector@153] - Exception when 
iterating over the metadata {}
java.lang.NullPointerException
at 
org.apache.bookkeeper.util.ZkUtils.getChildrenInSingleNode(ZkUtils.java:207)
at 
org.apache.bookkeeper.util.ZkUtils.getChildrenInSingleNode(ZkUtils.java:169)
at 
org.apache.bookkeeper.meta.FlatLedgerManager$1.preload(FlatLedgerManager.java:110)
at 
org.apache.bookkeeper.meta.FlatLedgerManager$1.hasNext(FlatLedgerManager.java:120)
at 
org.apache.bookkeeper.bookie.ScanAndCompareGarbageCollector.gc(ScanAndCompareGarbageCollector.java:104)
at 
org.apache.bookkeeper.bookie.GarbageCollectorThread.doGcLedgers(GarbageCollectorThread.java:371)
at 
org.apache.bookkeeper.bookie.GarbageCollectorThread.run(GarbageCollectorThread.java:329)
2016-11-23 09:03:25,944 - INFO  - [main:BookieServer@167] - Shutting down 
BookieServer
2016-11-23 09:03:25,945 - INFO  - [main:BookieNettyServer@127] - Shutting down 
BookieNettyServer
2016-11-23 09:03:25,950 - INFO  - [New I/O worker 
#10:PerChannelBookieClient@701] - Disconnected from bookie channel [id: 
0x7b660837, /127.0.0.1:48812 :> /127.0.0.1:13645]
2016-11-23 09:03:25,971 - INFO  - [main:Bookie@1096] - Shutting down 
Bookie-13645 with exitCode 0
2016-11-23 09:03:25,972 - INFO  - [main:Journal@970] - Shutting down Journal
2016-11-23 09:03:25,973 - ERROR - 
[ForceWriteThread:Journal$ForceWriteThread@433] - ForceWrite thread interrupted
java.lang.InterruptedException
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at 
org.apache.bookkeeper.bookie.Journal$ForceWriteThread.run(Journal.java:393)
2016-11-23 09:03:25,974 - WARN  -

[jira] [Commented] (BOOKKEEPER-966) change the bookieServer cmdline to make conf-file and option co-exist

2016-11-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15694724#comment-15694724
 ] 

ASF GitHub Bot commented on BOOKKEEPER-966:
---

Github user jiazhai commented on the issue:

https://github.com/apache/bookkeeper/pull/75
  
Jenkins error seems not related with this change. Opened tickiet 
BOOKKEEPER-975 for it.


> change the bookieServer cmdline to make conf-file and option co-exist
> -
>
> Key: BOOKKEEPER-966
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-966
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
>
> Currently, when using bookieServer cmdline to start a bookie, you will either 
> give it a cofiguration file by "-c booke.conf"; or add some options like 
> "   " in a fix 
> sequential.
> It may not satisfy some of the requirement. So changed it to be co-exist for 
> configuration file and options.
> By this change, it will first use settings in configuration file; and then 
> use options to overwrite some of the settings, if there are some options 
> provided.
> Here is an example after this change:
> BookieServer -c bookie.conf -z localhost:2181 -m /bookkeeper/ledgers -p 3181 
> -j /mnt/journal -l "/mnt/ledger1 /mnt/ledger2 /mnt/ledger3”
> Here, in this command:
> -z is for “Zookeeper client instance”;
> -m is for "Zookeeper ledgers root path for bookies";
> -p is for "bookie service port exported";
> -j is for "bookie journal directory";
> -l is for "bookie ledgers directories".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BOOKKEEPER-971) update bk codahale stats provider version

2016-11-24 Thread Jia Zhai (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BOOKKEEPER-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jia Zhai updated BOOKKEEPER-971:

Summary: update bk codahale stats provider version  (was: update bk stats 
provider: from codahale to yammer)

> update bk codahale stats provider version
> -
>
> Key: BOOKKEEPER-971
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-971
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
>
> update bk stats provider: from codahale to yammer.
> Currently io.dropwizard.metrics 3.1.0 is used most. will change version to 
> this version, and 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-971) update bk codahale stats provider version

2016-11-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695028#comment-15695028
 ] 

ASF GitHub Bot commented on BOOKKEEPER-971:
---

GitHub user jiazhai opened a pull request:

https://github.com/apache/bookkeeper/pull/83

BOOKKEEPER-971: update bk codahale stats provider version

Update bk stats provider: from codahale to yammer.
Currently io.dropwizard.metrics 3.1.0 is used most widely. will change 
version to 3.1.0.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jiazhai/bookkeeper-1 BOOKKEEPER-971

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/83.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #83


commit ba1c99f6829b4479e10b02fc1e4e368b8dd178df
Author: jiazhai 
Date:   2016-11-09T07:46:26Z

Merge pull request #1 from apache/master

catch up with parent stream at 11_9

commit 68052aa72fc11d13ca193d2a245102a970e4d6ca
Author: jiazhai 
Date:   2016-11-25T06:54:14Z

update codahale version




> update bk codahale stats provider version
> -
>
> Key: BOOKKEEPER-971
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-971
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
>
> update bk stats provider: from codahale to yammer.
> Currently io.dropwizard.metrics 3.1.0 is used most. will change version to 
> this version, and run the test



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: bookkeeper-master-git-pullrequest #161

2016-11-24 Thread Apache Jenkins Server 
See 


Changes:

[jia.zhai] update codahale version

--
[...truncated 385 lines...]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ 
bookkeeper-server ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 224 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ 
bookkeeper-server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @ 
bookkeeper-server ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 107 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ bookkeeper-server 
---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.

---
 T E S T S
---
Running org.apache.bookkeeper.proto.TestBackwardCompatCMS42
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.205 sec - in 
org.apache.bookkeeper.proto.TestBackwardCompatCMS42
Running org.apache.bookkeeper.proto.TestDeathwatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.498 sec - in 
org.apache.bookkeeper.proto.TestDeathwatcher
Running org.apache.bookkeeper.proto.NetworkLessBookieTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.269 sec - in 
org.apache.bookkeeper.proto.NetworkLessBookieTest
Running org.apache.bookkeeper.proto.TestBKStats
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.38 sec - in 
org.apache.bookkeeper.proto.TestBKStats
Running org.apache.bookkeeper.proto.TestPerChannelBookieClient
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.568 sec - in 
org.apache.bookkeeper.proto.TestPerChannelBookieClient
Running org.apache.bookkeeper.conf.NoSystemPropertiesConfigurationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.407 sec - in 
org.apache.bookkeeper.conf.NoSystemPropertiesConfigurationTest
Running org.apache.bookkeeper.conf.SystemPropertiesConfigurationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.434 sec - in 
org.apache.bookkeeper.conf.SystemPropertiesConfigurationTest
Running org.apache.bookkeeper.zookeeper.TestZooKeeperClient
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.774 sec - in 
org.apache.bookkeeper.zookeeper.TestZooKeeperClient
Running org.apache.bookkeeper.bookie.BookieJournalTest
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.565 sec - in 
org.apache.bookkeeper.bookie.BookieJournalTest
Running org.apache.bookkeeper.bookie.EntryLogTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.099 sec - in 
org.apache.bookkeeper.bookie.EntryLogTest
Running org.apache.bookkeeper.bookie.IndexCorruptionTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.51 sec - in 
org.apache.bookkeeper.bookie.IndexCorruptionTest
Running org.apache.bookkeeper.bookie.IndexPersistenceMgrTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.644 sec - in 
org.apache.bookkeeper.bookie.IndexPersistenceMgrTest
Running org.apache.bookkeeper.bookie.TestSyncThread
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.083 sec - in 
org.apache.bookkeeper.bookie.TestSyncThread
Running org.apache.bookkeeper.bookie.TestLedgerDirsManager
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.548 sec - in 
org.apache.bookkeeper.bookie.TestLedgerDirsManager
Running org.apache.bookkeeper.bookie.BookieShutdownTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.128 sec - in 
org.apache.bookkeeper.bookie.BookieShutdownTest
Running org.apache.bookkeeper.bookie.UpdateCookieCmdTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.47 sec - in 
org.apache.bookkeeper.bookie.UpdateCookieCmdTest
Running org.apache.bookkeeper.bookie.UpgradeTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.171 sec - in 
org.apache.bookkeeper.bookie.UpgradeTest
Running org.apache.bookkeeper.bookie.TestGcOverreplicatedLedger
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.901 sec - in 
org.apache.bookkeeper.bookie.TestGcOverreplicatedLedger
Running org.apache.bookkeeper.bookie.CookieTest
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.871 sec - in 
org.apache.bookkeeper.bookie.CookieTest

[jira] [Commented] (BOOKKEEPER-946) Provide an option to delay auto recovery of lost bookies

2016-11-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15693364#comment-15693364
 ] 

ASF GitHub Bot commented on BOOKKEEPER-946:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/82
  
Thank you  @rithin-shetty for the fix

LGTM +1

I have run the full suite many times on the same machines where they were 
failing very often and now that error does not occur anymore.

On the QA build the test did not fail (see 
https://builds.apache.org/job/bookkeeper-master-git-pullrequest/160/testReport/)








> Provide an option to delay auto recovery of lost bookies
> 
>
> Key: BOOKKEEPER-946
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-946
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
> Attachments: 
> org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt, 
> org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt
>
>
> If auto recovery is enabled, and a bookie goes down for upgrade or even if it 
> looses zk connection intermittently, the auditor detects it as a lost bookie 
> and starts under replication detection and the replication workers on other 
> bookie nodes start replicating the under replicated ledgers. All of this 
> stops once the bookie comes up but by then a few ledgers would get 
> replicated. Given the fact that we have multiple copies of data, it is 
> probably not necessary to start the recovery as soon as a bookie goes down. 
> We can probably wait for an hour or so and then start recovery. This should 
> cover cases like planned upgrade, intermittent network connectivity loss, 
> etc. The amount of time to wait can be an option and the default would be to 
> not wait at all(i.e. retain current behavior).
> Of course, if more than one bookie goes down within a short interval, we 
> could decide to start auto recovery without waiting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: bookkeeper-master #1576

2016-11-24 Thread Apache Jenkins Server 
See 

--
[...truncated 369 lines...]
[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ 
bookkeeper-server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @ 
bookkeeper-server ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 107 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ bookkeeper-server 
---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.

---
 T E S T S
---
Running org.apache.bookkeeper.proto.TestPerChannelBookieClient
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.762 sec - in 
org.apache.bookkeeper.proto.TestPerChannelBookieClient
Running org.apache.bookkeeper.proto.NetworkLessBookieTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.892 sec - in 
org.apache.bookkeeper.proto.NetworkLessBookieTest
Running org.apache.bookkeeper.proto.TestBackwardCompatCMS42
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.39 sec - in 
org.apache.bookkeeper.proto.TestBackwardCompatCMS42
Running org.apache.bookkeeper.proto.TestBKStats
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.152 sec - in 
org.apache.bookkeeper.proto.TestBKStats
Running org.apache.bookkeeper.proto.TestDeathwatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.159 sec - in 
org.apache.bookkeeper.proto.TestDeathwatcher
Running org.apache.bookkeeper.replication.BookieLedgerIndexTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.94 sec - in 
org.apache.bookkeeper.replication.BookieLedgerIndexTest
Running org.apache.bookkeeper.replication.TestAutoRecoveryAlongWithBookieServers
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.803 sec - in 
org.apache.bookkeeper.replication.TestAutoRecoveryAlongWithBookieServers
Running org.apache.bookkeeper.replication.AuditorBookieTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.285 sec - in 
org.apache.bookkeeper.replication.AuditorBookieTest
Running org.apache.bookkeeper.replication.AuditorPeriodicCheckTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.401 sec - in 
org.apache.bookkeeper.replication.AuditorPeriodicCheckTest
Running org.apache.bookkeeper.replication.TestLedgerUnderreplicationManager
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.897 sec - 
in org.apache.bookkeeper.replication.TestLedgerUnderreplicationManager
Running org.apache.bookkeeper.replication.TestReplicationWorker
Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.763 sec - 
in org.apache.bookkeeper.replication.TestReplicationWorker
Running org.apache.bookkeeper.replication.AuditorPeriodicBookieCheckTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.272 sec - in 
org.apache.bookkeeper.replication.AuditorPeriodicBookieCheckTest
Running org.apache.bookkeeper.replication.AuditorRollingRestartTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 95.74 sec - in 
org.apache.bookkeeper.replication.AuditorRollingRestartTest
Running org.apache.bookkeeper.replication.BookieAutoRecoveryTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.422 sec - in 
org.apache.bookkeeper.replication.BookieAutoRecoveryTest
Running org.apache.bookkeeper.replication.AutoRecoveryMainTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.464 sec - in 
org.apache.bookkeeper.replication.AutoRecoveryMainTest
Running org.apache.bookkeeper.replication.AuditorLedgerCheckerTest
Tests run: 33, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 127.645 sec 
<<< FAILURE! - in org.apache.bookkeeper.replication.AuditorLedgerCheckerTest
testDelayedAuditOfLostBookies[0](org.apache.bookkeeper.replication.AuditorLedgerCheckerTest)
  Time elapsed: 5.473 sec  <<< FAILURE!
java.lang.AssertionError: audit of lost bookie isn't delayed
at 
org.apache.bookkeeper.replication.AuditorLedgerCheckerTest._testDelayedAuditOfLostBookies(AuditorLedgerCheckerTest.java:345)
at 
org.apache.bookkeeper.replication.AuditorLedgerCheckerTest.testDelayedAuditOfLostBookies(AuditorLedgerCheckerTest.java:367)

testDelayedAuditOfLostBookies[1](org.apache.bookkeeper.replication.AuditorLedgerCheckerTest)
  Time elapsed: 5.401 sec  <<< FAILURE!
java.lang.AssertionError: audit of lost bookie isn't delayed
at 
org.apache.bookkeeper.replication.AuditorLedgerCheckerTest._testDelayedAuditOfLostBookies(AuditorLedgerCheckerTest.java:345)

[jira] [Commented] (BOOKKEEPER-588) SSL support

2016-11-24 Thread Enrico Olivelli (JIRA) 

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15693580#comment-15693580
 ] 

Enrico Olivelli commented on BOOKKEEPER-588:


sorry [~jujjuri] for late reply.
{quote}
Can't we use two ports on Bookie? one for secure connection and other for 
non-secure? We can be in this mode until all our clients move to secure and 
then re-roll bookies to accept only-secure connection.
{quote}

The change will be greater and it will impact bookie registration and discovery 
and ledger metadata. I think we already discussed about this during the BP-3 
meeting. [~hustlmsp] maybe can explain better. If you use StartTLS the bookie 
will listen only on one endpoint and no change should be done.
Maybe we should consider the Addrerss.isUnresolved case as you did in your 
patch.

{quote}
Start TLS can be a way too, but I fail to understand the security aspect of it. 
If Client has to request secure connection, what is going to stop a rogue 
client establishing connection with Bookie and continue in that way? Is your 
plan to make use of Authentication + StatTLS to avoid STRIPTLS attack? 
(https://en.wikipedia.org/wiki/Opportunistic_TLS#Weaknesses_and_mitigations)
{quote}
We will implement configuration flags to prevent unsecured connections. 
The ConnectionPeer interface introduced with  BOOKKEEPER-959  will be enhanced 
with a boolean "isSecure()" information in order to known that the comunication 
is encrypted. So the AuthPlugin will be able to deny authentication for every 
connection which did not run the StartTLS protocol.

For the mutual authentication case this can be  implemented even on the 
AuthPlugin as the AuthPlugin will be aware of the certificates which are 
exchanged on the wire. Bookies do no allow requests from Clients which are not 
in the 'authenticated' state and the SSL AuthPlugin will block clients without 
certificate (depending on the configuration)
So we can implement an option "allow only secure connections".

The STRIPTLS attack will not be our case, infact we can make the client close 
any connection to Bookies which do not accept StartTLS at all. In the SMTP 
world StartTLS is really optional and so there are many troubles.

{quote}
what is your approach on certificate expiry boundary? Will you let client fail 
and restart?
{quote}

My idea is that on the Bookie side a background thread started by the 
AuthPlugin will check connected clients and drop the connection. The client 
from the other side will need to open a new connection and read a new file . 
This is the very like what you are doing  in your patch.
The  way the client "reloads" the certificate is not very clear to me, I have 
to figure out a practical way for the system administrator to switch the file 
while the client is running







> SSL support
> ---
>
> Key: BOOKKEEPER-588
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-588
> Project: Bookkeeper
>  Issue Type: Sub-task
>Reporter: Ivan Kelly
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
> Attachments: 0001-MutualTLS-for-Bookkeeper.patch, 
> 0004-BOOKKEEPER-588-SSL-support-for-bookkeeper.patch
>
>
> SSL support using startTLS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (BOOKKEEPER-588) SSL support

2016-11-24 Thread Enrico Olivelli (JIRA) 

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15693580#comment-15693580
 ] 

Enrico Olivelli edited comment on BOOKKEEPER-588 at 11/24/16 3:48 PM:
--

sorry [~jujjuri] for late reply.
{quote}
Can't we use two ports on Bookie? one for secure connection and other for 
non-secure? We can be in this mode until all our clients move to secure and 
then re-roll bookies to accept only-secure connection.
{quote}

The change will be greater and it will impact bookie registration and discovery 
and ledger metadata. I think we already discussed about this during the BP-3 
meeting. [~hustlmsp] maybe can explain better. If you use StartTLS the bookie 
will listen only on one endpoint and no change should be done.
Maybe we should consider the Addrerss.isUnresolved case as you did in your 
patch.

{quote}
Start TLS can be a way too, but I fail to understand the security aspect of it. 
If Client has to request secure connection, what is going to stop a rogue 
client establishing connection with Bookie and continue in that way? Is your 
plan to make use of Authentication + StatTLS to avoid STRIPTLS attack? 
(https://en.wikipedia.org/wiki/Opportunistic_TLS#Weaknesses_and_mitigations)
{quote}
We will implement configuration flags to prevent unsecured connections. 
The ConnectionPeer interface introduced with  BOOKKEEPER-959  will be enhanced 
with a boolean "isSecure()" information in order to known that the comunication 
is encrypted. So the AuthPlugin will be able to deny authentication for every 
connection which did not run the StartTLS protocol.

For the mutual authentication case this can be  implemented even on the 
AuthPlugin as the AuthPlugin will be aware of the certificates which are 
exchanged on the wire (but we will have to deal with the truststore loading). 
Bookies do no allow requests from Clients which are not in the 'authenticated' 
state and the SSL AuthPlugin will block clients without certificate (depending 
on the configuration)
So we can implement an option "allow only secure connections".

The STRIPTLS attack will not be our case, infact we can make the client close 
any connection to Bookies which do not accept StartTLS at all. In the SMTP 
world StartTLS is really optional and so there are many troubles.

{quote}
what is your approach on certificate expiry boundary? Will you let client fail 
and restart?
{quote}

My idea is that on the Bookie side a background thread started by the 
AuthPlugin will check connected clients and drop the connection. The client 
from the other side will need to open a new connection and read a new file . 
This is the very like what you are doing  in your patch.
The  way the client "reloads" the certificate is not very clear to me, I have 
to figure out a practical way for the system administrator to switch the file 
while the client is running








was (Author: eolivelli):
sorry [~jujjuri] for late reply.
{quote}
Can't we use two ports on Bookie? one for secure connection and other for 
non-secure? We can be in this mode until all our clients move to secure and 
then re-roll bookies to accept only-secure connection.
{quote}

The change will be greater and it will impact bookie registration and discovery 
and ledger metadata. I think we already discussed about this during the BP-3 
meeting. [~hustlmsp] maybe can explain better. If you use StartTLS the bookie 
will listen only on one endpoint and no change should be done.
Maybe we should consider the Addrerss.isUnresolved case as you did in your 
patch.

{quote}
Start TLS can be a way too, but I fail to understand the security aspect of it. 
If Client has to request secure connection, what is going to stop a rogue 
client establishing connection with Bookie and continue in that way? Is your 
plan to make use of Authentication + StatTLS to avoid STRIPTLS attack? 
(https://en.wikipedia.org/wiki/Opportunistic_TLS#Weaknesses_and_mitigations)
{quote}
We will implement configuration flags to prevent unsecured connections. 
The ConnectionPeer interface introduced with  BOOKKEEPER-959  will be enhanced 
with a boolean "isSecure()" information in order to known that the comunication 
is encrypted. So the AuthPlugin will be able to deny authentication for every 
connection which did not run the StartTLS protocol.

For the mutual authentication case this can be  implemented even on the 
AuthPlugin as the AuthPlugin will be aware of the certificates which are 
exchanged on the wire. Bookies do no allow requests from Clients which are not 
in the 'authenticated' state and the SSL AuthPlugin will block clients without 
certificate (depending on the configuration)
So we can implement an option "allow only secure connections".

The STRIPTLS attack will not be our case, infact we can make the client close 
any connection to Bookies which do not accept StartTLS at all. In the SMTP