Jenkins build is back to stable : distributedlog-nightly-build #487

[Apache Jenkins Server]

See 

[GitHub] jiazhai opened a new issue #726: Unit Tests failure with BKMetadataVersionException

[GitBox]

jiazhai opened a new issue #726: Unit Tests failure with 
BKMetadataVersionException
URL: https://github.com/apache/bookkeeper/issues/726
 
 
   **BUG REPORT**
   
   1. Please describe the issue you observed:
   
   - What did you do?
   Run mvn test
   - What did you expect to see?
   Expect all tests passed
   - What did you see instead?
   some tests failed, with Exception: BKMetadataVersionException
   Failure cases:
   ```
 BookieRecoveryTest.testMetadataConflictWhenDelayingEnsembleChange ? 
BKMetadataVersion
 BookieRecoveryTest.testMetadataConflictWithRecovery ? BKMetadataVersion 
Bad le...
 BookieRecoveryTest.testSyncBookieRecoveryToRandomBookiesCheckForDupes ? 
BKMetadataVersion
 TestDelayEnsembleChange.testChangeEnsembleSecondBookieReadOnly ? 
BKMetadataVersion
 TestWatchEnsembleChange.testWatchEnsembleChange[1] ? BKMetadataVersion Bad 
led...
 UpdateLedgerCmdTest.testUpdateLedgersToHostname ? BKMetadataVersion Bad 
ledger...
 UpdateLedgerOpTest.testChangeEnsembleAfterRenaming:202 ? BKMetadataVersion 
Bad...
 UpdateLedgerOpTest.testLimitLessThanTotalLedgers ? BKMetadataVersion Bad 
ledge...
 UpdateLedgerOpTest.testManyLedgers ? BKMetadataVersion Bad ledger metadata 
ver...
 UpdateLedgerOpTest.testRenameWhenAddEntryInProgress ? BKMetadataVersion 
Bad le...
   ```
   
   get one stack example here:
   ```
2017-11-15 13:00:11,838 - ERROR - 
[BookKeeperClientWorker-OrderedSafeExecutor-0-0:PendingAddOp@354] - Write of 
ledger entry to quorum failed: L0 E10
org.apache.bookkeeper.client.BKException$BKMetadataVersionException: Bad 
ledger metadata version
at 
org.apache.bookkeeper.client.SyncCallbackUtils.finish(SyncCallbackUtils.java:68)
at 
org.apache.bookkeeper.client.SyncCallbackUtils$SyncAddCallback.addComplete(SyncCallbackUtils.java:264)
at 
org.apache.bookkeeper.client.PendingAddOp.submitCallback(PendingAddOp.java:359)
at 
org.apache.bookkeeper.client.LedgerHandle.errorOutPendingAdds(LedgerHandle.java:1300)
at 
org.apache.bookkeeper.client.LedgerHandle$3.safeRun(LedgerHandle.java:427)
at 
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
   ```
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie closed issue #613: Bookie: LRU FileInfo Eviction Policy

[GitBox]

sijie closed issue #613: Bookie: LRU FileInfo Eviction Policy
URL: https://github.com/apache/bookkeeper/issues/613
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #613: Bookie: LRU FileInfo Eviction Policy

[GitBox]

sijie commented on issue #613: Bookie: LRU FileInfo Eviction Policy
URL: https://github.com/apache/bookkeeper/issues/613#issuecomment-344420235
 
 
   This issue is already addressed by #513 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150927573
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
 
 Review comment:
   If hostname is used and the IP cookie exists, the verification will already 
fail immediately at the first step. The second boot will never succeed. If you 
take a look at the test case changes, it covers your case described here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on issue #712: Issue 544: Bootup cookie validation considers an 
empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#issuecomment-344357571
 
 
   @ivankelly I checked your changes. Frankly speaking, I don't see any real 
difference comparing to current pull request.  However I am fine with those 
renames. Just one question, why do you remove the return in the middle? It is a 
stop if the environment is healthy, you shouldn't run the fixing logic.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150924748
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
 
 Review comment:
   btw, I think a good solution would be to just delete the old cookie before 
creating the new one, using a multi().


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150923743
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
 
 Review comment:
   Say you have bookie, ip is 10.0.0.1, hostname is foobar.
   
   First boot it comes up with setUseHostNameAsBookieID(false), writes its 
cookie to /ledgers/cookies/10.0.0.1:3181.
   Second boot some disks are added, allow expansion is enabled and 
setUseHostNameAsBookieID is set to true. In this case, it reads a cookie from 
/ledgers/cookies/10.0.0.1:3181, updates the cookie for the new storage, writes 
to all the dirs and then tries to write the cookie to 
/ledgers/cookies/foobar:3181. However, this last write, it will use the version 
it read from  /ledgers/cookies/10.0.0.1:3181, which is wrong (it shouldn't even 
be a setData, it should be a create).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150916980
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
 
 Review comment:
   How's that possible after this change? Can you explain the sequence causing 
this bug? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150808708
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
 
 Review comment:
   @sijie a bug. If there is a change in configuration, it is possible that we 
will write to a different znode than the one we read rmCookie from in this 
method. However, we will use the Version returned by this method, which will 
result in a BadVersionException.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on issue #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on issue #712: Issue 544: Bootup cookie validation 
considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#issuecomment-344231541
 
 
   I created a pull request into your branch.
   
   https://github.com/sijie/bookkeeper/pull/1
   
   But alas, lest we think the end is near, there is a bug (will comment now).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on issue #553: Documentation for new API

[GitBox]

sijie commented on issue #553: Documentation for new API
URL: https://github.com/apache/bookkeeper/issues/553#issuecomment-344217221
 
 
   @eolivelli we can only remove old API after we deprecate the old api.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150789467
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
+  
ServerConfiguration conf,
+  
List dirs)
+throws InvalidCookieException, IOException {
+List missingDirs = Lists.newArrayList();
+List existedCookies = Lists.newArrayList();
+for (File dir : dirs) {
+checkDirectoryStructure(dir);
+try {
+Cookie c = Cookie.readFromDirectory(dir);
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(c);
+} else {
+masterCookie.verify(c);
+}
+existedCookies.add(c);
+} catch (FileNotFoundException fnf) {
+missingDirs.add(dir);
+}
+}
+return Pair.of(missingDirs, existedCookies);
+}
+
 public static void checkEnvironmentWithStorageExpansion(
 ServerConfiguration conf,
 RegistrationManager rm,
 List journalDirectories,
 List allLedgerDirs) throws BookieException {
 try {
-boolean newEnv = false;
-List missedCookieDirs = new ArrayList();
-List journalCookies = Lists.newArrayList();
-// try to read cookie from journal directory.
-for (File journalDirectory : journalDirectories) {
-try {
-Cookie journalCookie = 
Cookie.readFromDirectory(journalDirectory);
-journalCookies.add(journalCookie);
-if (journalCookie.isBookieHostCreatedFromIp()) {
-conf.setUseHostNameAsBookieID(false);
-} else {
-conf.setUseHostNameAsBookieID(true);
-}
-} catch (FileNotFoundException fnf) {
-newEnv = true;
-missedCookieDirs.add(journalDirectory);
-}
-}
-
+// 1. retrieve the instance id
 String instanceId = rm.getClusterInstanceId();
+
+// 2. build the master cookie from the configuration
 Cookie.Builder builder = Cookie.generateCookie(conf);
 if (null != instanceId) {
 builder.setInstanceId(instanceId);
 }
 Cookie masterCookie = builder.build();
-Versioned rmCookie = null;
-   

[GitHub] sijie commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150787234
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
 
 Review comment:
   if a new directory was not written with a cookie during storage expansion. 
the storage expansion will continue to fix it eventually. if an existing dir's 
cookie is not updated during storage expansion, there are two cases : 1) a 
bookie starts again without expansion, that the verification will fail 2) a 
bookie starts again with expansion, it is okay to continue as well because the 
master cookie and rm cookie validation is still valid (because of the 
superset). that's still a healthy environment. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150785358
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
+  
ServerConfiguration conf,
+  
List dirs)
+throws InvalidCookieException, IOException {
+List missingDirs = Lists.newArrayList();
+List existedCookies = Lists.newArrayList();
+for (File dir : dirs) {
+checkDirectoryStructure(dir);
+try {
+Cookie c = Cookie.readFromDirectory(dir);
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(c);
+} else {
+masterCookie.verify(c);
+}
+existedCookies.add(c);
+} catch (FileNotFoundException fnf) {
+missingDirs.add(dir);
+}
+}
+return Pair.of(missingDirs, existedCookies);
+}
+
 public static void checkEnvironmentWithStorageExpansion(
 ServerConfiguration conf,
 RegistrationManager rm,
 List journalDirectories,
 List allLedgerDirs) throws BookieException {
 try {
-boolean newEnv = false;
-List missedCookieDirs = new ArrayList();
-List journalCookies = Lists.newArrayList();
-// try to read cookie from journal directory.
-for (File journalDirectory : journalDirectories) {
-try {
-Cookie journalCookie = 
Cookie.readFromDirectory(journalDirectory);
-journalCookies.add(journalCookie);
-if (journalCookie.isBookieHostCreatedFromIp()) {
-conf.setUseHostNameAsBookieID(false);
-} else {
-conf.setUseHostNameAsBookieID(true);
-}
-} catch (FileNotFoundException fnf) {
-newEnv = true;
-missedCookieDirs.add(journalDirectory);
-}
-}
-
+// 1. retrieve the instance id
 String instanceId = rm.getClusterInstanceId();
+
+// 2. build the master cookie from the configuration
 Cookie.Builder builder = Cookie.generateCookie(conf);
 if (null != instanceId) {
 builder.setInstanceId(instanceId);
 }
 Cookie masterCookie = builder.build();
-Versioned rmCookie = null;
-   

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150784588
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
+  
ServerConfiguration conf,
+  
List dirs)
+throws InvalidCookieException, IOException {
+List missingDirs = Lists.newArrayList();
+List existedCookies = Lists.newArrayList();
+for (File dir : dirs) {
+checkDirectoryStructure(dir);
+try {
+Cookie c = Cookie.readFromDirectory(dir);
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(c);
+} else {
+masterCookie.verify(c);
+}
+existedCookies.add(c);
+} catch (FileNotFoundException fnf) {
+missingDirs.add(dir);
+}
+}
+return Pair.of(missingDirs, existedCookies);
+}
+
 public static void checkEnvironmentWithStorageExpansion(
 ServerConfiguration conf,
 RegistrationManager rm,
 List journalDirectories,
 List allLedgerDirs) throws BookieException {
 try {
-boolean newEnv = false;
-List missedCookieDirs = new ArrayList();
-List journalCookies = Lists.newArrayList();
-// try to read cookie from journal directory.
-for (File journalDirectory : journalDirectories) {
-try {
-Cookie journalCookie = 
Cookie.readFromDirectory(journalDirectory);
-journalCookies.add(journalCookie);
-if (journalCookie.isBookieHostCreatedFromIp()) {
-conf.setUseHostNameAsBookieID(false);
-} else {
-conf.setUseHostNameAsBookieID(true);
-}
-} catch (FileNotFoundException fnf) {
-newEnv = true;
-missedCookieDirs.add(journalDirectory);
-}
-}
-
+// 1. retrieve the instance id
 String instanceId = rm.getClusterInstanceId();
+
+// 2. build the master cookie from the configuration
 Cookie.Builder builder = Cookie.generateCookie(conf);
 if (null != instanceId) {
 builder.setInstanceId(instanceId);
 }
 Cookie masterCookie = builder.build();
-Versioned rmCookie = null;
-   

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150781772
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
+  
ServerConfiguration conf,
+  
List dirs)
+throws InvalidCookieException, IOException {
+List missingDirs = Lists.newArrayList();
+List existedCookies = Lists.newArrayList();
+for (File dir : dirs) {
+checkDirectoryStructure(dir);
+try {
+Cookie c = Cookie.readFromDirectory(dir);
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(c);
+} else {
+masterCookie.verify(c);
+}
+existedCookies.add(c);
+} catch (FileNotFoundException fnf) {
+missingDirs.add(dir);
+}
+}
+return Pair.of(missingDirs, existedCookies);
+}
+
 public static void checkEnvironmentWithStorageExpansion(
 ServerConfiguration conf,
 RegistrationManager rm,
 List journalDirectories,
 List allLedgerDirs) throws BookieException {
 try {
-boolean newEnv = false;
-List missedCookieDirs = new ArrayList();
-List journalCookies = Lists.newArrayList();
-// try to read cookie from journal directory.
-for (File journalDirectory : journalDirectories) {
-try {
-Cookie journalCookie = 
Cookie.readFromDirectory(journalDirectory);
-journalCookies.add(journalCookie);
-if (journalCookie.isBookieHostCreatedFromIp()) {
-conf.setUseHostNameAsBookieID(false);
-} else {
-conf.setUseHostNameAsBookieID(true);
-}
-} catch (FileNotFoundException fnf) {
-newEnv = true;
-missedCookieDirs.add(journalDirectory);
-}
-}
-
+// 1. retrieve the instance id
 String instanceId = rm.getClusterInstanceId();
+
+// 2. build the master cookie from the configuration
 Cookie.Builder builder = Cookie.generateCookie(conf);
 if (null != instanceId) {
 builder.setInstanceId(instanceId);
 }
 Cookie masterCookie = builder.build();
-Versioned rmCookie = null;
-   

[GitHub] sijie commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

sijie commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150778486
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
+  
ServerConfiguration conf,
+  
List dirs)
+throws InvalidCookieException, IOException {
+List missingDirs = Lists.newArrayList();
+List existedCookies = Lists.newArrayList();
+for (File dir : dirs) {
+checkDirectoryStructure(dir);
+try {
+Cookie c = Cookie.readFromDirectory(dir);
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(c);
+} else {
+masterCookie.verify(c);
+}
+existedCookies.add(c);
+} catch (FileNotFoundException fnf) {
+missingDirs.add(dir);
+}
+}
+return Pair.of(missingDirs, existedCookies);
+}
+
 public static void checkEnvironmentWithStorageExpansion(
 ServerConfiguration conf,
 RegistrationManager rm,
 List journalDirectories,
 List allLedgerDirs) throws BookieException {
 try {
-boolean newEnv = false;
-List missedCookieDirs = new ArrayList();
-List journalCookies = Lists.newArrayList();
-// try to read cookie from journal directory.
-for (File journalDirectory : journalDirectories) {
-try {
-Cookie journalCookie = 
Cookie.readFromDirectory(journalDirectory);
-journalCookies.add(journalCookie);
-if (journalCookie.isBookieHostCreatedFromIp()) {
-conf.setUseHostNameAsBookieID(false);
-} else {
-conf.setUseHostNameAsBookieID(true);
-}
-} catch (FileNotFoundException fnf) {
-newEnv = true;
-missedCookieDirs.add(journalDirectory);
-}
-}
-
+// 1. retrieve the instance id
 String instanceId = rm.getClusterInstanceId();
+
+// 2. build the master cookie from the configuration
 Cookie.Builder builder = Cookie.generateCookie(conf);
 if (null != instanceId) {
 builder.setInstanceId(instanceId);
 }
 Cookie masterCookie = builder.build();
-Versioned rmCookie = null;
-   

[GitHub] jiazhai closed pull request #725: Issue 724: fix wrong link for slack signup

[GitBox]

jiazhai closed pull request #725: Issue 724: fix wrong link for slack signup
URL: https://github.com/apache/bookkeeper/pull/725
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/site/community/slack.md b/site/community/slack.md
index bb1288646..291082572 100644
--- a/site/community/slack.md
+++ b/site/community/slack.md
@@ -6,4 +6,4 @@ There is an [Apache 
BookKeeper](http://apachebookkeeper.slack.com/) channel that
 
 The Slack channel is at 
[http://apachebookkeeper.slack.com/](http://apachebookkeeper.slack.com/).
 
-You can self-register at 
[https://apachebookkeeper.herokuapp.com/](http://apachebookkeeper.slack.com/).
+You can self-register at 
[https://apachebookkeeper.herokuapp.com/](https://apachebookkeeper.herokuapp.com/).


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150778031
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
 
 Review comment:
   > Thas was what I explained. We don't need manual intervention (and it makes 
things become complicated). A restart of the bookie should continue with 
storage expansion and succeed eventually.
   
   This won't be expansion though, because if the cookie exists, then the 
directory is non-empty so can't be used for expansion.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jiazhai closed issue #724: Slack signup link is wrong

[GitBox]

jiazhai closed issue #724: Slack signup link is wrong
URL: https://github.com/apache/bookkeeper/issues/724
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eolivelli commented on issue #553: Documentation for new API

[GitBox]

eolivelli commented on issue #553: Documentation for new API
URL: https://github.com/apache/bookkeeper/issues/553#issuecomment-344195264
 
 
   great, I will add some paragraphs, as the new API is still "experimental" is 
it better to leave the old API on foreground, OK ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie validation considers an empty journal to signify a new bookie

[GitBox]

ivankelly commented on a change in pull request #712: Issue 544: Bootup cookie 
validation considers an empty journal to signify a new bookie
URL: https://github.com/apache/bookkeeper/pull/712#discussion_r150773274
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -441,117 +335,187 @@ private void checkIfDirsOnSameDiskPartition(List 
dirs) throws DiskPartitio
 }
 }
 
+static Versioned readAndVerifyCookieFromRegistrationManager(Cookie 
masterCookie,
+
ServerConfiguration conf,
+
RegistrationManager rm)
+throws BookieException {
+// we need to loop through all possible bookie identifiers to ensure 
it is treated as a new environment
+// just because of bad configuration
+List addresses = 
Lists.newArrayListWithExpectedSize(3);
+try {
+// ip address
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(false).setAdvertisedAddress(null)));
+// host name
+addresses.add(getBookieAddress(
+new 
ServerConfiguration(conf).setUseHostNameAsBookieID(true).setAdvertisedAddress(null)));
+// advertised address
+if (null != conf.getAdvertisedAddress()) {
+addresses.add(getBookieAddress(conf));
+}
+} catch (UnknownHostException e) {
+throw new UnknownBookieIdException(e);
+}
+Versioned rmCookie = null;
+for (BookieSocketAddress address : addresses) {
+try {
+rmCookie = Cookie.readFromRegistrationManager(rm, address);
+// If allowStorageExpansion option is set, we should
+// make sure that the new set of ledger/index dirs
+// is a super set of the old; else, we fail the cookie check
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(rmCookie.getValue());
+} else {
+masterCookie.verify(rmCookie.getValue());
+}
+} catch (CookieNotFoundException e) {
+continue;
+}
+}
+return rmCookie;
+}
+
+private static Pair 
verifyAndGetMissingDirs(Cookie masterCookie,
+  
ServerConfiguration conf,
+  
List dirs)
+throws InvalidCookieException, IOException {
+List missingDirs = Lists.newArrayList();
+List existedCookies = Lists.newArrayList();
+for (File dir : dirs) {
+checkDirectoryStructure(dir);
+try {
+Cookie c = Cookie.readFromDirectory(dir);
+if (conf.getAllowStorageExpansion()) {
+masterCookie.verifyIsSuperSet(c);
+} else {
+masterCookie.verify(c);
+}
+existedCookies.add(c);
+} catch (FileNotFoundException fnf) {
+missingDirs.add(dir);
+}
+}
+return Pair.of(missingDirs, existedCookies);
+}
+
 public static void checkEnvironmentWithStorageExpansion(
 ServerConfiguration conf,
 RegistrationManager rm,
 List journalDirectories,
 List allLedgerDirs) throws BookieException {
 try {
-boolean newEnv = false;
-List missedCookieDirs = new ArrayList();
-List journalCookies = Lists.newArrayList();
-// try to read cookie from journal directory.
-for (File journalDirectory : journalDirectories) {
-try {
-Cookie journalCookie = 
Cookie.readFromDirectory(journalDirectory);
-journalCookies.add(journalCookie);
-if (journalCookie.isBookieHostCreatedFromIp()) {
-conf.setUseHostNameAsBookieID(false);
-} else {
-conf.setUseHostNameAsBookieID(true);
-}
-} catch (FileNotFoundException fnf) {
-newEnv = true;
-missedCookieDirs.add(journalDirectory);
-}
-}
-
+// 1. retrieve the instance id
 String instanceId = rm.getClusterInstanceId();
+
+// 2. build the master cookie from the configuration
 Cookie.Builder builder = Cookie.generateCookie(conf);
 if (null != instanceId) {
 builder.setInstanceId(instanceId);
 }
 Cookie masterCookie = builder.build();
-Versioned rmCookie = null;
-   

[GitHub] jiazhai commented on issue #724: Slack signup link is wrong

[GitBox]

jiazhai commented on issue #724: Slack signup link is wrong
URL: https://github.com/apache/bookkeeper/issues/724#issuecomment-344191812
 
 
   Thanks @sijie for reporting this. PR #725 opened for it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] eolivelli commented on issue #553: Documentation for new API

[GitBox]

eolivelli commented on issue #553: Documentation for new API
URL: https://github.com/apache/bookkeeper/issues/553#issuecomment-344191519
 
 
   I will do my best.
   Can you give some hint?
   I think that a bunch of examples and a link to javadocs (how can I do this?) 
will be enough


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie opened a new issue #724: Slack signup link is wrong

[GitBox]

sijie opened a new issue #724: Slack signup link is wrong
URL: https://github.com/apache/bookkeeper/issues/724
 
 
   
   **BUG REPORT**
   
   1. Please describe the issue you observed:
   
   - What did you do?
   
   Open https://bookkeeper.apache.org/community/slack/ and click the 
self-register link.
   
   - What did you expect to see?
   
   I should see the signup link not the signin link.
   
   - What did you see instead?
   
   I saw the signin link. The link of `https://apachebookkeeper.herokuapp.com/` 
was pointed to `http://apachebookkeeper.slack.com/`.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sijie opened a new issue #723: Storing ledger metadata in binary format

[GitBox]

sijie opened a new issue #723: Storing ledger metadata in binary format
URL: https://github.com/apache/bookkeeper/issues/723
 
 
   
   
   **FEATURE REQUEST**
   
   1. Please describe the feature you are requesting.
   
   Currently we stores ledger metadata in TextFormat protobuf not in binary 
format. Text format doesn't have a good backwards and forwards compatibility 
story as binary format. We should switch from text format to binary format. 
   
   In order to keep backward compatibility and maintain good debuggability as 
text format, we can consider writing both text format and binary format.
   
   2. Indicate the importance of this issue to you (blocker, must-have, 
should-have, nice-to-have). Are you currently using any workarounds to address 
this issue?
   
   *must-have* in 4.7
   
   3. Provide any additional detail on your proposed use case for this feature.
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Jenkins build is still unstable: bookkeeper-release-nightly-snapshot #102

[Apache Jenkins Server]

See