[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r202215167
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java
 ##
 @@ -206,9 +218,11 @@ public synchronized void readHeader() throws IOException {
 throw new IOException("Missing ledger signature while reading 
header for " + lf);
 }
 int version = bb.getInt();
-if (version != HEADER_VERSION) {
+if (version > CURRENT_HEADER_VERSION) {
 throw new IOException("Incompatible ledger version " + version 
+ " while reading header for " + lf);
 
 Review comment:
   This code change is fine. I spoke to @sijie  and @eolivelli  this morning. 
We needed a bigger gate to identify ondisk changes. I will open an issue for 
that. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r202215167
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java
 ##
 @@ -206,9 +218,11 @@ public synchronized void readHeader() throws IOException {
 throw new IOException("Missing ledger signature while reading 
header for " + lf);
 }
 int version = bb.getInt();
-if (version != HEADER_VERSION) {
+if (version > CURRENT_HEADER_VERSION) {
 throw new IOException("Incompatible ledger version " + version 
+ " while reading header for " + lf);
 
 Review comment:
   This code change is fine. I spoke to @sijie  and @eolivelli  this morning. 
We needed a bigger gate to identify ondisk changes. I will open a issue for 
that. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r202215064
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, 
ByteBuffer recBuff) throws
 + " but layout version (" + journalVersion
 + ") is too old to hold this");
 }
+} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) {
+if (journalVersion >= JournalChannel.V5) {
+int explicitLacBufLength = recBuff.getInt();
+ByteBuf explicitLacBuf = 
Unpooled.buffer(explicitLacBufLength);
+byte[] explicitLacBufArray = new 
byte[explicitLacBufLength];
+recBuff.get(explicitLacBufArray);
+explicitLacBuf.writeBytes(explicitLacBufArray);
+byte[] key = masterKeyCache.get(ledgerId);
+if (key == null) {
+key = ledgerStorage.readMasterKey(ledgerId);
+}
+LedgerDescriptor handle = 
handles.getHandle(ledgerId, key);
+handle.setExplicitLac(explicitLacBuf);
+} else {
+throw new IOException("Invalid journal. Contains 
explicitLAC " + " but layout version ("
++ journalVersion + ") is too old to hold 
this");
+}
 } else {
 
 Review comment:
   I mean any special entryIds are going to be treated as regular entryIds in 
this case. So my request is to create a mask to identify SPECIAL entries and 
filter them out. If there is anything special that the code doesn't know, flag 
an error and continue. I know this doesn't solve the previous release but 
future code changes should be good. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201737053
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java
 ##
 @@ -206,9 +218,11 @@ public synchronized void readHeader() throws IOException {
 throw new IOException("Missing ledger signature while reading 
header for " + lf);
 }
 int version = bb.getInt();
-if (version != HEADER_VERSION) {
+if (version > CURRENT_HEADER_VERSION) {
 throw new IOException("Incompatible ledger version " + version 
+ " while reading header for " + lf);
 
 Review comment:
   So this prevents software rollback period. isn't it? Should we consider 
coming up ? what are the downsides of that? yeah we may not read the 
explicitLAC other than that, is there anything else? I am worried if there are 
any situations where we make the cluster not usable. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201735151
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, 
ByteBuffer recBuff) throws
 + " but layout version (" + journalVersion
 + ") is too old to hold this");
 }
+} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) {
+if (journalVersion >= JournalChannel.V5) {
+int explicitLacBufLength = recBuff.getInt();
+ByteBuf explicitLacBuf = 
Unpooled.buffer(explicitLacBufLength);
+byte[] explicitLacBufArray = new 
byte[explicitLacBufLength];
+recBuff.get(explicitLacBufArray);
+explicitLacBuf.writeBytes(explicitLacBufArray);
+byte[] key = masterKeyCache.get(ledgerId);
+if (key == null) {
+key = ledgerStorage.readMasterKey(ledgerId);
+}
+LedgerDescriptor handle = 
handles.getHandle(ledgerId, key);
+handle.setExplicitLac(explicitLacBuf);
+} else {
+throw new IOException("Invalid journal. Contains 
explicitLAC " + " but layout version ("
++ journalVersion + ") is too old to hold 
this");
+}
 } else {
 
 Review comment:
   What happens in the case of rollback? The software won't understand the 
special entryId and falls into this section right? Do we handle it right in 
handle.addentry?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201730658
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, 
ByteBuffer recBuff) throws
 + " but layout version (" + journalVersion
 + ") is too old to hold this");
 }
+} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) {
+if (journalVersion >= JournalChannel.V5) {
+int explicitLacBufLength = recBuff.getInt();
+ByteBuf explicitLacBuf = 
Unpooled.buffer(explicitLacBufLength);
+byte[] explicitLacBufArray = new 
byte[explicitLacBufLength];
+recBuff.get(explicitLacBufArray);
+explicitLacBuf.writeBytes(explicitLacBufArray);
+byte[] key = masterKeyCache.get(ledgerId);
+if (key == null) {
+key = ledgerStorage.readMasterKey(ledgerId);
+}
+LedgerDescriptor handle = 
handles.getHandle(ledgerId, key);
+handle.setExplicitLac(explicitLacBuf);
+} else {
+throw new IOException("Invalid journal. Contains 
explicitLAC " + " but layout version ("
++ journalVersion + ") is too old to hold 
this");
 
 Review comment:
   Just curious, if the software is an older version, how can it even recognize 
that the entryId is METAENTRY_ID_LEDGER_EXPLICITLAC??


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent

[GitBox]

jvrao commented on a change in pull request #1532: ISSUE #1527: Make 
ExplicitLAC persistent
URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201730658
 
 

 ##
 File path: 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
 ##
 @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, 
ByteBuffer recBuff) throws
 + " but layout version (" + journalVersion
 + ") is too old to hold this");
 }
+} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) {
+if (journalVersion >= JournalChannel.V5) {
+int explicitLacBufLength = recBuff.getInt();
+ByteBuf explicitLacBuf = 
Unpooled.buffer(explicitLacBufLength);
+byte[] explicitLacBufArray = new 
byte[explicitLacBufLength];
+recBuff.get(explicitLacBufArray);
+explicitLacBuf.writeBytes(explicitLacBufArray);
+byte[] key = masterKeyCache.get(ledgerId);
+if (key == null) {
+key = ledgerStorage.readMasterKey(ledgerId);
+}
+LedgerDescriptor handle = 
handles.getHandle(ledgerId, key);
+handle.setExplicitLac(explicitLacBuf);
+} else {
+throw new IOException("Invalid journal. Contains 
explicitLAC " + " but layout version ("
++ journalVersion + ") is too old to hold 
this");
 
 Review comment:
   Just curious, if the software is an older version, how can it even recognize 
that the entryId is METAENTRY_ID_LEDGER_EXPLICITLAC??


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services