[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r202215167 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java ## @@ -206,9 +218,11 @@ public synchronized void readHeader() throws IOException { throw new IOException("Missing ledger signature while reading header for " + lf); } int version = bb.getInt(); -if (version != HEADER_VERSION) { +if (version > CURRENT_HEADER_VERSION) { throw new IOException("Incompatible ledger version " + version + " while reading header for " + lf); Review comment: This code change is fine. I spoke to @sijie and @eolivelli this morning. We needed a bigger gate to identify ondisk changes. I will open an issue for that. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r202215167 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java ## @@ -206,9 +218,11 @@ public synchronized void readHeader() throws IOException { throw new IOException("Missing ledger signature while reading header for " + lf); } int version = bb.getInt(); -if (version != HEADER_VERSION) { +if (version > CURRENT_HEADER_VERSION) { throw new IOException("Incompatible ledger version " + version + " while reading header for " + lf); Review comment: This code change is fine. I spoke to @sijie and @eolivelli this morning. We needed a bigger gate to identify ondisk changes. I will open a issue for that. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r202215064 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ## @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, ByteBuffer recBuff) throws + " but layout version (" + journalVersion + ") is too old to hold this"); } +} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) { +if (journalVersion >= JournalChannel.V5) { +int explicitLacBufLength = recBuff.getInt(); +ByteBuf explicitLacBuf = Unpooled.buffer(explicitLacBufLength); +byte[] explicitLacBufArray = new byte[explicitLacBufLength]; +recBuff.get(explicitLacBufArray); +explicitLacBuf.writeBytes(explicitLacBufArray); +byte[] key = masterKeyCache.get(ledgerId); +if (key == null) { +key = ledgerStorage.readMasterKey(ledgerId); +} +LedgerDescriptor handle = handles.getHandle(ledgerId, key); +handle.setExplicitLac(explicitLacBuf); +} else { +throw new IOException("Invalid journal. Contains explicitLAC " + " but layout version (" ++ journalVersion + ") is too old to hold this"); +} } else { Review comment: I mean any special entryIds are going to be treated as regular entryIds in this case. So my request is to create a mask to identify SPECIAL entries and filter them out. If there is anything special that the code doesn't know, flag an error and continue. I know this doesn't solve the previous release but future code changes should be good. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201737053 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java ## @@ -206,9 +218,11 @@ public synchronized void readHeader() throws IOException { throw new IOException("Missing ledger signature while reading header for " + lf); } int version = bb.getInt(); -if (version != HEADER_VERSION) { +if (version > CURRENT_HEADER_VERSION) { throw new IOException("Incompatible ledger version " + version + " while reading header for " + lf); Review comment: So this prevents software rollback period. isn't it? Should we consider coming up ? what are the downsides of that? yeah we may not read the explicitLAC other than that, is there anything else? I am worried if there are any situations where we make the cluster not usable. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201735151 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ## @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, ByteBuffer recBuff) throws + " but layout version (" + journalVersion + ") is too old to hold this"); } +} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) { +if (journalVersion >= JournalChannel.V5) { +int explicitLacBufLength = recBuff.getInt(); +ByteBuf explicitLacBuf = Unpooled.buffer(explicitLacBufLength); +byte[] explicitLacBufArray = new byte[explicitLacBufLength]; +recBuff.get(explicitLacBufArray); +explicitLacBuf.writeBytes(explicitLacBufArray); +byte[] key = masterKeyCache.get(ledgerId); +if (key == null) { +key = ledgerStorage.readMasterKey(ledgerId); +} +LedgerDescriptor handle = handles.getHandle(ledgerId, key); +handle.setExplicitLac(explicitLacBuf); +} else { +throw new IOException("Invalid journal. Contains explicitLAC " + " but layout version (" ++ journalVersion + ") is too old to hold this"); +} } else { Review comment: What happens in the case of rollback? The software won't understand the special entryId and falls into this section right? Do we handle it right in handle.addentry? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201730658 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ## @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, ByteBuffer recBuff) throws + " but layout version (" + journalVersion + ") is too old to hold this"); } +} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) { +if (journalVersion >= JournalChannel.V5) { +int explicitLacBufLength = recBuff.getInt(); +ByteBuf explicitLacBuf = Unpooled.buffer(explicitLacBufLength); +byte[] explicitLacBufArray = new byte[explicitLacBufLength]; +recBuff.get(explicitLacBufArray); +explicitLacBuf.writeBytes(explicitLacBufArray); +byte[] key = masterKeyCache.get(ledgerId); +if (key == null) { +key = ledgerStorage.readMasterKey(ledgerId); +} +LedgerDescriptor handle = handles.getHandle(ledgerId, key); +handle.setExplicitLac(explicitLacBuf); +} else { +throw new IOException("Invalid journal. Contains explicitLAC " + " but layout version (" ++ journalVersion + ") is too old to hold this"); Review comment: Just curious, if the software is an older version, how can it even recognize that the entryId is METAENTRY_ID_LEDGER_EXPLICITLAC?? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent
jvrao commented on a change in pull request #1532: ISSUE #1527: Make ExplicitLAC persistent URL: https://github.com/apache/bookkeeper/pull/1532#discussion_r201730658 ## File path: bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java ## @@ -784,6 +787,23 @@ public void process(int journalVersion, long offset, ByteBuffer recBuff) throws + " but layout version (" + journalVersion + ") is too old to hold this"); } +} else if (entryId == METAENTRY_ID_LEDGER_EXPLICITLAC) { +if (journalVersion >= JournalChannel.V5) { +int explicitLacBufLength = recBuff.getInt(); +ByteBuf explicitLacBuf = Unpooled.buffer(explicitLacBufLength); +byte[] explicitLacBufArray = new byte[explicitLacBufLength]; +recBuff.get(explicitLacBufArray); +explicitLacBuf.writeBytes(explicitLacBufArray); +byte[] key = masterKeyCache.get(ledgerId); +if (key == null) { +key = ledgerStorage.readMasterKey(ledgerId); +} +LedgerDescriptor handle = handles.getHandle(ledgerId, key); +handle.setExplicitLac(explicitLacBuf); +} else { +throw new IOException("Invalid journal. Contains explicitLAC " + " but layout version (" ++ journalVersion + ") is too old to hold this"); Review comment: Just curious, if the software is an older version, how can it even recognize that the entryId is METAENTRY_ID_LEDGER_EXPLICITLAC?? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services