HashZhang opened a new issue #2747:
URL: https://github.com/apache/bookkeeper/issues/2747
There is a little inconsistency in the last confirmed fetch while ledger is
open. for example:
I have 4 bookies run in local, their configurations are:
**bk_server_1.conf**
```
bookiePort=3181
advertisedAddress=127.0.0.1
allowLoopback=true
httpServerEnabled=false
httpServerPort=8080
httpServerClass=org.apache.bookkeeper.http.vertx.VertxHttpServer
journalDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-1
ledgerDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-data-1
metadataServiceUri=zk+hierarchical://localhost:2181/ledgers
zkEnableSecurity=false
storageserver.grpc.port=4181
dlog.bkcEnsembleSize=3
dlog.bkcWriteQuorumSize=2
dlog.bkcAckQuorumSize=2
storage.range.store.dirs=d:\\github\\apache-bookkeeper\\data\\bk-ranges-1
storage.serve.readonly.tables=false
storage.cluster.controller.schedule.interval.ms=30000
```
**bk_server_2.conf**
```
bookiePort=3182
advertisedAddress=127.0.0.1
allowLoopback=true
httpServerEnabled=false
httpServerPort=8081
httpServerClass=org.apache.bookkeeper.http.vertx.VertxHttpServer
journalDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-2
ledgerDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-data-2
metadataServiceUri=zk+hierarchical://localhost:2181/ledgers
zkEnableSecurity=false
storageserver.grpc.port=4182
dlog.bkcEnsembleSize=3
dlog.bkcWriteQuorumSize=2
dlog.bkcAckQuorumSize=2
storage.range.store.dirs=d:\\github\\apache-bookkeeper\\data\\bk-ranges-2
storage.serve.readonly.tables=false
storage.cluster.controller.schedule.interval.ms=30000
```
**bk_server_3.conf**
```
bookiePort=3183
advertisedAddress=127.0.0.1
allowLoopback=true
httpServerEnabled=false
httpServerPort=8082
httpServerClass=org.apache.bookkeeper.http.vertx.VertxHttpServer
journalDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-3
ledgerDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-data-3
metadataServiceUri=zk+hierarchical://localhost:2181/ledgers
zkEnableSecurity=false
storageserver.grpc.port=4183
dlog.bkcEnsembleSize=3
dlog.bkcWriteQuorumSize=2
dlog.bkcAckQuorumSize=2
storage.range.store.dirs=d:\\github\\apache-bookkeeper\\data\\bk-ranges-3
storage.serve.readonly.tables=false
storage.cluster.controller.schedule.interval.ms=30000
```
**bk_server_4.conf**
```
bookiePort=3184
advertisedAddress=127.0.0.1
allowLoopback=true
httpServerEnabled=false
httpServerPort=8083
httpServerClass=org.apache.bookkeeper.http.vertx.VertxHttpServer
journalDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-4
ledgerDirectories=d:\\github\\apache-bookkeeper\\data\\bk-server-data-4
metadataServiceUri=zk+hierarchical://localhost:2181/ledgers
zkEnableSecurity=false
storageserver.grpc.port=4184
dlog.bkcEnsembleSize=3
dlog.bkcWriteQuorumSize=2
dlog.bkcAckQuorumSize=2
storage.range.store.dirs=d:\\github\\apache-bookkeeper\\data\\bk-ranges-4
storage.serve.readonly.tables=false
storage.cluster.controller.schedule.interval.ms=30000
```
**enable these bookies and I have application shown below**:
```
BookKeeper bookKeeper = new BookKeeper("localhost:2181");
WriteHandle writeHandle = bookKeeper.newCreateLedgerOp()
.withEnsembleSize(3)
.withWriteQuorumSize(2)
.withAckQuorumSize(2)
.withDigestType(DigestType.CRC32C)
.withPassword("test".getBytes(StandardCharsets.UTF_8))
.execute().get();
long ledgerId = writeHandle.getId();
long entryId =
writeHandle.append("message1".getBytes(StandardCharsets.UTF_8));
System.out.println(entryId + " appended");
entryId =
writeHandle.append(ByteBuffer.wrap("message2".getBytes(StandardCharsets.UTF_8)));
System.out.println(entryId + " appended");
entryId =
writeHandle.append(Unpooled.wrappedBuffer("message3".getBytes(StandardCharsets.UTF_8)));
System.out.println(entryId + " appended");
ByteBuf byteBuf = PooledByteBufAllocator.DEFAULT.directBuffer();
byteBuf.writeBytes("message4".getBytes(StandardCharsets.UTF_8));
System.out.println("sleep start!");
//during sleep time, shut down one of the ensemble
TimeUnit.SECONDS.sleep(10);
entryId = writeHandle.append(byteBuf);
// writeHandle.close();
ReadHandle readHandle = bookKeeper.newOpenLedgerOp()
.withLedgerId(ledgerId)
.withDigestType(DigestType.CRC32C)
.withPassword("test".getBytes(StandardCharsets.UTF_8))
.execute().get();
LedgerEntries read = readHandle.read(0, 3);
read.forEach(ledgerEntry -> {
byte[] entryBytes = ledgerEntry.getEntryBytes();
System.out.println(new String(entryBytes,
StandardCharsets.UTF_8));
});
```
During sleep time, shut down one of the ensemble. Then during read operation
I would got Exception says:
```
2021-06-29 09:29:09,224 - ERROR - [main:LedgerHandle@754] - ReadAsync
exception on ledgerId:19 firstEntry:0 lastEntry:3 lastAddConfirmed:2
Exception in thread "main"
org.apache.bookkeeper.client.BKException$BKReadException: Error while reading
ledger
at
org.apache.bookkeeper.client.LedgerHandle.readAsync(LedgerHandle.java:756)
at org.apache.bookkeeper.client.api.ReadHandle.read(ReadHandle.java:58)
at org.apache.bookkeeper.TestClient.main(TestClient.java:74)
```
that is because the ledger is not closed and client would fetch last
confirmed from one of the ensemble bookies, but the write quorum is 2. if the
fetch request is sent to the bookie where message4 is not placed on, message4
cannot be read although is it confirmed. Therefore I think the Lac fetch
request should be sent to all bookies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
