[
https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240313#comment-13240313
]
[email protected] commented on BOOKKEEPER-193:
----------------------------------------------------------
bq. On 2012-03-28 08:10:21, fpj wrote:
bq. >
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java,
line 124
bq. > <https://reviews.apache.org/r/4481/diff/1/?file=95778#file95778line124>
bq. >
bq. > Can we perhaps use another latch here instead of time? Relying on
time doesn't always work and in many cases it will induce unnecessary waiting
time.
agreed. it could use another latch. will fix it in new patch.
bq. On 2012-03-28 08:10:21, fpj wrote:
bq. >
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java,
line 37
bq. > <https://reviews.apache.org/r/4481/diff/1/?file=95777#file95777line37>
bq. >
bq. > Possibly not a big deal, but I wonder if it is really necessary to
use this lock. In the way I read the code, I think it would work without it,
but perhaps I'm missing something important.
the lock is used to avoid modifying two variables updates & updatesToMerge map,
which is used to avoid inserting to updatesToMerge map during snapshot.
suppose two thread, one is doing insertion, the other one is done snapshot.
1) insertion thread: get the reference of updates map, tried to call #put.
2) snapshot thread: swap updates map to updatesToMerge map.
3) snapshot thread: iterates over the updatesToMerge map to merge them to
snapshot map. (it may take times)
4) insertion thread: execute put. since updates map object has been assigned to
updatesToMerge, so the put actually is applied in updatesToMerge, if the
insertion position is large than the current position of iteration, it is OK.
otherwise, the insertion will be lost.
the readwrite lock only blocks when modifying the updates & updatesToMerge
references, I think it would not be expensive.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4481/#review6481
-----------------------------------------------------------
On 2012-03-26 14:45:30, Sijie Guo wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4481/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-26 14:45:30)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. create a snapshot of bookie active ledgers, and then fetch the list of
zookeeper metadata, then it is safe to do garbage collection.
bq.
bq.
bq. This addresses bug BOOKKEEPER-193.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-193
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java
PRE-CREATION
bq.
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java
PRE-CREATION
bq.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java
PRE-CREATION
bq.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java
9d9bf22
bq.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java
100cdad
bq.
bq. Diff: https://reviews.apache.org/r/4481/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sijie
bq.
bq.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch
>
>
> currently, we encountered such case: ledger is garbage collected by mistake,
> and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO
> [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting
> deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] -
> Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] -
> Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] -
> Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at
> org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at
> org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at
> org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at
> org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at
> org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at
> org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at
> org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at
> org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at
> org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
