[
https://issues.apache.org/jira/browse/AMQ-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760958#comment-16760958
]
Alan Protasio edited comment on AMQ-7080 at 2/5/19 3:54 PM:
------------------------------------------------------------
I'm trying to do it here... but every time that think about i find a case where
i can break it..
Now I was implementing using HashIndex<FreePageKey, Boolean> where FreePageKey
is only a Long with a customized hashCode (and so, we would achieve the same
goal to "restrict the updates to the pages that have changed" - and also "
preallocated block of the first 10 pages or something like that." as the
hashIndex preallocate 'binCapacity'); In this solution i was thinking that for
instances pages from 1-1000 would have the same hash and so storage in the same
Bin (page) inside the HashIndex - so if pages 1-1000 get modified, only the
first page of the index would be written).
Then I created a pageFile Method to update this index:
void updatePageMap(Transaction tx, SequenceSet freeList, SequenceSet
allocatedList)
That was invoked on Transaction#commit
The problem here is the circular self dependency. When I call updatePageMap
inside the transaction.commit, i pass the transaction's free and allocated
pages. But the "updatePageMap" itself can allocate and write more pages using
the same transaction. If updatePageMap allocate one page for instance, the
freeList will have a stale data.
I cannot also create another transaction to call updatePageMap. As this second
transaction itself has to be committed creating another transaction.
To make short...
If i use the same transaction to update the SequenceSet index, I can have stale
data.
Transaction.commit -> UpdateIndex with freePages data (this operation can
allocate or free pages) -> If the UpdateIndex allocated/freed pages then we
need UpdateIndex again.
We can try to update the SequenceSet index until it stabilize and stop freeing
or allocating page (this seems not right).
If i use other transaction to call UpdateIndex, this new transaction will also
be committed creating a new transaction.
This will happen if i use any index implementation. :(
This is the change i was doing. See:
[https://github.com/alanprot/activemq/commit/18730a14db2f88d5a2ac8765fa1de50864fd2c4c#diff-1de6e29861ce2a712a3bd575dd25a013R663]
[https://github.com/alanprot/activemq/commit/18730a14db2f88d5a2ac8765fa1de50864fd2c4c#diff-b36dac7750d0c2eb9cd7e102704bee77R817]
I know you are not comfortable with this change... that's why i'm asking if its
not better put it behind a feature flag (config) and came back with the
data.free in the clean shutdown (so no change at all if this flag is no
enabled).
was (Author: alanprot):
I'm trying to do it here... but every time that think about i find a case where
i can break it..
Now I was implementing using HashIndex<FreePageKey, Boolean> where FreePageKey
is only a Long with a customized hashCode (and so, we would achieve the same
goal to "restrict the updates to the pages that have changed" - and also "
preallocated block of the first 10 pages or something like that." as the
hashIndex preallocate 'binCapacity'); In this solution i was thinking that for
instances pages from 1-1000 would have the same hash and so storage in the same
Bin (page) inside the HashIndex - so if pages 1-1000 get modified, only the
first page of the index would be written).
Then I created a pageFile Method to update this index:
void updatePageMap(Transaction tx, SequenceSet freeList, SequenceSet
allocatedList)
That was invoked on Transaction#commit
The problem here is the circular self dependency. When I call updatePageMap
inside the transaction.commit, i pass the transaction's free and allocated
pages. But the "updatePageMap" itself can allocate and write more pages using
the same transaction. If updatePageMap allocate one page for instance, the
freeList will have a stale data.
I cannot also create another transaction to call updatePageMap. As this second
transaction itself has to be committed creating another transaction.
To make short...
If i use the same transaction to update the SequenceSet index, I can have stale
data.
Transaction.commit -> UpdateIndex with freePages data (this operation can
allocate or free pages) -> If the UpdateIndex allocated/freed pages then we
need UpdateIndex again.
We can try to update the SequenceSet index until it stabilize and stop freeing
or allocating page (this seems not right).
If i use other transaction to call UpdateIndex, this new transaction will also
be committed creating a new transaction.
This will happen if i use any index implementation. :(
This is the change i was doing. See:
[https://github.com/alanprot/activemq/commit/18730a14db2f88d5a2ac8765fa1de50864fd2c4c#diff-1de6e29861ce2a712a3bd575dd25a013R663]
[https://github.com/alanprot/activemq/commit/18730a14db2f88d5a2ac8765fa1de50864fd2c4c#diff-b36dac7750d0c2eb9cd7e102704bee77R817]
> Keep track of free pages - Update db.free file during checkpoints
> -----------------------------------------------------------------
>
> Key: AMQ-7080
> URL: https://issues.apache.org/jira/browse/AMQ-7080
> Project: ActiveMQ
> Issue Type: Improvement
> Components: KahaDB
> Affects Versions: 5.15.6
> Reporter: Alan Protasio
> Assignee: Jean-Baptiste Onofré
> Priority: Major
> Fix For: 5.16.0
>
> Attachments: AMQ-7080-freeList-update.diff
>
>
> In a event of an unclean shutdown, Activemq loses the information about the
> free pages in the index. In order to recover this information, ActiveMQ read
> the whole index during shutdown searching for free pages and then save the
> db.free file. This operation can take a long time, making the failover
> slower. (during the shutdown, activemq will still hold the lock).
> From http://activemq.apache.org/shared-file-system-master-slave.html
> {quote}"If you have a SAN or shared file system it can be used to provide
> high availability such that if a broker is killed, another broker can take
> over immediately."
> {quote}
> Is important to note if the shutdown takes more than ACTIVEMQ_KILL_MAXSECONDS
> seconds, any following shutdown will be unclean. This broker will stay in
> this state unless the index is deleted (this state means that every failover
> will take more then ACTIVEMQ_KILL_MAXSECONDS, so, if you increase this time
> to 5 minutes, you fail over can take more than 5 minutes).
>
> In order to prevent ActiveMQ reading the whole index file to search for free
> pages, we can keep track of those on every Checkpoint. In order to do that we
> need to be sure that db.data and db.free are in sync. To achieve that we can
> have a attribute in the db.free page that is referenced by the db.data.
> So during the checkpoint we have:
> 1 - Save db.free and give a freePageUniqueId
> 2 - Save this freePageUniqueId in the db.data (metadata)
> In a crash, we can see if the db.data has the same freePageUniqueId as the
> db.free. If this is the case we can safely use the free page information
> contained in the db.free
> Now, the only way to read the whole index file again is IF the crash happens
> btw step 1 and 2 (what is very unlikely).
> The drawback of this implementation is that we will have to save db.free
> during the checkpoint, what can possibly increase the checkpoint time.
> Is also important to note that we CAN (and should) have stale data in db.free
> as it is referencing stale db.data:
> Imagine the timeline:
> T0 -> P1, P2 and P3 are free.
> T1 -> Checkpoint
> T2 -> P1 got occupied.
> T3 -> Crash
> In the current scenario after the Pagefile#load the P1 will be free and then
> the replay will mark P1 as occupied or will occupied another page (now that
> the recovery of free pages is done on shutdown)
> This change only make sure that db.data and db.free are in sync and showing
> the reality in T1 (checkpoint), If they are in sync we can trust the db.free.
> This is a really fast draft of what i'm suggesting... If you guys agree, i
> can create the proper patch after:
> [https://github.com/alanprot/activemq/commit/18036ef7214ef0eaa25c8650f40644dd8b4632a5]
>
> This is related to https://issues.apache.org/jira/browse/AMQ-6590
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
