Re: Can you change the kahadb journal file size between broker starts
Thanks again for your help. I created https://issues.apache.org/jira/browse/AMQ-6572 . I suspect that the second situation is probably an edge case which would be rarely hit (well, both are probably edge cases). I think the consequences of the second may be "harmless"...? -- View this message in context: http://activemq.2283324.n4.nabble.com/Can-you-change-the-kahadb-journal-file-size-between-broker-starts-tp4721155p4721257.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Can you change the kahadb journal file size between broker starts
Yes, I would expect maxFileLength should only be used on file allocation. My expectation is that the journal file length should be able to be changed in between restarts and have everything work. In the past I have increased the size many times before without issues so I'm not sure if the second use case is an issue (KahaDB might handle it elsewhere) but the first portion of the startup code seems like it might be a problem and need to be fixed. Can you create a new Jira? I should have time to take a peak at this next week to verify if a fix is needed. It should be pretty straightforward to write a quick unit test to check both scenarios (length increasing and decreasing) to show if there is an issue and validate any fix. On Fri, Jan 20, 2017 at 9:58 AM, jahlborn wrote: > So, i was poking around to see the actual implications of changing the max > journal file length, and i must say, the code doesn't seem to back up the > claim that it is a safe thing to do. > > https://fisheye.apache.org/browse/activemq-6/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal/Journal.java?hb=true#to310 > > This is the startup code. it attempts to adjust the total length by shaving > off an unused portion of the final data file. if you have made the > maxFileLength smaller since the last run, and you have actual data in the > last journal file which is after the new maxFileLength, then this > computation will incorrectly return a negative value. my suspicion is that > the length of the last data file should be used here instead of > maxFileLength. > > https://fisheye.apache.org/browse/activemq-6/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal/Journal.java?hb=true#to867 > > likewise, this code attempts to handle adjustments to maxFileLength since > the journal file was created, but i think it again fails. if the > maxFileLength has been increased since the data file was created, this would > seem to be setting an offset which is past the length of the current data > file. again, it seems like the length of the data file should be used > directly. > > in general, in order for this to work correctly, the maxFileLength should > only be used for code which is creating new files. all the other code > should be solely relying on the size of the existing file, right? > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Can-you-change-the-kahadb-journal-file-size-between-broker-starts-tp4721155p4721252.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Can you change the kahadb journal file size between broker starts
So, i was poking around to see the actual implications of changing the max journal file length, and i must say, the code doesn't seem to back up the claim that it is a safe thing to do. https://fisheye.apache.org/browse/activemq-6/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal/Journal.java?hb=true#to310 This is the startup code. it attempts to adjust the total length by shaving off an unused portion of the final data file. if you have made the maxFileLength smaller since the last run, and you have actual data in the last journal file which is after the new maxFileLength, then this computation will incorrectly return a negative value. my suspicion is that the length of the last data file should be used here instead of maxFileLength. https://fisheye.apache.org/browse/activemq-6/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal/Journal.java?hb=true#to867 likewise, this code attempts to handle adjustments to maxFileLength since the journal file was created, but i think it again fails. if the maxFileLength has been increased since the data file was created, this would seem to be setting an offset which is past the length of the current data file. again, it seems like the length of the data file should be used directly. in general, in order for this to work correctly, the maxFileLength should only be used for code which is creating new files. all the other code should be solely relying on the size of the existing file, right? -- View this message in context: http://activemq.2283324.n4.nabble.com/Can-you-change-the-kahadb-journal-file-size-between-broker-starts-tp4721155p4721252.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Can you change the kahadb journal file size between broker starts
That is the same with artemis actually. On Wed, Jan 18, 2017 at 7:53 AM, Christopher Shannon wrote: > Yes, that is safe to do. The journalMaxFileLength value is only used > for allocating new journal files and doesn't have any effect on > previous files. If you change the value the broker will continue to > work fine and be able to read the old files but all new allocated > files will be the new size. > > On Tue, Jan 17, 2017 at 4:28 PM, jahlborn wrote: >> I was wondering if it was safe to change the kahadb "journalMaxFileLength" >> cofiguration value between restarts of a broker. in other words, will the >> broker function correctly with different journal files being different >> sizes? >> >> >> >> -- >> View this message in context: >> http://activemq.2283324.n4.nabble.com/Can-you-change-the-kahadb-journal-file-size-between-broker-starts-tp4721155.html >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. -- Clebert Suconic
Re: Can you change the kahadb journal file size between broker starts
thanks! -- View this message in context: http://activemq.2283324.n4.nabble.com/Can-you-change-the-kahadb-journal-file-size-between-broker-starts-tp4721155p4721162.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Can you change the kahadb journal file size between broker starts
Yes, that is safe to do. The journalMaxFileLength value is only used for allocating new journal files and doesn't have any effect on previous files. If you change the value the broker will continue to work fine and be able to read the old files but all new allocated files will be the new size. On Tue, Jan 17, 2017 at 4:28 PM, jahlborn wrote: > I was wondering if it was safe to change the kahadb "journalMaxFileLength" > cofiguration value between restarts of a broker. in other words, will the > broker function correctly with different journal files being different > sizes? > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Can-you-change-the-kahadb-journal-file-size-between-broker-starts-tp4721155.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com.