[
https://issues.apache.org/jira/browse/HBASE-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ryan rawson resolved HBASE-2644.
--------------------------------
Resolution: Fixed
we removed sync markers after we found at SU they could cause apparent
corruption of HLogs (ouch!). They don't exist anymore. Fixed.
> Investigate sync 'voodoo' splitting WALs
> ----------------------------------------
>
> Key: HBASE-2644
> URL: https://issues.apache.org/jira/browse/HBASE-2644
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.90.0
>
>
> The sequence file sync seems to actually help for some weird reason when
> recovering parts of edits (odd -- it looks like it just adds a marker to the
> file). Investigate. This comes of review of hbase-2437. Below is copied
> from http://review.hbase.org/r/74/
> {code}
> no point to call .sync() here, it just wastes a bunch of IO to write "sync
> markers" which we don't make any real use of.
> Cosmin Lehene 6 days, 23 hours ago (May 25th, 2010, 9:07 a.m.)
> sync() used to call syncFs(). It looks like HBASE-2544 changed things a bit,
> but it doesn't only add the SequenceFile sync marker.
> I added this after I've seen inconsistent results when running splitLog on
> bigger hlogs. Try copying a log from the cluster locally and run splitLog
> from the command line a few times without flushing it after each append. I
> used to get inconsistent results between runs and calling sync fixed it.
> There's this "//TODO: test the split of a large (lots of regions > 500 file).
> In my tests it seems without hflush" in the TestHLogSplit.
> We could do some testing to figure out why would log entries be lost when
> running locally.
> What would be a better way to flush the writer?
> Todd Lipcon 5 days, 19 hours ago (May 26th, 2010, 1:31 p.m.)
> This seems really voodoo.. if anything we're probably masking a real bug by
> doing this. Can you write a unit test which shows this problem (even if it
> takes 30 minutes to run, would be good to have in our arsenal)
> Cosmin Lehene 2 days, 18 hours ago (May 29th, 2010, 2:13 p.m.)
> I can't reproduce it on hdfs-0.20. I can't compile hdfs-0.21 (again) for some
> reason. I'll give it another try some other time.
> Added the test. Also tried with a real 60MB log file.
> I'm not sure if we should leave the test active.
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
