> From: Jason Rutherglen <[email protected]> > Right thanks. I think replication is fairly simple, I don't know much > about the HDFS sync code, if one has sync'd on the HLog writer, then > an HLog reader should be able to read from there?
See my comments on HBASE-2357 regarding the undesirability of tailing HLogs directly from HDFS, IMHO. Goes something like: "[Read] slaves would need to access foreign store files for regions that are not open on the RS. So then tailing HLogs, more foreign files, at the slave is not unreasonable. But that is a major violation of assumptions that store files are private. Sharing store files will require a coordination dance between master and slaves upon compaction and flushes. Sharing active HLogs is more evil given the [HMaster] may become involved" i.e. splitting upon failure of the master RS. But the situation is more complicated, now we have distributed splitting (HBASE-1364). and "Also, the trouble with watching the WAL either on the slave side at the file or on the master side with WALObserver is that .writeToWAL(false) edits will be unnoticed until flush." - Andy
