> Sharing store files will require a coordination dance between master and > slaves upon compaction and flushes. Sharing active HLogs is more evil given > the [HMaster] may become involved
The log rollover happens relatively infrequently, however yes, when a log rolls over, it could be tricky on the slave side. We're assuming that the slave RS has the same regions as the master RS. With the Coprocessor system, each RS writes it's own HLogs, or perhaps there's no need for the slave to write logs? On Mon, Jun 13, 2011 at 7:21 AM, Andrew Purtell <[email protected]> wrote: >> From: Jason Rutherglen <[email protected]> >> Right thanks. I think replication is fairly simple, I don't know much >> about the HDFS sync code, if one has sync'd on the HLog writer, then >> an HLog reader should be able to read from there? > > See my comments on HBASE-2357 regarding the undesirability of tailing HLogs > directly from HDFS, IMHO. Goes something like: > > "[Read] slaves would need to access foreign store files for regions that are > not open on the RS. So then tailing HLogs, more foreign files, at the slave > is not unreasonable. But that is a major violation of assumptions that store > files are private. Sharing store files will require a coordination dance > between master and slaves upon compaction and flushes. Sharing active HLogs > is more evil given the [HMaster] may become involved" > > i.e. splitting upon failure of the master RS. But the situation is more > complicated, now we have distributed splitting (HBASE-1364). > > and > > "Also, the trouble with watching the WAL either on the slave side at the file > or on the master side with WALObserver is that .writeToWAL(false) edits will > be unnoticed until flush." > > - Andy > >
