>From what I found is that the main performance detriment comes from the fact >that we need to take a lock for each next/peek call of the StoreScanner. Even >when those are uncontended (which they are in 99.9% of the cases) the memory >read/writes barriers are expensive.
I doubt you'll see much improvement from this. The scanner stack is only reset if the set of HFiles for this store changes, i.e. a compaction or a concurrent flush (when using multithreaded flushing). It seems that would relatively rare. If anything we could a class like StoreScanner that does not need to synchronize any of its calls, but even there, the flush is asynchronous to any user action (unless we're blocked on the number of store files, in which case there bigger problem anyway). Did you see a specific issue? -- Lars ----- Original Message ----- From: ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com> To: "dev@hbase.apache.org" <dev@hbase.apache.org> Cc: Sent: Thursday, January 30, 2014 11:48 AM Subject: StoreScanner created for memstore flush should be bothered about updated readers? Hi All In case of flush we create a memstore flusher which in turn creates a StoreScanner backed by a Single ton MemstoreScanner. But this scanner also registers for any updates in the reader in the HStore. Is this needed? If this happens then any update on the reader may nullify the current heap and the entire Scanner Stack is reset, but this time with the other scanners for all the files that satisfies the last top key. So the flush that happens on the memstore holds the storefile scanners also in the heap that was recreated but originally the intention was to create a scanner on the memstore alone. Am i missing something here? Or what i observed is right? If so, then I feel that this step can be avoided. Regards Ram
