On Wed, Jul 11, 2018 at 12:49 PM, Manjeet Singh <manjeet.chand...@gmail.com> wrote: > Thanks Sean for your reply > > I still have some question un answered like > Q1: How Hbase syncronized with Hbase indexer.
The Lily Indexer for HBase (sometimes referred to as the "Lily HBase Indexer") is an independent project, linked to in my previous email. They would be best positioned to answer questions you have about how it works. I suggest you talk with that project. My understanding is that the Lily Indexer leverages the HBase replication system to get observe edits as they come into the system and then applies a corresponding change to the index it maintains. That means it has all the same configurable options and warnings that come with any HBase replication set up: http://hbase.apache.org/1.2/book.html#_cluster_replication > Q2 What optimization I can apply. You need to dig into the logs to find out if there is a problem talking to the indexers or if they are just lagging. If they're lagging, then I believe you can add more indexer nodes to scale up effective throughput. The Lily Indexer for HBase project would be better suited to answer that though. > Q3 As it's clear from my stats, data in OldWals is quite huge so it's not > getting clear my HMaster., how can I improve my HDFS space issue due to > this? The only way to safely decrease the size of the retained wals is to make it so they are no longer needed. That means either getting the Lily Indexer for HBase to catch up or reseting things and using a batch indexing method to fill in the gap.
