Hello, I have Hbase 1.2.0 cluster that has a lot of state*log files under /hbase/MasterProcWALs/. As a result, when I need to start hbase master, it can't start, gives me "java.lang.OutOfMemoryError: Direct buffer memory". I then manually remove all the files from /hbase/MasterProcWALs and the master starts fine. There are two bugs related to this: [HBASE-14712] - MasterProcWALs never clean up [HBASE-15100] - Master WALProcs still never clean up
But the version I have has these patches included (I see these listed in my CHANGES.txt). I have 3 clusters with the same version and all 3 have this problem. Any idea how to deal with this besides manually removing the state*.log files? Why would this not be cleaning? I attached the list of all the files there now. oldWALs dir is being cleaned up so I guess the CleanerChore process is doing its job. Not sure if that one does cleaning of MasterProcWALs. MasterProcWALs.txt <http://apache-hbase.679495.n3.nabble.com/file/t496798/MasterProcWALs.txt> -- Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-User-f4020416.html
