Thanks Doug. -- kuer
On 7月8日, 上午8时44分, Doug Judd <[email protected]> wrote: > Hi Kuer, > > Yes, this is the correct behavior. If there are still un-compacted > key/value pairs in the CellCache, then the commit log fragments that they > have been persisted in should not be removed. > > - Doug > > On Tue, Jul 7, 2009 at 5:28 PM, kuer <[email protected]> wrote: > > > Hi, Doug, > > > Here is the source code of purge() : > > > 227 int CommitLog::purge(int64_t revision) { > > 228 ScopedLock lock(m_mutex); > > 229 CommitLogFileInfo file_info; > > 230 String fname; > > 231 > > 232 try { > > 233 > > 234 while (!m_fragment_queue.empty()) { > > 235 file_info = m_fragment_queue.front(); > > 236 fname = file_info.log_dir + file_info.num; > > 237 if (file_info.revision <= revision) { > > 238 m_fs->remove(fname); > > 239 m_fragment_queue.pop_front(); > > 240 HT_INFOF("clgc Removed log fragment file='%s' revision= > > %lld", fname.c_str(), > > 241 (Lld)file_info.revision); > > 242 if (file_info.purge_log_dir) { > > 243 HT_INFOF("Removing commit log directory %s", > > file_info.log_dir.c_str()); > > 244 m_fs->rmdir(file_info.log_dir); > > 245 } > > 246 } > > 247 else { > > 248 > > 249 HT_WARNF("clgc LOG FRAGMENT PURGE breaking because %lld >= > > %lld (file='%s')", > > 250 (Lld)file_info.revision, (Lld)revision, > > fname.c_str()); > > 251 > > 252 break; > > 253 } > > 254 } > > 255 > > 256 } > > 257 catch (Hypertable::Exception &e) { > > 258 HT_ERRORF("Problem purging log fragment fname = '%s'", > > fname.c_str()); > > 259 return e.code(); > > 260 } > > 261 > > 262 return Error::OK; > > 263 } > > 264 > > > When thread execute to LINE 249 & 250, it will break the while loop, > > and return from purge(). In another word, It will not pop the fragment > > in m_fragment_queue, and it still there. When purging next time, it > > still handle the same fragment, and still break the while loop before > > m_fragment_queue.empty(). > > > In this situation, May I understand that there are always some > > fragment in the m_fragment_queue which never be purged ??? > > > -- kuer > > > On 7月8日, 上午7时54分, Doug Judd <[email protected]> wrote: > > > Hi kuer, > > > > I don't see any problem with the log snippet that you give below. The > > "clgc > > > LOG FRAGMENT PURGE" messages are just leftover debugging statements that > > I > > > forgot to remove in a recent checkin. Are you experiencing a real > > problem > > > with the database? If so, what is the symptom? > > > > - Doug > > > > On Tue, Jul 7, 2009 at 4:48 PM, kuer <[email protected]> wrote: > > > > > I found lots of message in rangeserver's log : > > > > > 2009-07-08 07:30:23,198 1231112512 Hypertable.RangeServer [WARN] (Lib/ > > > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because > > > > 1247008867417054087 >= 1247003041037972148 (file='/hypertable/servers/ > > > > 221.194.134.176_31060/log/user/75') > > > > 2009-07-08 07:30:23,198 1231112512 Hypertable.RangeServer [INFO] > > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 322964958 bytes > > > > 2009-07-08 07:30:43,197 1336011072 Hypertable.RangeServer [INFO] > > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 323476974 bytes > > > > 2009-07-08 07:31:03,198 1199642944 Hypertable.RangeServer [WARN] (Lib/ > > > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because > > > > 1247008867417054087 >= 1247003041037972148 (file='/hypertable/servers/ > > > > 221.194.134.176_31060/log/user/75') > > > > 2009-07-08 07:31:03,198 1199642944 Hypertable.RangeServer [INFO] > > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 323476974 bytes > > > > 2009-07-08 07:31:23,197 1294051648 Hypertable.RangeServer [INFO] > > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 323476974 bytes > > > > 2009-07-08 07:31:43,198 1178663232 Hypertable.RangeServer [WARN] (Lib/ > > > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because > > > > 1247008867417054087 >= 1247003041037972148 (file='/hypertable/servers/ > > > > 221.194.134.176_31060/log/user/75') > > > > 2009-07-08 07:31:43,198 1178663232 Hypertable.RangeServer [INFO] > > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 324501006 bytes > > > > > It seems that it will be never end. > > > > > I want to know : > > > > 1. How this happened? why to purge a future revision? > > > > 2. How to resolve/correct this problem? > > > > From the purge() function, the thread will stop purging if > > > > meeting with the 'fucture' revision, and purge() will end. The > > > > 'future' revision data is still there; when purging next time, it will > > > > complain it again. > > > > What's the rationale behind this? > > > > Would someone tell me kindly? > > > > > Thanks. > > > > > -- kuer --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
