Hi, all.
After restarting hypertable + kfs in my work-boxes, there are some
endless messages logging into Hypertable.Master.log :
2009-07-15 16:45:42,431 1350199616 Hypertable.Master [WARN] (Master/
MasterGc.cc:197) Error removing DFS file: /hypertable/tables/
storage_se/agPORT/4F852B7EB462D2CC311753C0/cs12
2009-07-15 16:45:42,432 1350199616 Hypertable.Master [WARN] (Master/
MasterGc.cc:197) Error removing DFS file: /hypertable/tables/
storage_se/agPORT/4F852B7EB462D2CC311753C0/cs14
2009-07-15 16:45:42,432 1350199616 Hypertable.Master [WARN] (Master/
MasterGc.cc:197) Error removing DFS file: /hypertable/tables/
storage_se/default/590FCA21F8FB70BB2A1F56A2/cs3
2009-07-15 16:45:42,433 1350199616 Hypertable.Master [WARN] (Master/
MasterGc.cc:197) Error removing DFS file: /hypertable/tables/
storage_se/default/590FCA21F8FB70BB2A1F56A2/cs4
...
...
...
2009-07-15 16:45:42,433 1350199616 Hypertable.Master [WARN] (Master/
MasterGc.cc:197) Error removing DFS file: /hypertable/tables/
storage_se/default/590FCA21F8FB70BB2A1F56A2/cs8
2009-07-15 16:45:42,434 1350199616 Hypertable.Master [WARN] (Master/
MasterGc.cc:197) Error removing DFS file: /hypertable/tables/
storage_se/default/13C3704B81B2919AE7F6A936/cs12
Then, I check the dfs.log :
2009-07-15 16:55:44,060 1319373120 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/agPORT/4F852B7EB462D2CC311753C0/cs10' - No such file or
directory
2009-07-15 16:55:44,061 1308883264 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/agPORT/4F852B7EB462D2CC311753C0/cs12' - No such file or
directory
2009-07-15 16:55:44,061 1130555712 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/agPORT/4F852B7EB462D2CC311753C0/cs14' - No such file or
directory
2009-07-15 16:55:44,062 1151535424 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/default/590FCA21F8FB70BB2A1F56A2/cs3' - No such file or
directory
2009-07-15 16:55:44,062 1141045568 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/default/590FCA21F8FB70BB2A1F56A2/cs4' - No such file or
directory
...
...
...
2009-07-15 16:55:44,063 1162025280 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/default/590FCA21F8FB70BB2A1F56A2/cs8' - No such file or
directory
2009-07-15 16:55:44,064 1172515136 kosmosBroker [ERROR] (kosmos/
KosmosBroker.cc:273) unlink failed: file='/hypertable/tables/
storage_se/default/13C3704B81B2919AE7F6A936/cs12' - No such file or
directory
When reading source code src/cc/Hypertable/Master/MasterGc.cc :
182 void
183 reap(CountMap &files_map) {
184 size_t nf = 0, nf_done = 0, nd = 0, nd_done = 0;
185 CountMap dirs_map; // reap empty range directories as well
186
187 foreach (const CountMap::value_type &v, files_map) {
188 if (!v.second) {
189 HT_DEBUGF("MasterGc: removing file %s", v.first);
190
191 if (!m_dryrun) {
192 try {
193 m_fs->remove(v.first);
194 ++nf_done;
195 }
196 catch (Exception &e) {
197 HT_WARNF("%s", e.what());
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ THIS LINE IS Logging-MESSAGE
COME FROM
198 }
199 }
200 ++nf;
201 }
202 char *p = strrchr(v.first, '/');
203
204 if (p) {
205 string dir_name(v.first, p - v.first);
206 insert_file(dirs_map, dir_name.c_str(), v.second);
207 }
208 }
209 foreach (const CountMap::value_type &v, dirs_map) {
210 if (!v.second) {
211 HT_DEBUGF("MasterGc: removing directory %s", v.first);
212
I found that reap(file_map) just remove the specified files, but do
NOT notify/mark the status of deletion to anyone else. Recursively,
the `file_map` was built in scan_metadata() function. Apparently,
scan_metadata() does NOT check the existence of the file to delete
when building deletion-file-list.
When Master.GC-ing, this reap operations would repeat. At present, I
donot read the source code totally, I donot understand the rational of
scan_metadata(). But the reap() do *delete* the exist file; So there
is risk that reap() will delete some WORKING ( IN-USE) file by name
which should be kept alive!!!
Any one can help me out ?
Thanks a lot.
-- kuer
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Hypertable Development" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/hypertable-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
