John,
Your right (of course) - calls to either of the following fucntions do
indeed free memory mapped data - my mistake.
ibis::fileManager::instance().flushDir(char *dir);
ibis::fileManager::instance().flushFile(char *file);
In src/fileManager.cpp the description for ibis::fileManager::flushDir says
"remove all files from the specified directory (include all sub
directories)"
I do have a question about the above statement as it dosn't seem to match
my experiences. Say I have a directory tree like /db/2015/09/{01,02,03,04}
e.g. four sub directories 01, 02, 03 and 04 which are fastbit partitions
that each have "-part.txt" and fastbit data and index files. Note in this
example everyting else is just a directory.
When I specify an actual "leaf" directory (e.g. an actual fastbit
directory) like "/db/2015/09/03" the flush works as expected and all files
in that directory are removed.
ibis::fileManager::instance().flushDir("/db/2015/09/03"); // works
just fine
But when I specify a parent directory like "/db/2015/09" or "/db/2015/"
which just holds lower level directories (e.g. NO "-part.txt" files) the
flush does not seem to iterate and work on the sub directories despite the
description.
ibis::fileManager::instance().flushDir("/db/2015/09"); //
fails to remove fastbit sub directories like "/db/2015/09/01" ....
FYI, I'm indifferent about the functionality it's just that the description
seems a bit misleading. Perhaps the comment to ibis::fileManager::flushDir
should be revised to say:
"remove all files from the specified directory (does not include all
sub directories)"
- Jon
On Mon, Sep 28, 2015 at 10:16 AM, K. John Wu <[email protected]> wrote:
> Hi, Jon,
>
> From what I can see in the file fileManager.cpp, both
> fileManager::clear and fileManager::flushFile attempts to remove files
> from both list of mapped files as well as those simply read into
> memory. If you know exactly what files you want to remove from the
> file cache, you definitely could invoke flushFile directly. If you
> encounters any problems, please let us know.
>
> If you are update a data directory, you can also use
> ibis::fileManager::flushDir.
>
> John
>
>
> On 9/28/15 9:32 AM, Jon Strabala wrote:
> > John,
> >
> > There are two functions e.g. in fileManager.cpp
> >
> > ibis::fileManager::clear // attempts to free resources across
> > all files
> > ibis::fileManager::flushFile // attempts to free resources for one file
> >
> > I believe that only ibis::fileManager::clear seems to call
> > mapped.clear(); and incore.clear().
> >
> > I don't think ibis::fileManager::flushFile removes memmorry map
> > references - is there a reason for this (or am I mistaken about this
> > function) ?
> >
> > Since I am updating partitions in real-time in a multi threaded
> > environment I need to make sure that there is no activity (e.g. set a
> > write lock) and also flush stale cached data. Right now I use
> > ibis::fileManager::clear which seems like overkill considering I might
> > only be updating one partition out of 500. I would like to use
> > ibis::fileManager::flushFile since it appears that it would have a
> > minimal performance impact, but as I said I don'tr think it will free
> > a memory mapped reference.
> >
> > Best Regards,
> >
> > Jon Strabala
> >
> >
> > _______________________________________________
> > FastBit-users mailing list
> > [email protected]
> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
> >
> _______________________________________________
> FastBit-users mailing list
> [email protected]
> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
