Hi Steven, I removed the raw data files too, but the query took more time to get result. With data files , it is "total CPU time 0.136008 s",while it is "total CPU time 2.52816 s" without data files.
2015-10-14 10:49 GMT+08:00 Enns, Steven <[email protected]>: > Hi John, > > Under high load we observe unusually high kernel CPU, many write syscalls > originating from text::startPositions, category.cpp:1547. We have > removed the raw data files from our text indexes to conserve disk space and > queries seemed to function without them. Can you help me understand what is > the purpose of this write call? Do we need to keep the raw data files? > > const bool isActiveData = > (thePart->getStateNoLocking() == ibis::part::STABLE_STATE && > (dir == thePart->currentDataDir() || > std::strcmp(dir, thePart->currentDataDir()) == 0)); > if (fdata == 0) { // failed to open data file, assume it is empty > if ((isActiveData || thePart->currentDataDir()== 0) > && thePart->nRows() > 0) { > rewind(fsp); > nold = thePart->nRows(); > for (unsigned j = 0; j <= nold; ++ j) (void) > fwrite(&pos, sizeof(int64_t), 1, fsp); } > fclose(fsp); > return; > } > > > > > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > >
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
