Realized there was a small race condition I hadn't accounted for in the add_fmri section of catalog.py. This updated webrev fixes that problem.
http://cr.opensolaris.org/~bpytlik/ips-search/ Thanks, Brock Brock Pytlik wrote: > Here's the webrev for the new search backend: > http://cr.opensolaris.org/~bpytlik/ips-search/ > > > 983: pkg search returns just one action per package/token-type combo > 1949: pkg search is inordinately slow > 2122: catalog file update needs to be atomic with respect to both other > updates and reads > > This is a complete changes the search backend as well. > It also adds support of Unix/file glob style matching. (ie, *, ?, [char] > and [!char] are supported by using fnmatch [1]) > Descriptions are now searchable, along with any "set" action in the > manifest. > The performance data is below. There is an increase in RAM requirements, > but the search performance is better, especially on local search. > Also, the large, multi-terabyte sparse file has been removed. > > > Future work: > Possibly offloading sorting work on main dictionary to Unix sort both > for possible speed gain and better control over RAM requirements. > Support for case insensitive matching (probably requires revving > communication protocol between server and client) > Support for boolean search (applied at both the package and action > level, ie package-and and action-and) (requires revving protocol) > Support for phrase search (probably requires revving protocol) > Support for structured searching (searching for "office" in a > description only, for example) > Revving the communication protocol to allow for more flexibility in > search criteria and results returned > Returning the entire action to the client, instead of the four pieces of > data currently returned. > > > > Performance Info (if the columns are all over the place,try using a > fixed width font): > > > Wall time > > server index from scratch 12h 40m 9:22.370 > client index (91) from scratch 38.746s > > install [EMAIL PROTECTED] real: 12:49.888 11:22.948 > user: 2:52.374 3:17.613 > sys: 1:12.638 1:04.843 > > upgrade to 91 real: 6:10.144 7:02.448 > user: 2:07.217 2:52.159 > sys: 21.975 24.088 > > local search suite pass one tot 58.579s 13.654s > local search suite pass one max 16.117s 1.630s > local search suite pass one min 1.262s 1.173s > > local search suite pass two tot 13.389s 12.565s > local search suite pass two max 1.397s 1.279s > local search suite pass two min 1.268s 1.163s > > remote search suite pass one tot 3.826s 4.892s > remote search suite pass one max 0.601s 1.423s > remote search suite pass one min 0.263s 0.253s > > remote search suite pass two tot 2.959s 2.771s > remote search suite pass two max 0.564s 0.268s > remote search suite pass two min 0.239s 0.246s > > Index time (according to dtrace) > gate search > install [EMAIL PROTECTED] 0 42s > upgrade to 91 0 121s > > Server RAM footprint > gate search > install [EMAIL PROTECTED] 16M 118M > upgrade to 91 16M 118M > server index from scratch 414M > > Client RAM footprint > gate search > install [EMAIL PROTECTED] 117M 198M (grows > from about 118M prior to indexing) > upgrade to 91 75M 148M (grows > from about 79M prior to indexing) > index from scratch, 91 107M > > Disk footprint Index > gate search > server 246M + 3.9T 121M > client @86 19M 29M > client @91 38M 29M > > Client side pkg rebuild-index (29M) > Cap Observed Memory Usage Passes Time > 50 66M 4 1:02.488s (46.073) (1.988) > 75 73M 3 56.335s (42.832) (1.968) > 100 85M 2 47.187s (33.721) (1.663) > 150 107M 1 34.635s (25.738) (1.226) > 300 107M 1 35.796s (25.750) (1.247) > > > Sever side pkg index from scratch > Cap Observed Memory Usage Passes Time > 100 159M 15 17:42.486s (16:08.920) > (28.461) > 250 246M 5 11:22.274s (10:06.505) > (13.011) > 500 414M 1 9:11.228s ( 8:05.560) ( > 5.792) > 1000 414M 1 9:47.286s ( 8:10.930) ( > 6.745) > > > > > > > [1] http://docs.python.org/lib/module-fnmatch.html > _______________________________________________ > pkg-discuss mailing list > [email protected] > http://mail.opensolaris.org/mailman/listinfo/pkg-discuss > _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
