Hi Dave, I'll leave your questions for the experts of Mark Logic. Just wanted to ask you whether you considered using faceted searching instead of sorting your results. Faceted searching is a lot quicker than sorting as it can rely fully on the indexes, where sorting most likely can't (MarkLogic: please correct me if I am wrong).
When faceted searching, the index information is used to calculate which indexed values are within the result set. This information can be presented to the end user as sensible search filters that could be added. It is not only possible to calculate which unique values exist within the result set, but the indexes can return the count of each value within the result set as well.. It is the approach that is used on for instance http://markmail.org/. Just enter any search term.. Kind regards, Geert > Drs. G.P.H. Josten Consultant http://www.daidalos.nl/ Daidalos BV Source of Innovation Hoekeindsehof 1-4 2665 JZ Bleiswijk Tel.: +31 (0) 10 850 1200 Fax: +31 (0) 10 850 1199 http://www.daidalos.nl/ KvK 27164984 De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend. > From: [email protected] > [mailto:[email protected]] On Behalf Of > Dave Feldmeier > Sent: maandag 27 april 2009 7:23 > To: [email protected] > Subject: [MarkLogic Dev General] Questions about results > ordering with element range indices > > I am sorting results and the performance is less than I would > like. A search without ordering takes 0.2 seconds and a > search with ordering takes 17 seconds. > > First some questions: > > > 1. For improving sort performance, is it sufficient to set > up an element range index, or must I also set the "range > value positions" radio button to "True"? > 2. Does an element range index improve performance if all > the element values are unique? > > > For the following example, I am doing a word query and > sorting based on the value of an element (each document has a > unique value of this element and the type is "string"). I > have set up an element range index for the element and set > the "range value positions" radio button to "True". > > Each document has two fragments and the fragment that I'm > retrieving for each probably is something like 20K. > > In the following trace (see below), the path is searchable > (good), the word query contributes a constraint (good), the > order by clause contributed a constraint (good), but then I > get the line: > > > Selected 4359 fragments to filter (2 ordered, 4357 unordered) > > > If I understand this line, it appears that the element range > index is not helping very much and that I'm fetching 4357 > fragments (even so, does 17 seconds seem reasonable for this > many fragments?). > > The MarkLogic version is 4.0-3 running on Red Hat Enterprise > Linux version 4. The hardware is eight 300G 10K RPM disks in > a RAID5 array with dual quad core 64-bit Intel 5405 > processors, and I'd think that this would be plenty fast. > > -Dave > > David Feldmeier > Twin Dolphin Software, Inc. > 303 Twin Dolphin Drive, Suite 600 > Redwood City, CA 94065 > > > 2009-04-26 00:40:34.000 Info: gazelle2-8007: /lib/patent.xqy > line 335: > this:search_sort(cts:element-query(expanded-QName("", > "ASSC_AENSC"), cts:word-query("test", ("lang=en"), 1), ()), > "PATNUM", "ascending", 1, 30, "US") > 2009-04-26 00:40:34.000 Info: gazelle2-8007: /lib/patent.xqy > line 335: Analyzing path for search: collection("US") > 2009-04-26 00:40:34.000 Info: gazelle2-8007: /lib/patent.xqy > line 335: Step 1 is searchable: collection("US") > 2009-04-26 00:40:34.000 Info: gazelle2-8007: /lib/patent.xqy > line 335: Path is fully searchable. > 2009-04-26 00:40:34.000 Info: gazelle2-8007: /lib/patent.xqy > line 335: Gathering constraints. > 2009-04-26 00:40:34.001 Info: gazelle2-8007: /lib/patent.xqy > line 335: Search query contributed 1 constraint: > cts:element-query(expanded-QName("", "ASSC_AENSC"), > cts:word-query("test", ("lang=en"), 1), ()) > 2009-04-26 00:40:34.001 Info: gazelle2-8007: /lib/patent.xqy > line 335: Order by clause contributed 1 range ordering > constraint for $i: order by $i/child::PATENT/child::PATNUM ascending > 2009-04-26 00:40:34.001 Info: gazelle2-8007: /lib/patent.xqy > line 335: Executing search. > 2009-04-26 00:40:34.091 Info: gazelle2-8007: /lib/patent.xqy > line 335: Selected 4359 fragments to filter (2 ordered, 4357 > unordered). > 2009-04-26 00:40:51.263 Info: gazelle2-8007: <qm:query-meters > xsi:schemaLocation="http://marklogic.com/xdmp/query-meters > query-meters.xsd" > <http://marklogic.com/xdmp/query-metersquery-meters.xsd> > xmlns:qm="http://marklogic.com/xdmp/query-meters"; > <http://marklogic.com/xdmp/query-meters> xmlns:xsi="http:/\ > /www.w3.org/2001/XMLSchema-instance"> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:elapsed-time>PT17.261083S</qm:elapsed-time> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:requests>1</qm:requests> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:list-cache-hits>722</qm:list-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:list-cache-misses>0</qm:list-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:in-memory-list-hits>0</qm:in-memory-list-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:expanded-tree-cache-hits>4688</qm:expanded-tree-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:expanded-tree-cache-misses>8385</qm:expanded-tree-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:compressed-tree-cache-hits>3948</qm:compressed-tree-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:compressed-tree-cache-misses>4437</qm:compressed-tree-cach > e-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:in-memory-compressed-tree-hits>0</qm:in-memory-compressed- > tree-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:value-cache-hits>1</qm:value-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:value-cache-misses>4545</qm:value-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:regexp-cache-hits>1</qm:regexp-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:regexp-cache-misses>4</qm:regexp-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:link-cache-hits>0</qm:link-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:link-cache-misses>8714</qm:link-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fragments-added>0</qm:fragments-added> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fragments-deleted>0</qm:fragments-deleted> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fs-program-cache-hits>1</qm:fs-program-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fs-program-cache-misses>0</qm:fs-program-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:db-program-cache-hits>0</qm:db-program-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:db-program-cache-misses>0</qm:db-program-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fs-main-module-sequence-cache-hits>0</qm:fs-main-module-se > quence-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fs-main-module-sequence-cache-misses>0</qm:fs-main-module- > sequence-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:db-main-module-sequence-cache-hits>0</qm:db-main-module-se > quence-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:db-main-module-sequence-cache-misses>0</qm:db-main-module- > sequence-cache-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fs-library-module-cache-hits>0</qm:fs-library-module-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:fs-library-module-cache-misses>0</qm:fs-library-module-cac > he-misses> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:db-library-module-cache-hits>0</qm:db-library-module-cache-hits> > 2009-04-26 00:40:51.263 Info: gazelle2-8007: > <qm:db-library-module-cache-misses>0</qm:db-library-module-cac > he-misses> > > > _______________________________________________ General mailing list [email protected] http://xqzone.com/mailman/listinfo/general
