The distinct-values function doesn't use indexes, and the predicate in your averageFrequency expression may not do what you want it to do. For future reference, you can search the list archives: http://marklogic.markmail.org/search/?q=distinct-values might have been illuminating, and still might be useful for background.
Anyway, instead of distinct-values and avg use these functions, which use range indexes: http://docs.marklogic.com/5.0doc/docapp.xqy#display.xqy?fname=http://pubs/5.0doc/apidoc/SearchBuiltins.xml&category=SearchBuiltins&function=cts:element-values http://docs.marklogic.com/5.0doc/docapp.xqy#display.xqy?fname=http://pubs/5.0doc/apidoc/SearchBuiltins.xml&category=SearchBuiltins&function=cts:frequency -- Mike On 10 Aug 2012, at 13:48 , Irvine, Joseph [USA] wrote: > Hello, > > The following query, for example, with roughly 6500 records took 55.34 > seconds according to the MarkLogic performance monitor. > > I changed the names and scenario, but used the same exact template. Assume in > this case that I have 6500 records. Each records the name of the fruit, and > how many days it took to get from the field to the supermarket. This query, > then, goes through and gets a unique list of the fruits, since there are > repeats. Bananas show up in one record as taking 5 days to get to Destination > A, while Bananas show up as taking 12 days to get to Destination B. In this > query, I want to get a list of all of the fruits represented as having been > transported in the 6500 records, and then list the average time that it took > for that fruit. We might find for example that Bananas get to the locations > quicker because they are air-freighted while Watermelons take longer since > they are shipped via ground transportation. That is basically what we are > looking for. > > QUERY: > > for $d in distinct-values(/Data/fruitName) > return > <item> > <fruitName>{$d}</fruitName> > <averageFrequency>{avg(/qdaData/daysToMarket[/qdaData/fruitName = > $d])}</averageFrequency> > </item> > > That seems highly excessive to me. Each record is composed of just 30 XML > fields. All 6500 records take up less than 30mb disk space cumulatively. The > server I am on is running 24gb RAM. I have all of the fields above as indexed > database elements within MarkLogic. Pretend the following is a rough version > of one of the XML files: > > <Data> > <fruitName>Banana</fruitName> > <daysToMarket>5</daysToMarket> > <finalDestination>Phoenix</finalDestination> > <origin>Argentina</origin> > <weight>5.5</weight> > ... etc > </Data> > > Thanks, > Joseph Irvine > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Michael Blakeley > Sent: Friday, August 10, 2012 1:13 PM > To: MarkLogic Developer Discussion > Subject: [External] Re: [MarkLogic Dev General] Query Speed > > It sounds likely that the .NET queries are not quite the same as the AB > queries. If you can identify a slow query, post it to the list and you may > get some useful tuning advice. > > -- Mike > > On 10 Aug 2012, at 09:52 , Irvine, Joseph [USA] wrote: > >> Hello, >> >> I have a basic .NET client that connects to a MarkLogic database with XCC. >> >> I built it around a database that had 1000 XML documents ingested at the >> time. The queries were quick, almost instantaneous at the time, and the >> program ran smoothly as a result. I have since increased the size of the >> database to 20,000 XML documents. While the forms I have through the >> Application Builder still respond incredibly quickly, the .NET queries are >> incredible slow, some taking several minutes to run, as opposed to the near >> instantaneous speed I had before. >> >> Is there some way to increase the speed for external queries? >> >> Thank you, >> Joseph Irvine >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
