Hi Clark, Funnily enough, that's just what I'm trying now... it's taking a while to reindex but I'll let you know what I find. What's strange is that the duplicated values don't even have any particularly unusual characters in them.
FWIW, I did try deleting the whole index and reindexing it first, but that still produced the duplicates. -Steve On 06/06/2011 12:34, Clark Richey wrote: > Steve, > Can you try changing the collation to Unicode Codepoint? I think your > troubles lie in the way comparisons are being handled by the collation for > your particular data set. > > > ---- > Clark D. Richey, Jr. > Principal Technologist, Federal > MarkLogic Corporation > 7950 Jones Branch Drive > McLean, VA 22102 > Phone: 240 - CLARK - 07 > (240 - 252 - 7507) > > > [email protected]<x-msg://25/[email protected]> > > > www.marklogic.com<http://www.marklogic.com/> > This e-mail and any accompanying attachments are confidential. The > information is intended solely for the use of the individual to whom it is > addressed. Any review, disclosure, copying, distribution, or use of this > e-mail communication by others is strictly prohibited. If you are not the > intended recipient, please notify us immediately by returning this message > to the sender and delete all copies. Thank you for your cooperation. > > > > > > > > > > > > > On 6/6/11 5:19 AM, "Steve Mallen"<[email protected]> wrote: > >> Hi Ron, >> >> Yes, it behaves exactly the same. I've also tried using the explicit >> option "item-order" and adding a matching default collation to the >> prolog of the query. >> >> What's interesting is that doing a cts:frequency() on the two "Pia, >> H.W." values gives different numbers when doing the frequency lookup as >> part of the same query (e.g. in a FLOWR). However, when searching for >> the matching documents by string value in an XPath I only get one of the >> frequencies. It is as if one of the values in the list is completey >> bogus. I am confused :( >> >> -Steve >> >> On 06/06/2011 09:52, Ron Hitchens wrote: >>> Does it behave the same way with an explicit collation >>> option on the call to cts:element-attribute-values that >>> matches the one in the range index definition? >>> >>> >>> http://developer.marklogic.com/pubs/4.2/apidocs/Lexicons.html#cts:element >>> -attribute-values >>> >>> On Jun 6, 2011, at 9:29 AM, Steve Mallen wrote: >>> >>>> Hi folks, >>>> >>>> I have another mystery for the lexicon experts :) >>>> >>>> I've been fetching values out of a string attribute range index in item >>>> order and processing them a batch at a time. But I have noticed that >>>> occasionally I get duplicate values out of the index. This is causing >>>> conflicting updates (since I use the URI-encoded name as the basis for >>>> document URIs and insert many documents per query). >>>> >>>> I would have expected the cts:element-attribute-values() to return a >>>> list of unique string values - I don't really want to have to do >>>> fn:distinct-values() over them, and wouldn't have thought this >>>> necessary. >>>> >>>> I have checked the exact character strings returned by the function and >>>> they are identical - I checked this by doing a string-to-codepoints() >>>> on >>>> the suspect values. Here is an example of what I'm calling and getting >>>> back from the server: >>>> >>>> cts:element-attribute-values(xs:QName('lexi'), >>>> xs:QName('formatted_name'), "Pia, H.", ("limit=50")) >>>> >>>> Pia, H. W. >>>> Pia, Hans Werner >>>> Piacente, V. >>>> Piacente, Vincenzo >>>> Piacenti, Marcello >>>> Piacentini-Sorbi, Sylvia >>>> Piacentini, A. >>>> Piacentini, Dario >>>> Piacentini, Giuseppe >>>> Piacentini, I. >>>> Piacentini, John C. >>>> Piacentini, Leonardo >>>> Piacentini, Lucia >>>> Piacentini, M. >>>> Piacentini, Mauro >>>> Piacentini, Paola >>>> Piacentino, Filippo >>>> Piacentino, G. >>>> Piacere, A >>>> Piačková, Veronika >>>> Piacquadio, A. M. >>>> Piacquadio, Piera >>>> Piaditis, George >>>> Piaget, A. >>>> Piaget, Gerald W. >>>> Piaget, Jean >>>> Piaggesi, A. >>>> Piaggi, GianCarlo >>>> Piaggi, Stefania >>>> Piaggio, G. >>>> Piaggio, Gilda >>>> Piaggio, Maurizio >>>> Piagnani, C. >>>> Piagnani, Maria Claudia >>>> Piagnerelli, M >>>> Piagnerelli, M. >>>> Piagnerelli, Michael >>>> Piaia, F. >>>> Piaia, Rosamari >>>> Piálek, Jaroslav >>>> Piall, E. M. >>>> Pialot, Olivier >>>> Pialoux, G. >>>> Pialoux, Vincent >>>> Pia, F. >>>> Pia, H. W. >>>> Pia, Hans Werner >>>> Pia, Julius >>>> Pia, M. G. >>>> Pia, S. >>>> >>>> Notice that "Pia, H.W." is returned from the top of the list and also >>>> near the bottom. Now this suggests to me that either: >>>> >>>> a) There is only one value in the index and the function call is >>>> returning it twice in different locations. >>>> >>>> or >>>> >>>> b) There are actually two values in the index, but they are somehow >>>> being normalised and returned as the same string value. >>>> >>>> I am using the "Root Collation" for the range index. Could that be the >>>> issue? This seems like it may be a bug to me, but perhaps I'm doing >>>> something silly. >>>> >>>> Many thanks for your thoughts, >>>> >>>> -Steve >>>> >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://developer.marklogic.com/mailman/listinfo/general >>> --- >>> Ron Hitchens {mailto:[email protected]} Ronsoft Technologies >>> +44 7879 358 212 (voice) http://www.ronsoft.com >>> +1 707 924 3878 (fax) Bit Twiddling At Its Finest >>> "No amount of belief establishes any fact." -Unknown >>> >>> >>> >>> >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> http://developer.marklogic.com/mailman/listinfo/general >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
