Hi David,
Am I understanding correctly if I say you need a way to quickly determine the
capitalized terms that are worth highlighting? Sounds like you are looking for
one of the functions that (to put it short) return the keys from indexes:
cts:words, cts:values, cts:element-words, etc..
You *could* replace this:
let $q := cts:or-query(("ADENOHYPOPHYSEAL
HORMONES","GROWTH","METABOLISM","REPRODUCTION"))
By:
let $q := cts:or-query(cts:words())
But that way the code of Kelly might result in a *lot* of highlights..
Kind regards,
Geert
>
Drs. G.P.H. Josten
Consultant
http://www.daidalos.nl/
Daidalos BV
Source of Innovation
Hoekeindsehof 1-4
2665 JZ Bleiswijk
Tel.: +31 (0) 10 850 1200
Fax: +31 (0) 10 850 1199
http://www.daidalos.nl/
KvK 27164984
De informatie - verzonden in of met dit emailbericht - is afkomstig van
Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit
bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit
bericht kunnen geen rechten worden ontleend.
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Lee, David
> Sent: donderdag 19 november 2009 22:17
> To: General Mark Logic Developer Discussion
> Subject: RE: [MarkLogic Dev General] RE: FreeText searching - Take Two
>
> Thank you kelly. Yes I was missing $cts:queries But if I am
> reading this right, it doesnt fundamentally change the
> problem or solution.
>
> The problem being that I have to iteratively query for each
> potential matched term in order to generate a link for it.
> In this case the iteration is done inside cts:highlight but
> its exactly the same as if I were to tokenize the text (which
> I'd have to do to generate the list of search terms) and do a
> cts:search on each term, then call hilight.
>
> This is OK in that the case I need to do this, there may be
> 5-10 searchable terms so it should run pretty quickly.
> I was just hoping for a way that avoided iteratively
> searching, which seems to be a ML performance hit that great
> steps are taken to avoid.
>
>
>
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Kelly Stirman
> Sent: Thursday, November 19, 2009 3:58 PM
> To: [email protected]
> Subject: [MarkLogic Dev General] RE: FreeText searching - Take Two
>
> David,
>
> This is what I thought you were trying to do:
>
> declare function local:new-text($query){
> let $query-text := cts:word-query-text($query)
> let $link := fn:concat("default.xqy?search=",$query-text)
> let $count := xdmp:estimate(cts:search(collection(),$query))
> return
> <a href="{$link}">{$query-text} [{$count}]</a> }; let
> $text := <doc>The anterior glandular lobe of the pituitary
> gland, also known as the adenohypophysis. It secretes the
> ADENOHYPOPHYSEAL HORMONES that regulate vital functions such
> as GROWTH; METABOLISM; and REPRODUCTION.</doc> let $q :=
> cts:or-query(("ADENOHYPOPHYSEAL
> HORMONES","GROWTH","METABOLISM","REPRODUCTION"))
> return
> cts:highlight($text,$q,local:new-text($cts:queries))
>
> -->
> <doc>The anterior glandular lobe of the pituitary gland, also
> known as the adenohypophysis. It secretes the <a
> href="default.xqy?search=ADENOHYPOPHYSEAL
> HORMONES">ADENOHYPOPHYSEAL HORMONES [0]</a> that regulate
> vital functions such as <a
> href="default.xqy?search=GROWTH">GROWTH [0]</a>; <a
> href="default.xqy?search=METABOLISM">METABOLISM [0]</a>; and
> <a href="default.xqy?search=REPRODUCTION">REPRODUCTION [0]</a>.</doc>
>
> (I don't have any documents in my database that match those terms)
>
> I think the key piece you were missing is the $cts:queries
> variable, which tells you which query matched the text. From
> the query that is returned you can use other functions to get
> the text from the query, etc.
>
> In the example above counts are generated for each word,
> which requires an additional search for each word. If you
> wanted to link to the first matching document for each word,
> I would simply pass the appropriate text to a new request and
> run the query when the user clicks on the word rather than
> running all the searches and providing direct links to the
> document. That way, you could build the page in one search
> instead of potentially many of searches.
>
> Kelly
>
> Message: 1
> Date: Wed, 18 Nov 2009 14:38:38 -0800
> From: "Lee, David" <[email protected]>
> Subject: RE: [MarkLogic Dev General] FreeText searching - Take Two
> To: "General Mark Logic Developer Discussion"
> <[email protected]>
> Message-ID: <dd37f70d78609d4e9587d473fc61e0a713f37...@postoffice>
> Content-Type: text/plain; charset="us-ascii"
>
> Thanks but I think I'm missing something really subtle (or
> really obvious !) I dont want to hilight the results of the
> search ... I want to hilight the results of a *previous*
> search given a new search.
>
> I think this is hard to explain.
>
>
>
> Let me try again.
>
> Suppose a *previous search* got me a document with an element
> with this string which I am displaying
>
>
>
>
>
> "The anterior glandular lobe of the pituitary gland, also
> known as the adenohypophysis. It secretes the
> ADENOHYPOPHYSEAL HORMONES that regulate vital functions such
> as GROWTH; METABOLISM; and REPRODUCTION."
>
>
>
> Now I want to find within this string matches such as
> "GROWTH" across the entire database, then link "GROWTH" to
> the result of that search,
>
> but "METABOLISM" to the results of that matches to "METABOLISM"
>
>
>
> So suppose I do a cts:search on "GROWTH" , "ADENOHYPOPHYSEAL
> HORMONES"
> , "METABOLISM"
>
> and get a bunch of results to many different documents
>
>
>
> The result I want would ultimately be
>
>
>
> The anterior glandular lobe of the pituitary gland, also
> known as the adenohypophysis. It secretes the <A href="
> get.xquery?doc3>ADENOHYPOPHYSEAL HORMONES</A> that regulate
> vital functions such as <A href="get.xquery?doc1">GROWTH</A>;<A
> href="get.xquery?doc1"> METABOLISM</A>; and REPRODUCTION."
>
>
>
> I dont want to highlight the results of the search, I want to
> highlight the original text , and replace say the
> document-uri of the corresponding hit for that keyword.
>
>
>
>
>
> So if I did as suggested
>
>
>
>
>
> let $paragraph = "... the above paragraph"
>
> $query = (list of upper case words in $paragraph)
>
>
>
> for $res in cts:search(collection(), $query)
>
> return
>
> cts:highlight($paragraph, $query, <A href="?????">{$cts:text}</A>)
>
>
>
> Whats the missing ????
>
> Where do I find which result corresponded to which word in
> the original search ?
>
> I cant find any API that given a search result points into
> which part of the query it matched.
>
> The only way I can figure to do this is to do a new
> cts:search() for each of the terms one by one that I want to match.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Danny Sokolsky
> Sent: Wednesday, November 18, 2009 4:46 PM
> To: General Mark Logic Developer Discussion
> Subject: RE: [MarkLogic Dev General] FreeText searching - Take Two
>
>
>
> And when you do the cts:highlight, you typically do it on a
> subset of the results (the first page, for example) so that
> you do not need to get every result from your search, which
> as you said, would be expensive.
> So the psuedo-code is something like:
>
>
>
> let $query := "hello"
>
> for $res in cts:search(collection(), $query)[1 to 10]
>
> return
>
> cts:highlight($res, $query, <b>{$cts:text}</b>)
>
>
>
> The Search API (search:search, for example) does a lot of
> this for you, so you might want to look into that for a more
> complete solution.
>
>
>
> -Danny
>
>
>
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Mike Sokolov
> Sent: Wednesday, November 18, 2009 1:30 PM
> To: General Mark Logic Developer Discussion
> Subject: Re: [MarkLogic Dev General] FreeText searching - Take Two
>
>
>
> If I understood you right, what you want is:
>
> let $query := cts:word-query ("ADENOHYPOPHYSEAL HORMONES", "GROWTH",
> ...)
> for $result in cts:search ( , $query)
> return cts:highlight ($result, $query, <match>$cts:text</match>)
>
> and then arrange for <match> nodes to be highlighted in your output?
>
> You'd guest a list of (whatever you searched for -
> documents?) with the highlighting applied...
>
> I'm confused because you seem to have already written that
> almost, so maybe I'm missing something?
>
> maybe what you're looking for is more consolidated output like:
>
> return <result uri="{base-uri($result)}">{cts:highlight ( ...
> ) // hit}</result> ?
>
> -Mike
>
> Lee, David wrote:
>
> This is a refinement on the question I asked the other day. I'm
> getting better at formulating my questions so maybe the
> advise might be closer :)
>
>
>
> Suppose I do a query and get a XML document which has a field
> that has text that looks like this :
>
>
>
> "The anterior glandular lobe of the pituitary gland, also
> known as the adenohypophysis. It secretes the
> ADENOHYPOPHYSEAL HORMONES that regulate vital functions such
> as GROWTH; METABOLISM; and REPRODUCTION."
>
>
>
> Those things in all upper case are likely terms that exist in
> other documents.
>
> What I'd like to do is to do a search for each of those terms
> across the entire DB, and if found, create links to either
> the highest scored find, or to a results page (either will do).
>
>
>
> I've poked around and found many little things that are part
> of a possible solution, but nothing that does exactly what I want.
>
>
>
> For example: cts:hilight() could be used to add the links to
> the words.
>
> and I could find a consolidated result set by using
>
> cts:search( ... , cts:word-query ( fn:tokenize(phrase) ) )
>
>
>
> to find all matches etc. I an easily get all the upper
> case words and
> create a search.
>
>
>
> But my problem is this. Suppose I create a search on all the
> upper case words ("ADENOHYPOPHYSEAL" , "GROWTH" , "METABOLISM")
>
> and get a result back with cts:search()
>
>
>
> How can I match up Which nodes of the result matched which
> word so I can hilight them ?
>
> e.g if I did
>
> for $result in cts:search( ..... all the words )
>
> // Which word did $result match ???
>
>
>
>
>
> My only thing I can think of is that i would have to
> iteratively loop through the terms and do a search one by one.
>
>
>
> for $word in ( big oh list of search words )
>
> for $result in cts:search( ... , $word )
>
> cts:hilight( $phrase , $word , { link the word } )
>
>
>
> My guess is that this will perform horribly. I'd rather
> get a single
> consolidated search then do some magic like
>
>
>
> for $result in cts:search( ... , all the words )
>
> cts:hilight( $phrase , the word that matched $result )
>
>
>
>
>
> Does this make any sense ? Is there an API or design pattern
> to do this ? Or should I do the outer loop instead ?
>
>
>
> I looked at cts:walk but it looked like it to use it for this
> would still involve looping on the cts:search() for each term matched.
>
>
>
>
>
> Thanks for any advise.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ----------------------------------------
>
> David A. Lee
>
> Senior Principal Software Engineer
>
> Epocrates, Inc.
>
> [email protected]
>
> 812-482-5224
>
>
>
>
>
>
>
>
>
>
> ________________________________
>
>
>
>
>
>
> _______________________________________________
> General mailing list
> [email protected]
> http://xqzone.com/mailman/listinfo/general
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://xqzone.marklogic.com/pipermail/general/attachments/2009
> 1118/2717e
> 14d/attachment.html
>
> ------------------------------
>
> _______________________________________________
> General mailing list
> [email protected]
> http://xqzone.com/mailman/listinfo/general
>
>
> End of General Digest, Vol 65, Issue 47
> ***************************************
> _______________________________________________
> General mailing list
> [email protected]
> http://xqzone.com/mailman/listinfo/general
> _______________________________________________
> General mailing list
> [email protected]
> http://xqzone.com/mailman/listinfo/general
>
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
