Re: [MarkLogic Dev General] Unfiltered, exact searches

2017-03-24 Thread Andreas Hubmer 
Hi,

Thanks for the explanation. I now understand why exact value queries do not
work without filtering in all cases.
Nevertheless, I am quite surprised and would not have expected it from all
my MarkLogic knowledge and previous discussions.

On the one hand MarkLogic is advertised as integration database. On the
other hand, from several MarkLogic consultants and from several
documentation sources, I've learned that searches should be performed
unfiltered for performance reasons. Thus I would expect that simple
equality queries (a standard operation in other DBMS) can be resolved
correctly without filtering. But this is not possible, as I've learned now.

Your proposed workaround is to setup range indexes. This is impossible when
the queried elements are not known beforehand.

I would be glad to hear of any other workaround or of a solution to this
dichotomy in an upcoming MarkLogic version.

Regards,
Andreas



2017-03-23 17:56 GMT+01:00 Mary Holstege :

>
> Not so much a bug, as a consequence of how indexing works. Value queries,
> even exact values queries are word searches with a spanning constraint. If
> you really want an equality search, set up a range index and do a range
> query.
>
> What is going on is that the index key for "new" (value exact) is same as
> the key for "NEW" (case-insensitive word) and as an unfiltered index
> resolution, the two cases cannot be distinguished without filtering. The
> flags are not baked into the lookup keys (if they were, index sixes would
> be much much larger). Phrases (in whitespace-separated languages) don't
> have this same issue because a word key cannot include the space between
> words, so an exact value query for "new status" will not match "New status"
> //Mary
>
>
> On 03/23/2017 12:40 AM, Andreas Hubmer wrote:
>
> Hi,
>
> There seems to be a bug related to unfiltered and exact value searches.
>
> We are using value queries in the Java API, but I've boiled it down to cts
> searches.
> The following snippet exhibits the wrong behavior:
>
> xquery version "1.0-ml";
> xdmp:document-insert("/bug/doc.xml", NEW)
> ;
>
> "Document is found: OK",
> cts:search(/,
>   cts:and-query((cts:directory-query("/bug/", "infinity"),
> cts:element-value-query(xs:QName("status"), "NEW", ("exact",
>   "unfiltered"
> )
>
> ,"---",
> "Document is not found: OK",
> cts:search(/,
>   cts:and-query((cts:directory-query("/bug/", "infinity"),
> cts:element-value-query(xs:QName("status"), "NEw", ("exact",
>   "unfiltered"
> )
>
> ,"---",
> "Document is found: WRONG",
> cts:search(/,
>   cts:and-query((cts:directory-query("/bug/", "infinity"),
> cts:element-value-query(xs:QName("status"), "new", ("exact",
>   "unfiltered"
> )
>
> We are using a database with fast-case-sensitive-searches and
> fast-diacritic-sensitive-searches turned on, while all other indexes are
> turned off.
> As far as I know only the two indexes are needed for unfiltered exact
> value searches.
>
> Regards,
> Andreas
>
>
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Unfiltered, exact searches

2017-03-23 Thread Mary Holstege 

Not so much a bug, as a consequence of how indexing works. Value queries, even 
exact values queries are word searches with a spanning constraint. If you 
really want an equality search, set up a range index and do a range query.

What is going on is that the index key for "new" (value exact) is same as the 
key for "NEW" (case-insensitive word) and as an unfiltered index resolution, 
the two cases cannot be distinguished without filtering. The flags are not 
baked into the lookup keys (if they were, index sixes would be much much 
larger). Phrases (in whitespace-separated languages) don't have this same issue 
because a word key cannot include the space between words, so an exact value 
query for "new status" will not match "New status"

//Mary

On 03/23/2017 12:40 AM, Andreas Hubmer wrote:
Hi,

There seems to be a bug related to unfiltered and exact value searches.

We are using value queries in the Java API, but I've boiled it down to cts 
searches.
The following snippet exhibits the wrong behavior:

xquery version "1.0-ml";
xdmp:document-insert("/bug/doc.xml", NEW)
;

"Document is found: OK",
cts:search(/,
  cts:and-query((cts:directory-query("/bug/", "infinity"), 
cts:element-value-query(xs:QName("status"), "NEW", ("exact",
  "unfiltered"
)

,"---",
"Document is not found: OK",
cts:search(/,
  cts:and-query((cts:directory-query("/bug/", "infinity"), 
cts:element-value-query(xs:QName("status"), "NEw", ("exact",
  "unfiltered"
)

,"---",
"Document is found: WRONG",
cts:search(/,
  cts:and-query((cts:directory-query("/bug/", "infinity"), 
cts:element-value-query(xs:QName("status"), "new", ("exact",
  "unfiltered"
)

We are using a database with fast-case-sensitive-searches and 
fast-diacritic-sensitive-searches turned on, while all other indexes are turned 
off.
As far as I know only the two indexes are needed for unfiltered exact value 
searches.

Regards,
Andreas



___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general


___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Unfiltered, exact searches

2017-03-23 Thread Geert Josten 
Hi Andreas,

Sounds like a bug indeed. It is as if it appends a case-insensitive flag 
despite the ‘exact’, because of the all-lowercase ’new’. Can you tell which 
version of MarkLogic you are running, and on which architecture?

Cheers,
Geert

From: 
>
 on behalf of Andreas Hubmer 
>
Reply-To: MarkLogic Developer Discussion 
>
Date: Thursday, March 23, 2017 at 8:40 AM
To: MarkLogic Developer Discussion 
>
Subject: [MarkLogic Dev General] Unfiltered, exact searches

Hi,

There seems to be a bug related to unfiltered and exact value searches.

We are using value queries in the Java API, but I've boiled it down to cts 
searches.
The following snippet exhibits the wrong behavior:

xquery version "1.0-ml";
xdmp:document-insert("/bug/doc.xml", NEW)
;

"Document is found: OK",
cts:search(/,
  cts:and-query((cts:directory-query("/bug/", "infinity"), 
cts:element-value-query(xs:QName("status"), "NEW", ("exact",
  "unfiltered"
)

,"---",
"Document is not found: OK",
cts:search(/,
  cts:and-query((cts:directory-query("/bug/", "infinity"), 
cts:element-value-query(xs:QName("status"), "NEw", ("exact",
  "unfiltered"
)

,"---",
"Document is found: WRONG",
cts:search(/,
  cts:and-query((cts:directory-query("/bug/", "infinity"), 
cts:element-value-query(xs:QName("status"), "new", ("exact",
  "unfiltered"
)

We are using a database with fast-case-sensitive-searches and 
fast-diacritic-sensitive-searches turned on, while all other indexes are turned 
off.
As far as I know only the two indexes are needed for unfiltered exact value 
searches.

Regards,
Andreas
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general