Re: [basex-talk] Possible to Speed Up This Lookup?
On Fri, 2022-07-15 at 22:00 +, Eliot Kimber wrote:
> I profiled the db:open() and collection() calls and they were
> basically instantaneous.
>
> Moving to using db:attribute() and the index makes my process 100x
> times faster. Now I feel like a bit of an idiot for not having
> figured this out sooner.
Nah, optimization is tricksy. Glad it's working!
>
> Somebody needs to write the Optimizing BaseX book….
>
> Cheers,
>
> E.
>
> _
> Eliot Kimber
> Sr Staff Content Engineer
> O: 512 554 9368
> M: 512 554 9368
> servicenow.com<https://www.servicenow.com>
> LinkedIn<https://www.linkedin.com/company/servicenow> |
> Twitter<https://twitter.com/servicenow> |
> YouTube<https://www.youtube.com/user/servicenowinc> |
> Facebook<https://www.facebook.com/servicenow>
>
> From: BaseX-Talk on
> behalf of Liam R. E. Quin
> Date: Friday, July 15, 2022 at 4:55 PM
> To:
> basex-talk@mailman.uni-konstanz.de >
> Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
> [External Email]
>
>
> On Fri, 2022-07-15 at 21:20 +, Eliot Kimber wrote:
> >
> > return (db:option('attrindex'),
> > for $node in $map
> > return prof:track(db:open($rkDatabase)/doc-where-used-
> > index/where-used-entry/@key[xs:integer(.) eq 9151416])?time !
> > util:formatTime(.)
>
> can you take the db:open() call out and see how much faster the
> expression runs without it?
>
> return (db:option('attrindex'),
> let $db := db:open($rkDatabase)
> return
> for $node in $map
>
> return
> prof:track($db/doc-where-used-index/where-used-entry
> /@key[xs:integer(.) eq 9151416])?time ! util:formatTime(.)
>
> (if i have that right)
>
> --
> Liam Quin,
> https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dDcIyNVA$
> <
> https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dDcIyNVA$
> >
> Available for XML/Document/Information Architecture/XSLT/
> XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
> Barefoot Web-slave, antique illustrations:
> https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dk_syZfc$
> <
> https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dk_syZfc$
> >
--
Liam Quin, https://www.delightfulcomputing.com/
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
Re: [basex-talk] Possible to Speed Up This Lookup?
Hi Eliot -
Others are much more proficient and knowledgeable, but I try the following:
1. ABI := Always Be Indexing :)
2. I think that Christian has expressed that, with indexes in play, many
queries can experience significant optimizations by reversing the query
pattern; eg starting with the attribute value and working back up from
there, etc.
3. The optimized query view can be super helpful in this regard, too.
I'm sure there are other, better habits and strategies, I know (I just
don't have them ingrained! ). Other users will share, I'm sure.
Have a pleasant weekend!
Best,
Bridger
On Fri, Jul 15, 2022, 6:01 PM Eliot Kimber
wrote:
> I profiled the db:open() and collection() calls and they were basically
> instantaneous.
>
>
>
> Moving to using db:attribute() and the index makes my process 100x times
> faster. Now I feel like a bit of an idiot for not having figured this out
> sooner.
>
>
>
> Somebody needs to write the Optimizing BaseX book….
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *BaseX-Talk on behalf
> of Liam R. E. Quin
> *Date: *Friday, July 15, 2022 at 4:55 PM
> *To: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex-talk] Possible to Speed Up This Lookup?
>
> [External Email]
>
>
> On Fri, 2022-07-15 at 21:20 +, Eliot Kimber wrote:
> >
> > return (db:option('attrindex'),
> > for $node in $map
> > return prof:track(db:open($rkDatabase)/doc-where-used-
> > index/where-used-entry/@key[xs:integer(.) eq 9151416])?time !
> > util:formatTime(.)
>
> can you take the db:open() call out and see how much faster the
> expression runs without it?
>
> return (db:option('attrindex'),
> let $db := db:open($rkDatabase)
> return
> for $node in $map
>
> return
> prof:track($db/doc-where-used-index/where-used-entry
> /@key[xs:integer(.) eq 9151416])?time ! util:formatTime(.)
>
> (if i have that right)
>
> --
> Liam Quin,
> https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dDcIyNVA$
> <https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dDcIyNVA$>
> Available for XML/Document/Information Architecture/XSLT/
> XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
> Barefoot Web-slave, antique illustrations:
> https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dk_syZfc$
> <https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dk_syZfc$>
>
Re: [basex-talk] Possible to Speed Up This Lookup?
I profiled the db:open() and collection() calls and they were basically
instantaneous.
Moving to using db:attribute() and the index makes my process 100x times
faster. Now I feel like a bit of an idiot for not having figured this out
sooner.
Somebody needs to write the Optimizing BaseX book….
Cheers,
E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>
From: BaseX-Talk on behalf of Liam
R. E. Quin
Date: Friday, July 15, 2022 at 4:55 PM
To: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
[External Email]
On Fri, 2022-07-15 at 21:20 +, Eliot Kimber wrote:
>
> return (db:option('attrindex'),
> for $node in $map
> return prof:track(db:open($rkDatabase)/doc-where-used-
> index/where-used-entry/@key[xs:integer(.) eq 9151416])?time !
> util:formatTime(.)
can you take the db:open() call out and see how much faster the
expression runs without it?
return (db:option('attrindex'),
let $db := db:open($rkDatabase)
return
for $node in $map
return
prof:track($db/doc-where-used-index/where-used-entry
/@key[xs:integer(.) eq 9151416])?time ! util:formatTime(.)
(if i have that right)
--
Liam Quin,
https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dDcIyNVA$<https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dDcIyNVA$>
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave, antique illustrations:
https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dk_syZfc$<https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!Ge6_HxIH-uWYEol5ImGje0BVsQ_FKkbp1x9czcNl7d0t6BKvoQ1v0sENn6_kHvOQI0dfFmEig6LEfg3dk_syZfc$>
Re: [basex-talk] Possible to Speed Up This Lookup?
On Fri, 2022-07-15 at 21:20 +, Eliot Kimber wrote:
>
> return (db:option('attrindex'),
> for $node in $map
> return prof:track(db:open($rkDatabase)/doc-where-used-
> index/where-used-entry/@key[xs:integer(.) eq 9151416])?time !
> util:formatTime(.)
can you take the db:open() call out and see how much faster the
expression runs without it?
return (db:option('attrindex'),
let $db := db:open($rkDatabase)
return
for $node in $map
return
prof:track($db/doc-where-used-index/where-used-entry
/@key[xs:integer(.) eq 9151416])?time ! util:formatTime(.)
(if i have that right)
--
Liam Quin, https://www.delightfulcomputing.com/
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
Re: [basex-talk] Possible to Speed Up This Lookup?
Yes, that is 100x faster.
Clearly I need to slow down and RTFM a bit more closely.
Thanks,
E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>
From: Tim Thompson
Date: Friday, July 15, 2022 at 4:40 PM
To: Eliot Kimber
Cc: Liam R. E. Quin , basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
[External Email]
Is there a performance difference when using db:attribute() instead[1]?
Tim
[1]
https://docs.basex.org/wiki/Database_Module#db:attribute<https://urldefense.com/v3/__https:/docs.basex.org/wiki/Database_Module*db:attribute__;Iw!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcFb-Z4lHQ$>
--
Tim A. Thompson (he, him)
Librarian for Applied Metadata Research
Yale University Library
On Fri, Jul 15, 2022 at 5:21 PM Eliot Kimber
mailto:eliot.kim...@servicenow.com>> wrote:
Actually, I think I spoke too soon.
This test:
return (db:option('attrindex'),
for $node in $map
return
prof:track(db:open($rkDatabase)/doc-where-used-index/where-used-entry/@key[xs:integer(.)
eq 9151416])?time ! util:formatTime(.)
Returns:
true
00.030 seconds
Where my current database context is the database named in $rkDatabase.
A small improvement (0.036 to 0.030) but not dramatic.
So I think I’m still missing some detail, probably a configuration issue.
Cheers,
E,
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://urldefense.com/v3/__https:/www.linkedin.com/company/servicenow__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcFPeRPeww$>
|
Twitter<https://urldefense.com/v3/__https:/twitter.com/servicenow__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcGPMd9U7w$>
|
YouTube<https://urldefense.com/v3/__https:/www.youtube.com/user/servicenowinc__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcHUl5mSEQ$>
|
Facebook<https://urldefense.com/v3/__https:/www.facebook.com/servicenow__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcEdjcThVg$>
From: Eliot Kimber
mailto:eliot.kim...@servicenow.com>>
Date: Friday, July 15, 2022 at 3:39 PM
To: Liam R. E. Quin mailto:l...@fromoldbooks.org>>,
basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>
mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
Yes, turning on the attribute index definitely improves performa nce
dramatically, which is no surprise. Not sure how I missed the need to expressly
turn on these indexes. Doing that now.
Cheers,
E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://urldefense.com/v3/__https:/www.linkedin.com/company/servicenow__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcFPeRPeww$>
|
Twitter<https://urldefense.com/v3/__https:/twitter.com/servicenow__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcGPMd9U7w$>
|
YouTube<https://urldefense.com/v3/__https:/www.youtube.com/user/servicenowinc__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcHUl5mSEQ$>
|
Facebook<https://urldefense.com/v3/__https:/www.facebook.com/servicenow__;!!N4vogdjhuJM!Ca2zI2049S4xTcwSnRICwcUHLqboyNh8rMFfVglLT9-A39rtWpP0ztq55CticIRYCsegnqwW0D7StcEdjcThVg$>
From: Eliot Kimber
mailto:eliot.kim...@servicenow.com>>
Date: Friday, July 15, 2022 at 3:01 PM
To: Liam R. E. Quin mailto:l...@fromoldbooks.org>>,
basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>
mailto:basex-talk@mailman.uni-konstanz.de>>
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
There are on the order of 50K index entry items.
I think the attribute index is probably the answer—for some reason I assumed
that would be indexed by default but I see that it is not looking at the
database details.
I’ll try enabling the index and see what happens.
Cheers,
E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://urldefense.com/v3/__https:/www.linkedin.com
Re: [basex-talk] Possible to Speed Up This Lookup?
Is there a performance difference when using db:attribute() instead[1]?
Tim
[1] https://docs.basex.org/wiki/Database_Module#db:attribute
--
Tim A. Thompson (he, him)
Librarian for Applied Metadata Research
Yale University Library
On Fri, Jul 15, 2022 at 5:21 PM Eliot Kimber
wrote:
> Actually, I think I spoke too soon.
>
>
>
> This test:
>
> return (db:option('attrindex'),
>
> for $node in $map
>
> return
> prof:track(db:open($rkDatabase)/doc-where-used-index/where-used-entry/@key[xs:integer(.)
> eq 9151416])?time ! util:formatTime(.)
>
>
>
> Returns:
>
>
>
> true
>
> 00.030 seconds
>
>
>
> Where my current database context is the database named in $rkDatabase.
>
>
>
> A small improvement (0.036 to 0.030) but not dramatic.
>
>
>
> So I think I’m still missing some detail, probably a configuration issue.
>
>
>
> Cheers,
>
>
>
> E,
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Eliot Kimber
> *Date: *Friday, July 15, 2022 at 3:39 PM
> *To: *Liam R. E. Quin ,
> basex-talk@mailman.uni-konstanz.de
> *Subject: *Re: [basex-talk] Possible to Speed Up This Lookup?
>
> Yes, turning on the attribute index definitely improves performa nce
> dramatically, which is no surprise. Not sure how I missed the need to
> expressly turn on these indexes. Doing that now.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Eliot Kimber
> *Date: *Friday, July 15, 2022 at 3:01 PM
> *To: *Liam R. E. Quin ,
> basex-talk@mailman.uni-konstanz.de
> *Subject: *Re: [basex-talk] Possible to Speed Up This Lookup?
>
> There are on the order of 50K index entry items.
>
>
>
> I think the attribute index is probably the answer—for some reason I
> assumed that would be indexed by default but I see that it is not looking
> at the database details.
>
>
>
> I’ll try enabling the index and see what happens.
>
>
>
> Cheers,
>
>
>
> E.
>
>
>
> _
>
> *Eliot Kimber*
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> servicenow.com <https://www.servicenow.com>
>
> LinkedIn <https://www.linkedin.com/company/servicenow> | Twitter
> <https://twitter.com/servicenow> | YouTube
> <https://www.youtube.com/user/servicenowinc> | Facebook
> <https://www.facebook.com/servicenow>
>
>
>
> *From: *Liam R. E. Quin
> *Date: *Friday, July 15, 2022 at 1:22 PM
> *To: *Eliot Kimber ,
> basex-talk@mailman.uni-konstanz.de
> *Subject: *Re: [basex-talk] Possible to Speed Up This Lookup?
>
> [External Email]
>
>
> On Fri, 2022-07-15 at 18:04 +, Eliot Kimber wrote:
> > collection($rkDatabase)/doc-where-used-index/where-used-
> > entry[string(@key) eq string(db:node-id($node))]
> >
> > Where the markup is shown in [1] below.
> >
> > Using prof:track() I’m measuring a consistent 0.036 seconds per
> > lookup,
>
> could it be opening the collection that takes time? Or are there
> hundreds of thousands of where-used-entry elements, and, if so, do you
> have an attribute value index?
>
> --
> Liam Quin,
> https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$
> <https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$>
> Available for XML/Document/Information Architecture/XSLT/
> XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
> Barefoot Web-slave, antique illustrations:
> https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$
> <https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$>
>
Re: [basex-talk] Possible to Speed Up This Lookup?
Actually, I think I spoke too soon.
This test:
return (db:option('attrindex'),
for $node in $map
return
prof:track(db:open($rkDatabase)/doc-where-used-index/where-used-entry/@key[xs:integer(.)
eq 9151416])?time ! util:formatTime(.)
Returns:
true
00.030 seconds
Where my current database context is the database named in $rkDatabase.
A small improvement (0.036 to 0.030) but not dramatic.
So I think I’m still missing some detail, probably a configuration issue.
Cheers,
E,
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>
From: Eliot Kimber
Date: Friday, July 15, 2022 at 3:39 PM
To: Liam R. E. Quin , basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
Yes, turning on the attribute index definitely improves performa nce
dramatically, which is no surprise. Not sure how I missed the need to expressly
turn on these indexes. Doing that now.
Cheers,
E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>
From: Eliot Kimber
Date: Friday, July 15, 2022 at 3:01 PM
To: Liam R. E. Quin , basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
There are on the order of 50K index entry items.
I think the attribute index is probably the answer—for some reason I assumed
that would be indexed by default but I see that it is not looking at the
database details.
I’ll try enabling the index and see what happens.
Cheers,
E.
_
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>
From: Liam R. E. Quin
Date: Friday, July 15, 2022 at 1:22 PM
To: Eliot Kimber ,
basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Possible to Speed Up This Lookup?
[External Email]
On Fri, 2022-07-15 at 18:04 +, Eliot Kimber wrote:
> collection($rkDatabase)/doc-where-used-index/where-used-
> entry[string(@key) eq string(db:node-id($node))]
>
> Where the markup is shown in [1] below.
>
> Using prof:track() I’m measuring a consistent 0.036 seconds per
> lookup,
could it be opening the collection that takes time? Or are there
hundreds of thousands of where-used-entry elements, and, if so, do you
have an attribute value index?
--
Liam Quin,
https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$<https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$>
Available for XML/Document/Information Architecture/XSLT/
XSL/XQuery/Web/Text Processing/A11Y training, work & consulting.
Barefoot Web-slave, antique illustrations:
https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$<https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$>
Re: [basex-talk] Possible to Speed Up This Lookup?
Yes, turning on the attribute index definitely improves performa nce dramatically, which is no surprise. Not sure how I missed the need to expressly turn on these indexes. Doing that now. Cheers, E. _ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Eliot Kimber Date: Friday, July 15, 2022 at 3:01 PM To: Liam R. E. Quin , basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Possible to Speed Up This Lookup? There are on the order of 50K index entry items. I think the attribute index is probably the answer—for some reason I assumed that would be indexed by default but I see that it is not looking at the database details. I’ll try enabling the index and see what happens. Cheers, E. _ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Liam R. E. Quin Date: Friday, July 15, 2022 at 1:22 PM To: Eliot Kimber , basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Possible to Speed Up This Lookup? [External Email] On Fri, 2022-07-15 at 18:04 +, Eliot Kimber wrote: > collection($rkDatabase)/doc-where-used-index/where-used- > entry[string(@key) eq string(db:node-id($node))] > > Where the markup is shown in [1] below. > > Using prof:track() I’m measuring a consistent 0.036 seconds per > lookup, could it be opening the collection that takes time? Or are there hundreds of thousands of where-used-entry elements, and, if so, do you have an attribute value index? -- Liam Quin, https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$<https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$> Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$<https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$>
Re: [basex-talk] Possible to Speed Up This Lookup?
There are on the order of 50K index entry items. I think the attribute index is probably the answer—for some reason I assumed that would be indexed by default but I see that it is not looking at the database details. I’ll try enabling the index and see what happens. Cheers, E. _ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Liam R. E. Quin Date: Friday, July 15, 2022 at 1:22 PM To: Eliot Kimber , basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Possible to Speed Up This Lookup? [External Email] On Fri, 2022-07-15 at 18:04 +, Eliot Kimber wrote: > collection($rkDatabase)/doc-where-used-index/where-used- > entry[string(@key) eq string(db:node-id($node))] > > Where the markup is shown in [1] below. > > Using prof:track() I’m measuring a consistent 0.036 seconds per > lookup, could it be opening the collection that takes time? Or are there hundreds of thousands of where-used-entry elements, and, if so, do you have an attribute value index? -- Liam Quin, https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$<https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQXLy9rZQ$> Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$<https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!C1fj636Ib2y0kENz2AwiUJZj6IyBT5mrb5YE_kSoMeqI0fFs95OpyWpldUalcS47Lg7Dt-1d0gNC7xyQWGj0qnM$>
Re: [basex-talk] Possible to Speed Up This Lookup?
On Fri, 2022-07-15 at 18:04 +, Eliot Kimber wrote: > collection($rkDatabase)/doc-where-used-index/where-used- > entry[string(@key) eq string(db:node-id($node))] > > Where the markup is shown in [1] below. > > Using prof:track() I’m measuring a consistent 0.036 seconds per > lookup, could it be opening the collection that takes time? Or are there hundreds of thousands of where-used-entry elements, and, if so, do you have an attribute value index? -- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
