Re: [basex-talk] [XQUERY] Filter and paging with window clause
Ok, I see. Thanks a lot for the great help again! M. On 28/08/2018 16:05, Christian Grün wrote: You can use the fn:sort function for that: for tumbling window $w in sort( db:open("adatabase")[every $f in $filters satisfies $f(.)], (), function($doc) { db:path($doc) } ) ... Sorting is a blocking operator, though: All documents will need to be checked first in order to tell which one will be the smallest or largest hit. Best, Christian On Tue, Aug 28, 2018 at 4:01 PM Marco Lettere wrote: and what if, a pervert I know, would like to have the docs also globally ordered by any sort criteria? Any chance of squeezing the order by into this elegant FLOWR? Thanks again. M. On 28/08/2018 15:41, Christian Grün wrote: Hi Marco, Here is a minimized representation of your last query: for tumbling window $w in db:open("adatabase")[every $f in $filters satisfies $f(.)] start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Cheers, Christian On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere wrote: Oh, yes. Thank you Martin. That is obviously a good point. The input is a database with several thousands of documents stored in it. The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. I need to check whether I can compress all this into a simple predicate as you've shown in your example. where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. Thanks for your suggestion in the meanwhile. Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. let $seq := for $doc in db:open("adatabase") where not(false() = ($filters ! (.($doc))) return $doc let $out := for tumbling window $w in $seq start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Regards, Marco. On 28/08/2018 14:19, Martin Honnen wrote: On 28.08.2018 11:04, Marco Lettere wrote: here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. So what is the filter condition? What is the input sequence? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ()) Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? Does let $page-size := 10 for tumbling window $page in (1 to 100)[. mod 2 = 0] start at $sp when $sp mod $page-size = 1 end at $np when ($np + 1) mod $page-size = 1 return {$page} help? It returns 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
Re: [basex-talk] [XQUERY] Filter and paging with window clause
You can use the fn:sort function for that: for tumbling window $w in sort( db:open("adatabase")[every $f in $filters satisfies $f(.)], (), function($doc) { db:path($doc) } ) ... Sorting is a blocking operator, though: All documents will need to be checked first in order to tell which one will be the smallest or largest hit. Best, Christian On Tue, Aug 28, 2018 at 4:01 PM Marco Lettere wrote: > > and what if, a pervert I know, would like to have the docs also > globally ordered by any sort criteria? > Any chance of squeezing the order by into this elegant FLOWR? > Thanks again. > M. > > On 28/08/2018 15:41, Christian Grün wrote: > > Hi Marco, > > > > Here is a minimized representation of your last query: > > > >for tumbling window $w in > > db:open("adatabase")[every $f in $filters satisfies $f(.)] > >start $first at $s when ($s = 1 + ($page - 1) * $count) > >end $last at $e when $e - $s = $count - 1 > >return $w > > > > Cheers, > > Christian > > > > > > On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere wrote: > >> Oh, yes. Thank you Martin. > >> That is obviously a good point. > >> The input is a database with several thousands of documents stored in it. > >> The filters are a dynamic sequence of xpath expressions to be applied on > >> the documents expressed as function references. > >> I need to check whether I can compress all this into a simple predicate as > >> you've shown in your example. > >> where clause would have been much more easier in conjunction with the map > >> operator (!) but if it works I'm going to coope with the predicate. > >> Thanks for your suggestion in the meanwhile. > >> > >> Here a simplified example of my current approach where you could think of > >> $filters as a sequence of boolean functions to be applied to the documents > >> in the db. > >> > >>let $seq := > >> for $doc in db:open("adatabase") > >> where not(false() = ($filters ! (.($doc))) > >> return $doc > >>let $out := > >> for tumbling window $w in $seq > >> start $first at $s when ($s = 1 + ($page - 1) * $count) > >> end $last at $e when $e - $s = $count - 1 > >> return $w > >> > >> Regards, > >> Marco. > >> > >> On 28/08/2018 14:19, Martin Honnen wrote: > >> > >> On 28.08.2018 11:04, Marco Lettere wrote: > >> > >> here's a question related to XQuery, sorry for being slightly off-topic. > >> > >> I'm struggling to find a way to combine the windowing clause and FLOWR in > >> order to get a paged result of a subset of items which respect a given > >> filter. > >> > >> Of course I'm able to get this by first applying the filter to the whole > >> input and then a second FLOWR for the windowing clause. > >> > >> > >> So what is the filter condition? What is the input sequence? > >> > >> The closer I get is [1] which is not what I'd need because I get > >> 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page > >> thus 2,4,6,8,10,12,14,16,18,20. > >> > >> Thanks for any help on this in the meanwhile I'll stick to my naive > >> solution. > >> > >> Marco. > >> > >> [1] > >> > >> let $page := 1 > >> let $count := 10 > >> return > >> for tumbling window $w in (1 to 100) > >> start $first at $s when ($s = 1 + ($page - 1) * $count) > >> end $last at $e when ($e - $s = $count - 1) > >> return > >> $w ! (if (. mod 2 = 0 ) then . else ()) > >> > >> > >> > >> Can't you simply use the filter as a predicate on the expression you use > >> in the windowing for clause? > >> > >> Does > >> > >> let $page-size := 10 > >> for tumbling window $page in (1 to 100)[. mod 2 = 0] > >>start at $sp when $sp mod $page-size = 1 > >>end at $np when ($np + 1) mod $page-size = 1 > >> return {$page} > >> > >> help? > >> > >> It returns > >> > >> 2 4 6 8 10 12 14 16 18 20 > >> 22 24 26 28 30 32 34 36 38 40 > >> 42 44 46 48 50 52 54 56 58 60 > >> 62 64 66 68 70 72 74 76 78 80 > >> 82 84 86 88 90 92 94 96 98 100 > >> > >> > >> > >> >
Re: [basex-talk] [XQUERY] Filter and paging with window clause
and what if, a pervert I know, would like to have the docs also globally ordered by any sort criteria? Any chance of squeezing the order by into this elegant FLOWR? Thanks again. M. On 28/08/2018 15:41, Christian Grün wrote: Hi Marco, Here is a minimized representation of your last query: for tumbling window $w in db:open("adatabase")[every $f in $filters satisfies $f(.)] start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Cheers, Christian On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere wrote: Oh, yes. Thank you Martin. That is obviously a good point. The input is a database with several thousands of documents stored in it. The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. I need to check whether I can compress all this into a simple predicate as you've shown in your example. where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. Thanks for your suggestion in the meanwhile. Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. let $seq := for $doc in db:open("adatabase") where not(false() = ($filters ! (.($doc))) return $doc let $out := for tumbling window $w in $seq start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Regards, Marco. On 28/08/2018 14:19, Martin Honnen wrote: On 28.08.2018 11:04, Marco Lettere wrote: here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. So what is the filter condition? What is the input sequence? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ()) Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? Does let $page-size := 10 for tumbling window $page in (1 to 100)[. mod 2 = 0] start at $sp when $sp mod $page-size = 1 end at $np when ($np + 1) mod $page-size = 1 return {$page} help? It returns 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
Re: [basex-talk] [XQUERY] Filter and paging with window clause
Wow! Works ... of course :-D And I'd say rather fast! Thanks a lot Christian and Martin! Auf bald! On 28/08/2018 15:41, Christian Grün wrote: Hi Marco, Here is a minimized representation of your last query: for tumbling window $w in db:open("adatabase")[every $f in $filters satisfies $f(.)] start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Cheers, Christian On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere wrote: Oh, yes. Thank you Martin. That is obviously a good point. The input is a database with several thousands of documents stored in it. The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. I need to check whether I can compress all this into a simple predicate as you've shown in your example. where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. Thanks for your suggestion in the meanwhile. Here a simplified example of my current approach where you could think of $filters as a sequence of boolean functions to be applied to the documents in the db. let $seq := for $doc in db:open("adatabase") where not(false() = ($filters ! (.($doc))) return $doc let $out := for tumbling window $w in $seq start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Regards, Marco. On 28/08/2018 14:19, Martin Honnen wrote: On 28.08.2018 11:04, Marco Lettere wrote: here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. So what is the filter condition? What is the input sequence? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ()) Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? Does let $page-size := 10 for tumbling window $page in (1 to 100)[. mod 2 = 0] start at $sp when $sp mod $page-size = 1 end at $np when ($np + 1) mod $page-size = 1 return {$page} help? It returns 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
Re: [basex-talk] [XQUERY] Filter and paging with window clause
Hi Marco, Here is a minimized representation of your last query: for tumbling window $w in db:open("adatabase")[every $f in $filters satisfies $f(.)] start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Cheers, Christian On Tue, Aug 28, 2018 at 3:27 PM Marco Lettere wrote: > > Oh, yes. Thank you Martin. > That is obviously a good point. > The input is a database with several thousands of documents stored in it. > The filters are a dynamic sequence of xpath expressions to be applied on the > documents expressed as function references. > I need to check whether I can compress all this into a simple predicate as > you've shown in your example. > where clause would have been much more easier in conjunction with the map > operator (!) but if it works I'm going to coope with the predicate. > Thanks for your suggestion in the meanwhile. > > Here a simplified example of my current approach where you could think of > $filters as a sequence of boolean functions to be applied to the documents in > the db. > > let $seq := > for $doc in db:open("adatabase") > where not(false() = ($filters ! (.($doc))) > return $doc > let $out := > for tumbling window $w in $seq > start $first at $s when ($s = 1 + ($page - 1) * $count) > end $last at $e when $e - $s = $count - 1 > return $w > > Regards, > Marco. > > On 28/08/2018 14:19, Martin Honnen wrote: > > On 28.08.2018 11:04, Marco Lettere wrote: > > here's a question related to XQuery, sorry for being slightly off-topic. > > I'm struggling to find a way to combine the windowing clause and FLOWR in > order to get a paged result of a subset of items which respect a given filter. > > Of course I'm able to get this by first applying the filter to the whole > input and then a second FLOWR for the windowing clause. > > > So what is the filter condition? What is the input sequence? > > The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 > as result for page 1 but I'd really want 10 results per page thus > 2,4,6,8,10,12,14,16,18,20. > > Thanks for any help on this in the meanwhile I'll stick to my naive solution. > > Marco. > > [1] > > let $page := 1 > let $count := 10 > return >for tumbling window $w in (1 to 100) >start $first at $s when ($s = 1 + ($page - 1) * $count) >end $last at $e when ($e - $s = $count - 1) >return > $w ! (if (. mod 2 = 0 ) then . else ()) > > > > Can't you simply use the filter as a predicate on the expression you use in > the windowing for clause? > > Does > > let $page-size := 10 > for tumbling window $page in (1 to 100)[. mod 2 = 0] > start at $sp when $sp mod $page-size = 1 > end at $np when ($np + 1) mod $page-size = 1 > return {$page} > > help? > > It returns > > 2 4 6 8 10 12 14 16 18 20 > 22 24 26 28 30 32 34 36 38 40 > 42 44 46 48 50 52 54 56 58 60 > 62 64 66 68 70 72 74 76 78 80 > 82 84 86 88 90 92 94 96 98 100 > > > >
Re: [basex-talk] [XQUERY] Filter and paging with window clause
Oh, yes. Thank you Martin. That is obviously a good point. The input is a database with several thousands of documents stored in it. The filters are a dynamic sequence of xpath expressions to be applied on the documents expressed as function references. I need to check whether I can compress all this into a simple predicate as you've shown in your example. where clause would have been much more easier in conjunction with the map operator (!) but if it works I'm going to coope with the predicate. Thanks for your suggestion in the meanwhile. Here a simplified example of my current approach where you could think of /$filters/ as a sequence of boolean functions to be applied to the documents in the db. let $seq := for $doc in db:open("adatabase") where not(false() = ($filters ! (.($doc))) return $doc let $out := for tumbling window $w in $seq start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when $e - $s = $count - 1 return $w Regards, Marco. On 28/08/2018 14:19, Martin Honnen wrote: On 28.08.2018 11:04, Marco Lettere wrote: here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. So what is the filter condition? What is the input sequence? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ()) Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? Does let $page-size := 10 for tumbling window $page in (1 to 100)[. mod 2 = 0] start at $sp when $sp mod $page-size = 1 end at $np when ($np + 1) mod $page-size = 1 return {$page} help? It returns 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
Re: [basex-talk] [XQUERY] Filter and paging with window clause
On 28.08.2018 11:04, Marco Lettere wrote: here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. So what is the filter condition? What is the input sequence? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ()) Can't you simply use the filter as a predicate on the expression you use in the windowing for clause? Does let $page-size := 10 for tumbling window $page in (1 to 100)[. mod 2 = 0] start at $sp when $sp mod $page-size = 1 end at $np when ($np + 1) mod $page-size = 1 return {$page} help? It returns 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
[basex-talk] [XQUERY] Filter and paging with window clause
Hi all, here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. But I was wondering whether it's possible to streamline this into one single FLOWR. Two bonus questions: 1. Will this actually be more efficient than the naive (two FLOWR) solution or is the almighty optimizer able to blend the two subsequent FLOWR on its own? 2. What if the input is unordered and I also would like to introduce sorting? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ())