Re: How can I do efficient FSIndex lookup?

2019-09-09 Thread Mario Juric
Thanks, Marshall. This is definitely a possibility, although it means a tighter coupling of our upstream components with the downstream parts in the pipeline that produce the different outputs depending on configuration, so we need to think about how to do this properly. Cheers, Mario

Re: How can I do efficient FSIndex lookup?

2019-09-09 Thread Marshall Schor
ok. Re: getting the top 5 or 10 items: Here's a technique you may find of use: Put the items into a Java PriorityQueue.  Keep a piece of data which is the bottom item, and in your insert-into-the-queue code, check if the item to-be-inserted is below that, and if so, skip it. This gives a very ef

Re: How can I do efficient FSIndex lookup?

2019-09-09 Thread Mario Juric
Great! Looking forward to try it out as soon we have the time. We will need a testing period before we put this into production, but the first step is to get it to build and work end to end. Cheers Mario > On 8 Sep 2019, at 19:40 , Richard Eckart de Castilho wrote: > > Hi Mario,

Re: How can I do efficient FSIndex lookup?

2019-09-09 Thread Mario Juric
Hi, Once again thanks for the response. It is really appreciated :) I tried the moveTo(fs) instead of just using an iterator constructed from the FS, and this appeared to give me all items of the specified type when I didn’t set any values on it, which was an accidental experiment, but when I s

Re: How can I do efficient FSIndex lookup?

2019-09-08 Thread Richard Eckart de Castilho
Hi Mario, > On 6. Sep 2019, at 08:50, Mario Juric wrote: > > We are still on 2.x since we are awaiting the next major DKPro release with > UIMA 3 because of dependencies. DKPro Core 2.0.0 is now available :) Cheers, -- Richard

Re: How can I do efficient FSIndex lookup?

2019-09-06 Thread Marshall Schor
Please don't add to the indexes, the FS you're temporarily using as the argument for the moveTo operation.  (and of course, if you don't add it, you won't need to remove it...) If you describe your use case in a bit more detail, I can perhaps comment on this more. -Marshall On 9/6/2019 2:50 AM,

Re: How can I do efficient FSIndex lookup?

2019-09-06 Thread Marshall Schor
Sorry, no. But you can make one "key" per CAS instance, and reuse it (you'll need to keep some kind of a reference to it). -Marshall On 9/5/2019 5:42 PM, Richard Eckart de Castilho wrote: > On 5. Sep 2019, at 23:40, Marshall Schor wrote: >> The normal way to get the "binary search" kind of behav

Re: How can I do efficient FSIndex lookup?

2019-09-05 Thread Mario Juric
Hi, Thanks for responding. I tried with a temporary FS where the key value was set, but I got every annotation from the index, so that didn’t appear to change anything, and it also broke my unit tests immediately. I also stepped through the iterator implementation and found construction of th

Re: How can I do efficient FSIndex lookup?

2019-09-05 Thread Richard Eckart de Castilho
On 5. Sep 2019, at 23:40, Marshall Schor wrote: > > The normal way to get the "binary search" kind of behavior is to get a plain > iterator over the sorted index, and then use the moveTo method, specifying a > target FS as the one to move to. The target FS can be a "temporary" FS, one > that is

Re: How can I do efficient FSIndex lookup?

2019-09-05 Thread Marshall Schor
Perhaps the use of a filtered iterator went in the wrong direction. The normal way to get the "binary search" kind of behavior is to get a plain iterator over the sorted index, and then use the moveTo method, specifying a target FS as the one to move to.  The target FS can be a "temporary" FS, one

How can I do efficient FSIndex lookup?

2019-09-04 Thread Mario Juric
Hi, I created a custom FSIndex for an annotation type in the hope of speeding up lookup based on one of it’s fields, but after some profiling I found to my surprise that this doesn’t appear to be what I get. I specified the index to be sorted according to two fields where the first is a key and