Right Way to Read vectors from Index

2024-02-09 Thread Uthra
Hi,
Our project uses Lucene 9_7_0 and we have a requirement of frequent 
vector read operation from the Index for a set of documents. We tried two 
approaches 
1. Index vector as Stored field and retrieve whenever needed using StoredFields 
APIs.
2. Using LeafReader’s API to read vector. Here the Random accessing of 
documents is very slow.
Which one is the right approach and can you suggest me a better approach.Also 
why isn’t there a straightforward API like the StoredFields API to read vector.

Regards,
Uthra

Re: Right Way to Read vectors from Index

2024-02-11 Thread Uthra
Hi Michael,
The use case is to handle index updates along with its vector field 
without resending the vector in change data every time. The change data will 
consist of only “updated_field(s):value(s)” wherein I will read the vector 
value from Index to update the document.

Thanks,
Uthra

> On 09-Feb-2024, at 7:13 PM, Michael Wechner  wrote:
> 
> Can you describe your use case in more detail (beyond having to read the 
> vectors)?
> 
> Thanks
> 
> Michael
> 
> Am 09.02.24 um 12:28 schrieb Uthra:
>> Hi,
>>  Our project uses Lucene 9_7_0 and we have a requirement of frequent 
>> vector read operation from the Index for a set of documents. We tried two 
>> approaches
>> 1. Index vector as Stored field and retrieve whenever needed using 
>> StoredFields APIs.
>> 2. Using LeafReader’s API to read vector. Here the Random accessing of 
>> documents is very slow.
>> Which one is the right approach and can you suggest me a better 
>> approach.Also why isn’t there a straightforward API like the StoredFields 
>> API to read vector.
>> 
>> Regards,
>> Uthra
> 
> 
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org



Re: Right Way to Read vectors from Index

2024-02-12 Thread Uthra
@Gautam - Thanks for your response. Our leaf reader API approach is the one you 
mentioned. I wanted to make sure the best way to read vectors for our case.

@Michael - Yes Michael that’s the case here.

Regards,
Uthra

> On 12-Feb-2024, at 1:23 PM, Michael Wechner  wrote:
> 
> thanks for explainig, Uthra!
> 
> IIUC the text / data for which the vector was originally generated was not 
> changed, only some other data (e.g. meta data) which is also part of the 
> Lucene document, right?
> So, if you want to update the other data within the Lucene document, you 
> first retrieve the Lucene document, create a new Lucene document, update the 
> changed data, but keep the unchanged vector, which means you don't need to 
> re-generate the vector, right?
> 
> Thanks
> 
> Michael
> 
> 
> 
> Am 11.02.24 um 13:39 schrieb Uthra:
>> Hi Michael,
>>  The use case is to handle index updates along with its vector field 
>> without resending the vector in change data every time. The change data will 
>> consist of only “updated_field(s):value(s)” wherein I will read the vector 
>> value from Index to update the document.
>> 
>> Thanks,
>> Uthra
>> 
>>> On 09-Feb-2024, at 7:13 PM, Michael Wechner  
>>> wrote:
>>> 
>>> Can you describe your use case in more detail (beyond having to read the 
>>> vectors)?
>>> 
>>> Thanks
>>> 
>>> Michael
>>> 
>>> Am 09.02.24 um 12:28 schrieb Uthra:
>>>> Hi,
>>>>Our project uses Lucene 9_7_0 and we have a requirement of frequent 
>>>> vector read operation from the Index for a set of documents. We tried two 
>>>> approaches
>>>> 1. Index vector as Stored field and retrieve whenever needed using 
>>>> StoredFields APIs.
>>>> 2. Using LeafReader’s API to read vector. Here the Random accessing of 
>>>> documents is very slow.
>>>> Which one is the right approach and can you suggest me a better 
>>>> approach.Also why isn’t there a straightforward API like the StoredFields 
>>>> API to read vector.
>>>> 
>>>> Regards,
>>>> Uthra
>>> 
>>> -
>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>> 
>> 
>> -
>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>> 
> 
> 
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org