Re: Is there a way to store binary data (byte[]) in DocValues?

2013-08-12 Thread Robert Muir
On Mon, Aug 12, 2013 at 12:25 PM, Mathias Lux wrote: > > Another thing for not using the the SORTED_SET and SORTED > implementations is, that Solr currently works with Strings on that and > I want to have a small memory footprint for millions of images ... > which does not go well with immutables.

Re: Is there a way to store binary data (byte[]) in DocValues?

2013-08-12 Thread Mathias Lux
Hi Robert, I'm basically "mis-using" Solr for content based image search. So I have indexed fields (hashes) for candidate selection, i.e. 1,500 candidate results retrieved with the IndexSearcher by hashes, which I then have to re-rank based on numeric vectors I'm storing in byte[] arrays. I had an

Re: Is there a way to store binary data (byte[]) in DocValues?

2013-08-12 Thread Robert Muir
On Mon, Aug 12, 2013 at 8:38 AM, Mathias Lux wrote: > Hi! > > I'm basically searching for a method to put byte[] data into Lucene > DocValues of type BINARY (see [1]). Currently only primitives and > Strings are supported according to [1]. > > I know that this can be done with a custom update hand

Re: Is there a way to store binary data (byte[]) in DocValues?

2013-08-12 Thread Mathias Lux
Hi! That's what I'm doing currently, but it ends up in StoredField implementations, which create an overhead on decompression I want to avoid. cheers, Mathias On Mon, Aug 12, 2013 at 3:11 PM, Raymond Wiker wrote: > base64-encode the binary data? That will give you strings, at the expense > of s

Re: Is there a way to store binary data (byte[]) in DocValues?

2013-08-12 Thread Raymond Wiker
base64-encode the binary data? That will give you strings, at the expense of some storage overhead. On Mon, Aug 12, 2013 at 2:38 PM, Mathias Lux wrote: > Hi! > > I'm basically searching for a method to put byte[] data into Lucene > DocValues of type BINARY (see [1]). Currently only primitives an

Is there a way to store binary data (byte[]) in DocValues?

2013-08-12 Thread Mathias Lux
Hi! I'm basically searching for a method to put byte[] data into Lucene DocValues of type BINARY (see [1]). Currently only primitives and Strings are supported according to [1]. I know that this can be done with a custom update handler, but I'd like to avoid that. cheers, Mathias [1] http://wik