Thanks yonik! I do have some unused fields inside the csv file. But they are not empty. They are numeric they can be anything between 0 to 10,000 Can I do something like f.unused.map=*:98765
yatir -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yonik Seeley Sent: Thursday, August 09, 2007 10:41 PM To: solr-user@lucene.apache.org Subject: Re: question: how to divide the indexing into sperate domains Hmmm, I think you can map an empty (zero length) value to something else via f.foo.map=:something But that column does currently need to be there in the CSV. Specifying default values in a per-request basis is interesting, and something we could perhaps support in the future. The quickest way to index your data right now would probably be to change the file, adding another value at the end of each file. I think it could even be an empty value (just add a "," at the end of each line), and then you could map that via f.domain.map=:98765 btw, 300M records is a lot for one Solr instance... I hope you've got a big box with a lot of memory, and aren't too concerned with your query latency. Otherwise you can do some partitioning by domain. -Yonik On 8/9/07, Ben Shlomo, Yatir <[EMAIL PROTECTED]> wrote: > Hi! > > say I have 300 csv files that I need to index. > > Each one holds millions of lines (each line is a few fields separated by > commas) > > Each csv file represents a different domain of data (e,g, file1 is > computers, file2 is flowers, etc....) > > There is no indication of the domain ID in the data inside the csv file > > > > When I search I would like to specify the id of a specific domain > > And I want solr to search only in this domain - to save time and reduce > the number of matches > > I need to specify during indexing - the domain id of the csv file being > indexed > > How do I do it ? > > > > > > Thanks > > > > > > > > p.s. > > I wish I could index like this: > > curl > http://localhost:8080/solr/update/csv?stream.file=test.csv&fieldnames=fi > eld1,field2&f.domain.value=98765 > <http://localhost:8080/solr/update/csv?stream.file=test.csv&fieldnames=f > ield1,field2&f.domain.value=98765> (where 98765 is the domain id for > ths specific csv file) > >
