-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Thats what i though too... we see what the speed difference actually
is... running some tests now...
Thanks for the info!
- --Tiernan
On 19/10/2011 16:07, Dyer, James wrote:
>
> Not that I am doing this with any of my indexes, but I'm pretty sure
the "get doc from solr, modify, commit back to solr" approach really is
that simple. Just be sure you are storing the exact raw data that came
from your database (typically you would). The problem with this approach
is it potentially could be very slow if you're updating lots of documents.
>
>
>
> James Dyer
>
> E-Commerce Systems
>
> Ingram Content Group
>
> (615) 213-4311
>
>
>
> From: Tiernan OToole [mailto:lsmart...@gmail.com]
> Sent: Wednesday, October 19, 2011 10:01 AM
> To: solr-user@lucene.apache.org
> Cc: Dyer, James
> Subject: Re: Solr MultiValue Fields and adding values
>
>
>
>
> Thanks for the comment. Sounds like too much of a change in all
fairness... I have actually made a tweak to my DB to allow multiple
names, and storing them off the main table. my query then only needs to
query the IDs, and then the second table to get the names. but i will
keep the comments in mind and see how things go over the next while.
>
> As a side note, if i were to go down the "get doc from solr, modify,
commit back to solr" is it really that simple? run a query on Solr, get
the document, add the extra data, and insert back to solr?
>
> Thanks.
>
> --Tiernan
>
> On 19/10/2011 15:26, Dyer, James wrote:
> > While Solr/Lucene can't support
>
> true document updates, there are 2 ways you might be able to work
>
> around this in your situation.
>
>
>
>
>
>
>
> > 1. If you store all of the fields, you can write something
>
> that will read back everything already indexed to the document,
>
> append whatever data you want, then write it back. This will
>
> increase index size and possibly make indexing too slow. On the
>
> other hand, it might be more efficient than requiring the database
>
> to return everything in order.
>
>
>
>
>
>
>
> > 2. You could store your data as multiple documents per id
>
> (pick something else as your unique id). Then use the grouping
>
> functionality to roll up on your unique id whenever you query.
>
> This will mean changes to your application, probably a bigger
>
> index, and likely somewhat slower querying. But the performance
>
> losses might be slight and this seems to me like it maybe would be
>
> a good solution in your case. Perhaps it would make it so you
>
> wouldn't have to entirely re-index each month or so. See
>
> http://wiki.apache.org/solr/FieldCollapsing for more information.
>
>
>
>
>
>
>
> > James Dyer
>
>
>
> > E-Commerce Systems
>
>
>
> > Ingram Content Group
>
>
>
> > (615) 213-4311
>
>
>
>
>
>
>
>
>
>
>
> > -----Original Message-----
>
>
>
> > From: Tiernan OToole [mailto:lsmart...@gmail.com]
>
>
>
> > Sent: Wednesday, October 19, 2011 5:11 AM
>
>
>
> > To: solr-user@lucene.apache.org
>
>
>
> > Cc: Otis Gospodnetic
>
>
>
> > Subject: Re: Solr MultiValue Fields and adding values
>
>
>
>
>
>
>
>
>
>
>
> > I was hoping that wasent going to be the case... I ended up
>
> querying for
>
>
>
> > all unique IDs in the DB, and then querying for each unique
>
> ID and
>
>
>
> > getting all names, and then inserting them that way... Seems
>
> a lot
>
>
>
> > slower than in theory it really should be...
>
>
>
>
>
>
>
> > Thanks.
>
>
>
>
>
>
>
> > --Tiernan
>
>
>
>
>
>
>
> > On 18/10/2011 23:20, Otis Gospodnetic wrote:
>
>
>
> > > Hi,
>
>
>
>
>
>
>
> > > You'll need to construct the whole document and index it
>
> as such. You
>
>
>
> > can't append values to document fields.
>
>
>
>
>
>
>
> > > Otis
>
>
>
> > > ----
>
>
>
>
>
>
>
> > > Sematext :: http://sematext.com/ :: Solr - Lucene -
>
> Nutch
>
>
>
> > > Lucene ecosystem search :: http://search-lucene.com/
>
>
>
>
>
>
>
>
>
>
>
> > >> ________________________________
>
>
>
> > >> From: Tiernan OToole <lsmart...@gmail.com>
>
>
>
> > >> To: solr-user@lucene.apache.org
>
>
>
> > >> Sent: Tuesday, October 18, 2011 11:41 AM
>
>
>
> > >> Subject: Solr MultiValue Fields and adding values
>
>
>
> > >>
>
>
>
> > >> Good morning.
>
>
>
> > >>
>
>
>
> > >> I asked this question on StackOverflow, but though
>
> this group may be
>
>
>
> > >> able to help... the question is available on SO
>
> here:
>
>
>
> > http://bit.ly/r6MAWU
>
>
>
> > >>
>
>
>
> > >> here goes:
>
>
>
> > >>
>
>
>
> > >> I am building a search engine, and have a not so
>
> unique ID for a lot of
>
>
>
> > >> different names... So, for example, there could be
>
> an id of B0051QVF7A
>
>
>
> > >> which would have multiple names like "Kindle"
>
> "Amazon Kindle" "Amazon
>
>
>
> > >> Kindle 3G" "Kindle Ebook Reader" "New Kindle" etc.
>
>
>
> > >>
>
>
>
> > >> The problem, and question i have, is that i am
>
> trying to enter this data
>
>
>
> > > >from a DB of 11 ish million rows. each is being read
>
> one at a time. So i
>
>
>
> > >> dont have all the names of each ID. I am adding new
>
> documents to the
>
>
>
> > >> list each time.
>
>
>
> > >>
>
>
>
> > >> What i am trying to find out is how do i add names
>
> to an existing
>
>
>
> > >> Document? if i am reading documentation correctly,
>
> it seems to overwrite
>
>
>
> > >> the whole document, not add extra info to the
>
> field... i just want to
>
>
>
> > >> add an extra name to the document multivalue
>
> field...
>
>
>
> > >>
>
>
>
> > >> I know this could cause some weird and wonderful
>
> "issues" if a name is
>
>
>
> > >> removed (in the example above, "New Kindle" could be
>
> removed when a
>
>
>
> > >> newer Kindle gets released) but i am thinking of
>
> recreating the index
>
>
>
> > >> every now and again, to clear out issues like that
>
> (once a month or so.
>
>
>
> > >> Its taking about 45min currently to create the
>
> index).
>
>
>
> > >>
>
>
>
> > >> So, how do you add a value to a multivalue field in
>
> solr for an existing
>
>
>
> > >> document?
>
>
>
> > >>
>
>
>
> > >> Thanks in advance.
>
>
>
> > >>
>
>
>
> > >> --Tiernan
>
>
>
> > >>
>
>
>
> > >>
>
>
>
> > >>
>
>
>
>
>
>
>
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6e7SIACgkQW5AKVqf62MHy/QCdG+6+cy/PllGZ8DNO3y4N2kcy
qBsAn3JX1zINU/OJV77+9JrNT6waZ6br
=47Sr
-----END PGP SIGNATURE-----
