Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-10 Thread Otis Gospodnetic
Hi, - Original Message From: Jake Luciani jak...@gmail.com To: solr-user@lucene.apache.org Sent: Wed, March 9, 2011 8:07:00 PM Subject: Re: True master-master fail-over without data gaps (choosing CA in CAP) Yeah sure. Let me update this on the Solandra wiki. I'll send across

Re: True master-master fail-over without data gaps

2011-03-09 Thread Michael Sokolov
is not solr's specialty. From: Otis Gospodnetic [otis_gospodne...@yahoo.com] Sent: Tuesday, March 08, 2011 11:45 PM To: solr-user@lucene.apache.org Subject: True master-master fail-over without data gaps Hello, What are some common or good ways to handle indexing

Re: True master-master fail-over without data gaps

2011-03-09 Thread Jason Rutherglen
If you're using the delta import handler the problem would seem to go away because you can have two separate masters running at all times, and if one fails, you can then point the slaves to the secondary master, that is guaranteed to be in sync because it's been importing from the same database?

RE: True master-master fail-over without data gaps

2011-03-09 Thread Robert Petersen
4:14 AM To: solr-user@lucene.apache.org Cc: Jonathan Rochkind Subject: Re: True master-master fail-over without data gaps Yes, I think this should be pushed upstream - insert a tee in the document stream so that all documents go to both masters. Then use a load balancer to make requests

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
Hi, - Original Message If you're using the delta import handler the problem would seem to go away because you can have two separate masters running at all times, and if one fails, you can then point the slaves to the secondary master, that is guaranteed to be in sync because it's

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
Hi, - Original Message From: Robert Petersen rober...@buy.com To: solr-user@lucene.apache.org Sent: Wed, March 9, 2011 11:40:56 AM Subject: RE: True master-master fail-over without data gaps If you have a wrapper, like an indexer app which prepares solr docs and sends them

Re: True master-master fail-over without data gaps

2011-03-09 Thread Jason Rutherglen
Oh, there is no DB involved.  Think of a document stream continuously coming in, a component listening to that stream, grabbing docs, and pushing it to master(s). I don't think Solr is designed for this use case, eg, I wouldn't expect deterministic results with the current architecture as

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
specialty. From: Otis Gospodnetic [otis_gospodne...@yahoo.com] Sent: Tuesday, March 08, 2011 11:45 PM To: solr-user@lucene.apache.org Subject: True master-master fail-over without data gaps Hello, What are some common or good ways

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
Hi, - Original Message Oh, there is no DB involved. Think of a document stream continuously coming in, a component listening to that stream, grabbing docs, and pushing it to master(s). I don't think Solr is designed for this use case, eg, I wouldn't expect

RE: True master-master fail-over without data gaps

2011-03-09 Thread Robert Petersen
operation, repointing the slaves to master02 and restarting or reloading them etc... -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 8:52 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ From: Otis Gospodnetic [otis_gospodne...@yahoo.com] Sent: Tuesday, March 08, 2011 11:45 PM To: solr-user@lucene.apache.org Subject: True master-master fail-over without data

Re: True master-master fail-over without data gaps

2011-03-09 Thread Walter Underwood
On Mar 9, 2011, at 9:02 AM, Otis Gospodnetic wrote: You mean it's not possible to have 2 masters that are in nearly real-time sync? How about with DRBD? I know people use DRBD to keep 2 Hadoop NNs (their edit logs) in sync to avoid the current NN SPOF, for example, so I'm thinking this

RE: True master-master fail-over without data gaps

2011-03-09 Thread Robert Petersen
...but the index resides on disk doesn't it??? lol -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 9:06 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps Hi, - Original

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 8:52 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps Hi, - Original Message From: Robert Petersen rober...@buy.com To: solr-user

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
Petersen rober...@buy.com To: solr-user@lucene.apache.org Sent: Wed, March 9, 2011 12:07:27 PM Subject: RE: True master-master fail-over without data gaps ...but the index resides on disk doesn't it??? lol -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne

Re: True master-master fail-over without data gaps

2011-03-09 Thread Markus Jelsma
RAMdisk ...but the index resides on disk doesn't it??? lol -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 9:06 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps Hi

Re: True master-master fail-over without data gaps

2011-03-09 Thread Jason Rutherglen
This is why there's block cipher cryptography. On Wed, Mar 9, 2011 at 9:11 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: On disk, yes, but only indexed, and thus far enough from the original content to make storing terms in Lucene's inverted index. Otis Sematext ::

RE: True master-master fail-over without data gaps

2011-03-09 Thread Robert Petersen
: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 8:52 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps Hi, - Original Message From: Robert Petersen rober...@buy.com To: solr-user

Re: True master-master fail-over without data gaps

2011-03-09 Thread Otis Gospodnetic
9, 2011 12:16:31 PM Subject: RE: True master-master fail-over without data gaps I guess you could put a LB between slaves and masters, never thought of that! :) -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 9:10

Re: True master-master fail-over without data gaps

2011-03-09 Thread Jonathan Rochkind
and their size. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ From: Otis Gospodnetic [otis_gospodne...@yahoo.com] Sent: Tuesday, March 08, 2011 11:45 PM To: solr-user@lucene.apache.org Subject: True master-master fail-over

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Otis Gospodnetic
Hi, - Original Message From: Walter Underwood wun...@wunderwood.org On Mar 9, 2011, at 9:02 AM, Otis Gospodnetic wrote: You mean it's not possible to have 2 masters that are in nearly real-time sync? How about with DRBD? I know people use DRBD to keep 2 Hadoop NNs (their

RE: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Robert Petersen
@lucene.apache.org Subject: Re: True master-master fail-over without data gaps (choosing CA in CAP) Hi, - Original Message From: Walter Underwood wun...@wunderwood.org On Mar 9, 2011, at 9:02 AM, Otis Gospodnetic wrote: You mean it's not possible to have 2 masters

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Otis Gospodnetic
, 2011 9:47 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps (choosing CA in CAP) Hi, - Original Message From: Walter Underwood wun...@wunderwood.org On Mar 9, 2011, at 9:02 AM, Otis Gospodnetic wrote: You mean it's

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Jake Luciani
-master fail-over without data gaps (choosing CA in CAP) Hi, - Original Message From: Walter Underwood wun...@wunderwood.org On Mar 9, 2011, at 9:02 AM, Otis Gospodnetic wrote: You mean it's not possible to have 2 masters that are in nearly real-time sync

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Smiley, David W.
[mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 9:47 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps (choosing CA in CAP) Hi, - Original Message From: Walter Underwood wun...@wunderwood.org On Mar 9, 2011, at 9

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Jason Rutherglen
- Nutch Lucene ecosystem search :: http://search-lucene.com/ -Original  Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 9:47 AM To: solr-user@lucene.apache.org Subject:  Re: True master-master fail-over without data gaps (choosing CA

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Jake Luciani
- Nutch Lucene ecosystem search :: http://search-lucene.com/ -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Wednesday, March 09, 2011 9:47 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps (choosing CA

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Otis Gospodnetic
] Sent: Wednesday, March 09, 2011 9:47 AM To: solr-user@lucene.apache.org Subject: Re: True master-master fail-over without data gaps (choosing CA in CAP) Hi, - Original Message From: Walter Underwood wun...@wunderwood.org On Mar 9, 2011, at 9:02 AM, Otis

Re: True master-master fail-over without data gaps (choosing CA in CAP)

2011-03-09 Thread Jake Luciani
-lucene.com/ - Original Message From: Jake Luciani jak...@gmail.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org Sent: Wed, March 9, 2011 6:04:13 PM Subject: Re: True master-master fail-over without data gaps (choosing CA in CAP) Jason, It's predecessor did

True master-master fail-over without data gaps

2011-03-08 Thread Otis Gospodnetic
Hello, What are some common or good ways to handle indexing (master) fail-over? Imagine you have a continuous stream of incoming documents that you have to index without losing any of them (or with losing as few of them as possible). How do you set up you masters? In other words, you can't

RE: True master-master fail-over without data gaps

2011-03-08 Thread Jonathan Rochkind
] Sent: Tuesday, March 08, 2011 11:45 PM To: solr-user@lucene.apache.org Subject: True master-master fail-over without data gaps Hello, What are some common or good ways to handle indexing (master) fail-over? Imagine you have a continuous stream of incoming documents that you have to index without