Re: Digg 4 Preview on TWiT

[Terje Marthinussen]

http://twitter.com/nk/status/17903187277
Another "not using" joke?

Re: Digg 4 Preview on TWiT

[Jeremy Davis]

That is an interesting statistic. 1 TB per node?
Care to share any more info on the specs of this cluster? Drive types/Cores
per node/etc...
-JD


On Tue, Jul 6, 2010 at 12:01 PM, Prashant Malik  wrote:

> This is a ridiculous statement by some newbie I guess , We today have a 150
> node Cassandra cluster running Inbox search supporting close to 500M users
> and over 150TB of data  growing rapidly everyday.
>
> I am on pager for this monster :) so its pretty funny to hear this
> statement.
>
> - Prashant
>
>
> On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman <
> avinash.laksh...@gmail.com> wrote:
>
>> FB Inbox Search still runs on Cassandra and will continue to do so. I
>> should know since I maintain it :).
>>
>> Cheers
>> Avinash
>>
>> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss wrote:
>>
>>> On 2010-07-05 15:40, Eric Evans wrote:
>>> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
>>> >> This person's understanding is that Facebook 'no longer contributes to
>>> >> nor uses Cassandra.':
>>> >>
>>> >> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
>>> >
>>> > Last I heard, Facebook was still using Cassandra for what they had
>>> > always used it for, Inbox Search. Last I heard, there were no plans in
>>> > place to change that.
>>>
>>> I had the opportunity to talk with some Facebook infrastructure
>>> engineers in San Francisco over the past few weeks. They are no longer
>>> using Cassandra, even for inbox search.
>>>
>>> Inbox search was intended to be an initial push for using Cassandra more
>>> broadly, not the primary target of the Cassandra design. Unfortunately,
>>> Facebook's engineers later decided that Cassandra wasn't the right
>>> answer to the right question for Facebook's purposes.
>>>
>>> That decision isn't an indictment of Cassandra's capability; it's
>>> confirmation that Cassandra isn't everything to everyone. But we already
>>> knew that. :-)
>>>
>>> --
>>> David Strauss
>>>   | da...@fourkitchens.com
>>>   | +1 512 577 5827 [mobile]
>>> Four Kitchens
>>>   | http://fourkitchens.com
>>>   | +1 512 454 6659 [office]
>>>   | +1 512 870 8453 [direct]
>>>
>>>
>>
>

Re: Digg 4 Preview on TWiT

[Benjamin Black]

Thanks, second funniest thing I've read this month!

On Tue, Jul 6, 2010 at 4:13 PM, Matt Su  wrote:
> Thanks for all your guys’ information.
>
> This thread make us raised a concern: we choose Cassandra because
> FB,Twitter,Digg are using them, and we’re doubting whether Cassandra is
> definitely trustable.
>
> The question is what action will we take, if after a few time, these big
> tech company really start to leave Cassandra.
>
>
>
> Will we have the confidence to trust Apache Cassandra, instead of following
> these tech company’s storage solution. J
>
>
>
> Thanks and Regards.
>
> 
>
> From: Prashant Malik [mailto:pma...@gmail.com]
> Sent: Tuesday, July 06, 2010 5:36 PM
> To: user@cassandra.apache.org; b...@dehora.net
> Subject: Re: Digg 4 Preview on TWiT
>
>
>
> I have gone through the appropriate channel  here  at FB  to make sure that
> the correct information is presented.
>
> the article has now been updated to
>
> " (Update: just for reference, we’re told via email that Facebook, “no
> longer contributes to nor uses Cassandra.” Update 2: we are now being told –
> and Facebook has confirmed – that Cassandra is actually still employed by
> the company for, among other things, Inbox Search.) "
>
> Thanks
> Prashant
>
> On Tue, Jul 6, 2010 at 1:19 PM, Bill de hÓra  wrote:
>
> Nonetheless, thanks for clearing that one up. And that's some serious
> volume you've got there :)
>
> Bill
>
> On Tue, 2010-07-06 at 12:01 -0700, Prashant Malik wrote:
>> This is a ridiculous statement by some newbie I guess , We today have
>> a 150 node Cassandra cluster running Inbox search supporting close to
>> 500M users
>> and over 150TB of data  growing rapidly everyday.
>>
>> I am on pager for this monster :) so its pretty funny to hear this
>> statement.
>>
>> - Prashant
>>
>
>> On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman
>>  wrote:
>>         FB Inbox Search still runs on Cassandra and will continue to
>>         do so. I should know since I maintain it :).
>>
>>         Cheers
>>         Avinash
>>
>>
>>
>>         On Tue, Jul 6, 2010 at 3:34 AM, David Strauss
>>          wrote:
>>                 On 2010-07-05 15:40, Eric Evans wrote:
>>                 > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra
>>                 wrote:
>>                 >> This person's understanding is that Facebook 'no
>>                 longer contributes to
>>                 >> nor uses Cassandra.':
>>                 >>
>>                 >>
>>                 http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
>>                 >
>>                 > Last I heard, Facebook was still using Cassandra for
>>                 what they had
>>                 > always used it for, Inbox Search. Last I heard,
>>                 there were no plans in
>>                 > place to change that.
>>
>>
>>                 I had the opportunity to talk with some Facebook
>>                 infrastructure
>>                 engineers in San Francisco over the past few weeks.
>>                 They are no longer
>>                 using Cassandra, even for inbox search.
>>
>>                 Inbox search was intended to be an initial push for
>>                 using Cassandra more
>>                 broadly, not the primary target of the Cassandra
>>                 design. Unfortunately,
>>                 Facebook's engineers later decided that Cassandra
>>                 wasn't the right
>>                 answer to the right question for Facebook's purposes.
>>
>>                 That decision isn't an indictment of Cassandra's
>>                 capability; it's
>>                 confirmation that Cassandra isn't everything to
>>                 everyone. But we already
>>                 knew that. :-)
>>
>>                 --
>>                 David Strauss
>>                   | da...@fourkitchens.com
>>                   | +1 512 577 5827 [mobile]
>>                 Four Kitchens
>>                   | http://fourkitchens.com
>>                   | +1 512 454 6659 [office]
>>                   | +1 512 870 8453 [direct]
>>
>>
>>
>>
>
>

Re: Digg 4 Preview on TWiT

[Carlos Alvarez]

On Tue, Jul 6, 2010 at 8:13 PM, Matt Su  wrote:
> This thread make us raised a concern: we choose Cassandra because
> FB,Twitter,Digg are using them, and we’re doubting whether Cassandra is
> definitely trustable.

If cassandra is definitely trustable is something that you have to
find by yourself, not something that can be settled by the technical
(or not technicals, you will never know) decitions made at other
companies.

I'd say that while looking at what others are doing is a very
important part of the decition making process, but doing what others
are doing just because others are doing that, is a kind of herd
behaviour.

Before using cassandra (or any other open source) I would advice to
wonder "what if we have to fix bugs by ourselves?". You'll always have
the source code and, even in a extreme situation, you will be able to
support your infraestructure: the beauty of open source.

Carlos.

Re: Digg 4 Preview on TWiT

[Juho Mäkinen]

Nice to hear, 150 nodes is quite a lot. I have another question on the
topic: I've read that most of the data in facebook is stored as
key=>value -pairs which are cached to memcached layer and then stored
to mysql as simple key-value -pairs for persistence (so no relations
in mysql). Are you still doing this, or have you switched to store the
key-value -pairs in cassandra instead of mysql? What else are you
storing in cassandra than just the inbox search?

 - Juho Mäkinen

On Tue, Jul 6, 2010 at 10:01 PM, Prashant Malik  wrote:
> This is a ridiculous statement by some newbie I guess , We today have a 150
> node Cassandra cluster running Inbox search supporting close to 500M users
> and over 150TB of data  growing rapidly everyday.
>
> I am on pager for this monster :) so its pretty funny to hear this
> statement.
>
> - Prashant
>
> On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman
>  wrote:
>>
>> FB Inbox Search still runs on Cassandra and will continue to do so. I
>> should know since I maintain it :).
>>
>> Cheers
>> Avinash
>>
>> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss 
>> wrote:
>>>
>>> On 2010-07-05 15:40, Eric Evans wrote:
>>> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
>>> >> This person's understanding is that Facebook 'no longer contributes to
>>> >> nor uses Cassandra.':
>>> >>
>>> >> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
>>> >
>>> > Last I heard, Facebook was still using Cassandra for what they had
>>> > always used it for, Inbox Search. Last I heard, there were no plans in
>>> > place to change that.
>>>
>>> I had the opportunity to talk with some Facebook infrastructure
>>> engineers in San Francisco over the past few weeks. They are no longer
>>> using Cassandra, even for inbox search.
>>>
>>> Inbox search was intended to be an initial push for using Cassandra more
>>> broadly, not the primary target of the Cassandra design. Unfortunately,
>>> Facebook's engineers later decided that Cassandra wasn't the right
>>> answer to the right question for Facebook's purposes.
>>>
>>> That decision isn't an indictment of Cassandra's capability; it's
>>> confirmation that Cassandra isn't everything to everyone. But we already
>>> knew that. :-)
>>>
>>> --
>>> David Strauss
>>>   | da...@fourkitchens.com
>>>   | +1 512 577 5827 [mobile]
>>> Four Kitchens
>>>   | http://fourkitchens.com
>>>   | +1 512 454 6659 [office]
>>>   | +1 512 870 8453 [direct]
>>>
>>
>
>

Re: Digg 4 Preview on TWiT

[Joe Stump]

On Jul 6, 2010, at 6:18 PM, David Strauss wrote:

> Then I'll tell my friend at Facebook to stick to topics he's qualified
> to speak about. :-)

You might want to clarify that this advice applies to all topics of discussion 
and not just Facebook related ones. ;)

--Joe

Re: Digg 4 Preview on TWiT

[David Strauss]

Then I'll tell my friend at Facebook to stick to topics he's qualified
to speak about. :-)

On 2010-07-06 13:21, Avinash Lakshman wrote:
> FB Inbox Search still runs on Cassandra and will continue to do so. I
> should know since I maintain it :).
>  
> Cheers
> Avinash
> 
> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss  > wrote:
> 
> On 2010-07-05 15:40, Eric Evans wrote:
> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
> >> This person's understanding is that Facebook 'no longer
> contributes to
> >> nor uses Cassandra.':
> >>
> >> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> >
> > Last I heard, Facebook was still using Cassandra for what they had
> > always used it for, Inbox Search. Last I heard, there were no plans in
> > place to change that.
> 
> I had the opportunity to talk with some Facebook infrastructure
> engineers in San Francisco over the past few weeks. They are no longer
> using Cassandra, even for inbox search.
> 
> Inbox search was intended to be an initial push for using Cassandra more
> broadly, not the primary target of the Cassandra design. Unfortunately,
> Facebook's engineers later decided that Cassandra wasn't the right
> answer to the right question for Facebook's purposes.
> 
> That decision isn't an indictment of Cassandra's capability; it's
> confirmation that Cassandra isn't everything to everyone. But we already
> knew that. :-)
> 
> --
> David Strauss
>   | da...@fourkitchens.com 
>   | +1 512 577 5827 [mobile]
> Four Kitchens
>   | http://fourkitchens.com 
>   | +1 512 454 6659 [office]
>   | +1 512 870 8453 [direct]
> 
> 


-- 
David Strauss
   | da...@fourkitchens.com
   | +1 512 577 5827 [mobile]
Four Kitchens
   | http://fourkitchens.com
   | +1 512 454 6659 [office]
   | +1 512 870 8453 [direct]



signature.asc
Description: OpenPGP digital signature

RE: Digg 4 Preview on TWiT

[Matt Su]

Thanks for all your guys' information.

This thread make us raised a concern: we choose Cassandra because 
FB,Twitter,Digg are using them, and we're doubting whether Cassandra is 
definitely trustable.

The question is what action will we take, if after a few time, these big tech 
company really start to leave Cassandra.

 

Will we have the confidence to trust Apache Cassandra, instead of following 
these tech company's storage solution. :-)

 

Thanks and Regards.



From: Prashant Malik [mailto:pma...@gmail.com] 
Sent: Tuesday, July 06, 2010 5:36 PM
To: user@cassandra.apache.org; b...@dehora.net
Subject: Re: Digg 4 Preview on TWiT

 

I have gone through the appropriate channel  here  at FB  to make sure that 
the correct information is presented.

the article has now been updated to 

" (Update: just for reference, we're told via email that Facebook, "no longer 
contributes to nor uses Cassandra." Update 2: we are now being told - and 
Facebook has confirmed - that Cassandra is actually still employed by the 
company for, among other things, Inbox Search.) "

Thanks
Prashant

On Tue, Jul 6, 2010 at 1:19 PM, Bill de hÓra  wrote:

Nonetheless, thanks for clearing that one up. And that's some serious
volume you've got there :)

Bill


On Tue, 2010-07-06 at 12:01 -0700, Prashant Malik wrote:
> This is a ridiculous statement by some newbie I guess , We today have
> a 150 node Cassandra cluster running Inbox search supporting close to
> 500M users
> and over 150TB of data  growing rapidly everyday.
>
> I am on pager for this monster :) so its pretty funny to hear this
> statement.
>
> - Prashant
>

> On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman
>  wrote:
> FB Inbox Search still runs on Cassandra and will continue to
> do so. I should know since I maintain it :).
>
> Cheers
> Avinash
>
>
>
> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss
>  wrote:
> On 2010-07-05 15:40, Eric Evans wrote:
> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra
> wrote:
> >> This person's understanding is that Facebook 'no
> longer contributes to
> >> nor uses Cassandra.':
> >>
> >>
> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> >
> > Last I heard, Facebook was still using Cassandra for
> what they had
> > always used it for, Inbox Search. Last I heard,
> there were no plans in
> > place to change that.
>
>
> I had the opportunity to talk with some Facebook
> infrastructure
> engineers in San Francisco over the past few weeks.
> They are no longer
> using Cassandra, even for inbox search.
>
> Inbox search was intended to be an initial push for
> using Cassandra more
> broadly, not the primary target of the Cassandra
> design. Unfortunately,
> Facebook's engineers later decided that Cassandra
> wasn't the right
> answer to the right question for Facebook's purposes.
>
> That decision isn't an indictment of Cassandra's
> capability; it's
> confirmation that Cassandra isn't everything to
> everyone. But we already
> knew that. :-)
>
> --
> David Strauss
>   | da...@fourkitchens.com
>   | +1 512 577 5827 [mobile]
> Four Kitchens
>   | http://fourkitchens.com
>   | +1 512 454 6659 [office]
>   | +1 512 870 8453 [direct]
>
>
>
>



 

Re: Digg 4 Preview on TWiT

[Prashant Malik]

I have gone through the appropriate channel  here  at FB  to make sure that
the correct information is presented.

the article has now been updated to

" (*Update*: just for reference, we’re told via email that Facebook, “no
longer contributes to nor uses Cassandra.” *Update 2*: we are now being told
– and Facebook has confirmed – that Cassandra is actually still employed by
the company for, among other things, Inbox Search.) "

Thanks
Prashant

On Tue, Jul 6, 2010 at 1:19 PM, Bill de hÓra  wrote:

> Nonetheless, thanks for clearing that one up. And that's some serious
> volume you've got there :)
>
> Bill
>
> On Tue, 2010-07-06 at 12:01 -0700, Prashant Malik wrote:
> > This is a ridiculous statement by some newbie I guess , We today have
> > a 150 node Cassandra cluster running Inbox search supporting close to
> > 500M users
> > and over 150TB of data  growing rapidly everyday.
> >
> > I am on pager for this monster :) so its pretty funny to hear this
> > statement.
> >
> > - Prashant
> >
> > On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman
> >  wrote:
> > FB Inbox Search still runs on Cassandra and will continue to
> > do so. I should know since I maintain it :).
> >
> > Cheers
> > Avinash
> >
> >
> >
> > On Tue, Jul 6, 2010 at 3:34 AM, David Strauss
> >  wrote:
> > On 2010-07-05 15:40, Eric Evans wrote:
> > > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra
> > wrote:
> > >> This person's understanding is that Facebook 'no
> > longer contributes to
> > >> nor uses Cassandra.':
> > >>
> > >>
> > http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> > >
> > > Last I heard, Facebook was still using Cassandra for
> > what they had
> > > always used it for, Inbox Search. Last I heard,
> > there were no plans in
> > > place to change that.
> >
> >
> > I had the opportunity to talk with some Facebook
> > infrastructure
> > engineers in San Francisco over the past few weeks.
> > They are no longer
> > using Cassandra, even for inbox search.
> >
> > Inbox search was intended to be an initial push for
> > using Cassandra more
> > broadly, not the primary target of the Cassandra
> > design. Unfortunately,
> > Facebook's engineers later decided that Cassandra
> > wasn't the right
> > answer to the right question for Facebook's purposes.
> >
> > That decision isn't an indictment of Cassandra's
> > capability; it's
> > confirmation that Cassandra isn't everything to
> > everyone. But we already
> > knew that. :-)
> >
> > --
> > David Strauss
> >   | da...@fourkitchens.com
> >   | +1 512 577 5827 [mobile]
> > Four Kitchens
> >   | http://fourkitchens.com
> >   | +1 512 454 6659 [office]
> >   | +1 512 870 8453 [direct]
> >
> >
> >
> >
>
>
>

Re: Digg 4 Preview on TWiT

[Bill de hÓra]

Nonetheless, thanks for clearing that one up. And that's some serious
volume you've got there :)

Bill

On Tue, 2010-07-06 at 12:01 -0700, Prashant Malik wrote:
> This is a ridiculous statement by some newbie I guess , We today have
> a 150 node Cassandra cluster running Inbox search supporting close to
> 500M users 
> and over 150TB of data  growing rapidly everyday.
> 
> I am on pager for this monster :) so its pretty funny to hear this
> statement. 
> 
> - Prashant
> 
> On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman
>  wrote:
> FB Inbox Search still runs on Cassandra and will continue to
> do so. I should know since I maintain it :).
>  
> Cheers
> Avinash
> 
> 
> 
> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss
>  wrote:
> On 2010-07-05 15:40, Eric Evans wrote:
> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra
> wrote:
> >> This person's understanding is that Facebook 'no
> longer contributes to
> >> nor uses Cassandra.':
> >>
> >>
> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> >
> > Last I heard, Facebook was still using Cassandra for
> what they had
> > always used it for, Inbox Search. Last I heard,
> there were no plans in
> > place to change that.
> 
> 
> I had the opportunity to talk with some Facebook
> infrastructure
> engineers in San Francisco over the past few weeks.
> They are no longer
> using Cassandra, even for inbox search.
> 
> Inbox search was intended to be an initial push for
> using Cassandra more
> broadly, not the primary target of the Cassandra
> design. Unfortunately,
> Facebook's engineers later decided that Cassandra
> wasn't the right
> answer to the right question for Facebook's purposes.
> 
> That decision isn't an indictment of Cassandra's
> capability; it's
> confirmation that Cassandra isn't everything to
> everyone. But we already
> knew that. :-)
> 
> --
> David Strauss
>   | da...@fourkitchens.com
>   | +1 512 577 5827 [mobile]
> Four Kitchens
>   | http://fourkitchens.com
>   | +1 512 454 6659 [office]
>   | +1 512 870 8453 [direct]
> 
> 
> 
> 

Re: Digg 4 Preview on TWiT

[Richard L. Burton III]

Thanks Avinash

It's sad to see engineers ready to switch from one solution to another,
simply because they hear rumors about Facebook or some other large website
moving away from it. The part the really bothers me is how people were ready
to look for an alternative solution before they even verified this rumor or
even heard the reason behind the rumor.

I would love to hear more about data modeling with Cassandra. I have gather
a lot of good information from reading various presentations by Benjamin
Black, Jonathan Ellis and others. The most important piece of the puzzle is
to understand how you intend to access the data and then model everything
based upon that.

Cheers,

Richard L. Burton III
http://www.SmartCodeLLC.com

On Tue, Jul 6, 2010 at 9:21 AM, Avinash Lakshman  wrote:

> FB Inbox Search still runs on Cassandra and will continue to do so. I
> should know since I maintain it :).
>
> Cheers
> Avinash
>
> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss wrote:
>
>> On 2010-07-05 15:40, Eric Evans wrote:
>> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
>> >> This person's understanding is that Facebook 'no longer contributes to
>> >> nor uses Cassandra.':
>> >>
>> >> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
>> >
>> > Last I heard, Facebook was still using Cassandra for what they had
>> > always used it for, Inbox Search. Last I heard, there were no plans in
>> > place to change that.
>>
>> I had the opportunity to talk with some Facebook infrastructure
>> engineers in San Francisco over the past few weeks. They are no longer
>> using Cassandra, even for inbox search.
>>
>> Inbox search was intended to be an initial push for using Cassandra more
>> broadly, not the primary target of the Cassandra design. Unfortunately,
>> Facebook's engineers later decided that Cassandra wasn't the right
>> answer to the right question for Facebook's purposes.
>>
>> That decision isn't an indictment of Cassandra's capability; it's
>> confirmation that Cassandra isn't everything to everyone. But we already
>> knew that. :-)
>>
>> --
>> David Strauss
>>   | da...@fourkitchens.com
>>   | +1 512 577 5827 [mobile]
>> Four Kitchens
>>   | http://fourkitchens.com
>>   | +1 512 454 6659 [office]
>>   | +1 512 870 8453 [direct]
>>
>>
>


-- 
-Richard L. Burton III

Re: Digg 4 Preview on TWiT

[Prashant Malik]

This is a ridiculous statement by some newbie I guess , We today have a 150
node Cassandra cluster running Inbox search supporting close to 500M users
and over 150TB of data  growing rapidly everyday.

I am on pager for this monster :) so its pretty funny to hear this
statement.

- Prashant

On Tue, Jul 6, 2010 at 6:21 AM, Avinash Lakshman  wrote:

> FB Inbox Search still runs on Cassandra and will continue to do so. I
> should know since I maintain it :).
>
> Cheers
> Avinash
>
> On Tue, Jul 6, 2010 at 3:34 AM, David Strauss wrote:
>
>> On 2010-07-05 15:40, Eric Evans wrote:
>> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
>> >> This person's understanding is that Facebook 'no longer contributes to
>> >> nor uses Cassandra.':
>> >>
>> >> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
>> >
>> > Last I heard, Facebook was still using Cassandra for what they had
>> > always used it for, Inbox Search. Last I heard, there were no plans in
>> > place to change that.
>>
>> I had the opportunity to talk with some Facebook infrastructure
>> engineers in San Francisco over the past few weeks. They are no longer
>> using Cassandra, even for inbox search.
>>
>> Inbox search was intended to be an initial push for using Cassandra more
>> broadly, not the primary target of the Cassandra design. Unfortunately,
>> Facebook's engineers later decided that Cassandra wasn't the right
>> answer to the right question for Facebook's purposes.
>>
>> That decision isn't an indictment of Cassandra's capability; it's
>> confirmation that Cassandra isn't everything to everyone. But we already
>> knew that. :-)
>>
>> --
>> David Strauss
>>   | da...@fourkitchens.com
>>   | +1 512 577 5827 [mobile]
>> Four Kitchens
>>   | http://fourkitchens.com
>>   | +1 512 454 6659 [office]
>>   | +1 512 870 8453 [direct]
>>
>>
>

Re: Digg 4 Preview on TWiT

[Avinash Lakshman]

FB Inbox Search still runs on Cassandra and will continue to do so. I should
know since I maintain it :).

Cheers
Avinash

On Tue, Jul 6, 2010 at 3:34 AM, David Strauss wrote:

> On 2010-07-05 15:40, Eric Evans wrote:
> > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
> >> This person's understanding is that Facebook 'no longer contributes to
> >> nor uses Cassandra.':
> >>
> >> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> >
> > Last I heard, Facebook was still using Cassandra for what they had
> > always used it for, Inbox Search. Last I heard, there were no plans in
> > place to change that.
>
> I had the opportunity to talk with some Facebook infrastructure
> engineers in San Francisco over the past few weeks. They are no longer
> using Cassandra, even for inbox search.
>
> Inbox search was intended to be an initial push for using Cassandra more
> broadly, not the primary target of the Cassandra design. Unfortunately,
> Facebook's engineers later decided that Cassandra wasn't the right
> answer to the right question for Facebook's purposes.
>
> That decision isn't an indictment of Cassandra's capability; it's
> confirmation that Cassandra isn't everything to everyone. But we already
> knew that. :-)
>
> --
> David Strauss
>   | da...@fourkitchens.com
>   | +1 512 577 5827 [mobile]
> Four Kitchens
>   | http://fourkitchens.com
>   | +1 512 454 6659 [office]
>   | +1 512 870 8453 [direct]
>
>

Re: Digg 4 Preview on TWiT

[Bill de hÓra]

On Tue, 2010-07-06 at 05:59 -0500, Colin Clark wrote:
> What were the right questions?  I view Facebook's move away from
> Cassandra as somewhat significant.

For here, I guess it's only significant if there are interesting
technical reasons. I find Cassandra's design tradeoffs close to optimal,
so I'm naturally curious if there's some axis (eg partial ordering of
writes, trading off latency for consistency etc) involved or an
interesting domain problem (eg graph processing). 

> And are they indeed using HBase then, and if so, what were the right
> answers?

Lots of companies do or don't adopt technology for non-technical
reasons. Facebook I gather has made big investments in Hadoop, I'd say
it's natural to look at things that run on that ecosystem.

Bill

> 
> On 7/6/2010 5:34 AM, David Strauss wrote: 
> > On 2010-07-05 15:40, Eric Evans wrote:
> >   
> > > On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
> > > 
> > > > This person's understanding is that Facebook 'no longer contributes to
> > > > nor uses Cassandra.':
> > > > 
> > > > http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> > > >   
> > > Last I heard, Facebook was still using Cassandra for what they had
> > > always used it for, Inbox Search. Last I heard, there were no plans in
> > > place to change that.
> > > 
> > I had the opportunity to talk with some Facebook infrastructure
> > engineers in San Francisco over the past few weeks. They are no longer
> > using Cassandra, even for inbox search.
> > 
> > Inbox search was intended to be an initial push for using Cassandra more
> > broadly, not the primary target of the Cassandra design. Unfortunately,
> > Facebook's engineers later decided that Cassandra wasn't the right
> > answer to the right question for Facebook's purposes.
> > 
> > That decision isn't an indictment of Cassandra's capability; it's
> > confirmation that Cassandra isn't everything to everyone. But we already
> > knew that. :-)
> > 
> >   

Re: Digg 4 Preview on TWiT

[Colin Clark]

What were the right questions?  I view Facebook's move away from 
Cassandra as somewhat significant.


And are they indeed using HBase then, and if so, what were the right 
answers?


On 7/6/2010 5:34 AM, David Strauss wrote:

On 2010-07-05 15:40, Eric Evans wrote:
   

On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
 

This person's understanding is that Facebook 'no longer contributes to
nor uses Cassandra.':

http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
   

Last I heard, Facebook was still using Cassandra for what they had
always used it for, Inbox Search. Last I heard, there were no plans in
place to change that.
 

I had the opportunity to talk with some Facebook infrastructure
engineers in San Francisco over the past few weeks. They are no longer
using Cassandra, even for inbox search.

Inbox search was intended to be an initial push for using Cassandra more
broadly, not the primary target of the Cassandra design. Unfortunately,
Facebook's engineers later decided that Cassandra wasn't the right
answer to the right question for Facebook's purposes.

That decision isn't an indictment of Cassandra's capability; it's
confirmation that Cassandra isn't everything to everyone. But we already
knew that. :-)

   

Re: Digg 4 Preview on TWiT

[David Strauss]

On 2010-07-05 15:40, Eric Evans wrote:
> On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
>> This person's understanding is that Facebook 'no longer contributes to
>> nor uses Cassandra.':
>>
>> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
> 
> Last I heard, Facebook was still using Cassandra for what they had
> always used it for, Inbox Search. Last I heard, there were no plans in
> place to change that.

I had the opportunity to talk with some Facebook infrastructure
engineers in San Francisco over the past few weeks. They are no longer
using Cassandra, even for inbox search.

Inbox search was intended to be an initial push for using Cassandra more
broadly, not the primary target of the Cassandra design. Unfortunately,
Facebook's engineers later decided that Cassandra wasn't the right
answer to the right question for Facebook's purposes.

That decision isn't an indictment of Cassandra's capability; it's
confirmation that Cassandra isn't everything to everyone. But we already
knew that. :-)

-- 
David Strauss
   | da...@fourkitchens.com
   | +1 512 577 5827 [mobile]
Four Kitchens
   | http://fourkitchens.com
   | +1 512 454 6659 [office]
   | +1 512 870 8453 [direct]



signature.asc
Description: OpenPGP digital signature

Re: Digg 4 Preview on TWiT

[Eric Evans]

On Sun, 2010-07-04 at 13:14 +0100, Bill de hÓra wrote:
> This person's understanding is that Facebook 'no longer contributes to
> nor uses Cassandra.':
> 
> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/

Last I heard, Facebook was still using Cassandra for what they had
always used it for, Inbox Search. Last I heard, there were no plans in
place to change that.

> I assume it's accurate - policy reasons wouldn't interest me as much
> as technical ones. 

My understanding is that their new initiatives use (or will use) HBase.
I was never able to get anyone to go into detail on why.

-- 
Eric Evans
eev...@rackspace.com

Re: Digg 4 Preview on TWiT

[S Ahmed]

Agreed, what exactly did they replace it with.

On Sun, Jul 4, 2010 at 8:14 AM, Bill de hÓra  wrote:

> On Mon, 2010-06-28 at 11:51 -0500, Eric Evans wrote:
> > On Mon, 2010-06-28 at 07:53 -0700, Kochheiser,Todd W - TOK-DITT-1 wrote:
> > > On a related but separate note: While I am fairly new to Cassandra and
> > > have only been following the mailing lists for a few months, the
> > > conversation with Kevin Rose on TWiT made me curious if the versions
> > > of Cassandra that Digg, Twitter, and Facebook are using may end up
> > > being forks of the Apache project or old versions.
> >
> > Facebook and Apache have diverged (technically we're the fork). To the
> > best of my knowledge, this has always been the case.
>
> This person's understanding is that Facebook 'no longer contributes to
> nor uses Cassandra.':
>
> http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/
>
> I assume it's accurate - policy reasons wouldn't interest me as much as
> technical ones.
>
> Bill
>
>
>

Re: Digg 4 Preview on TWiT

[Bill de hÓra]

On Mon, 2010-06-28 at 11:51 -0500, Eric Evans wrote:
> On Mon, 2010-06-28 at 07:53 -0700, Kochheiser,Todd W - TOK-DITT-1 wrote:
> > On a related but separate note: While I am fairly new to Cassandra and
> > have only been following the mailing lists for a few months, the
> > conversation with Kevin Rose on TWiT made me curious if the versions
> > of Cassandra that Digg, Twitter, and Facebook are using may end up
> > being forks of the Apache project or old versions.
> 
> Facebook and Apache have diverged (technically we're the fork). To the
> best of my knowledge, this has always been the case.

This person's understanding is that Facebook 'no longer contributes to
nor uses Cassandra.':

http://redmonk.com/sogrady/2010/05/17/beyond-cassandra/

I assume it's accurate - policy reasons wouldn't interest me as much as
technical ones. 

Bill

Re: Digg 4 Preview on TWiT

[Ryan King]

On Mon, Jun 28, 2010 at 9:35 AM, Chris Goffinet  wrote:
> Digg is not forking Cassandra. We use 0.6 for production, with a few
> in-house patches (related to our infrastructure). The biggest difference
> with our branch and apache 0.6 branch is we have the work Kelvin and Twitter
> has done in regards to Vector Clocks + Distributed Counters. This will never
> go into 0.6, but should hit 0.7 hopefully soon. We will start to move to 0.7
> once it gets more stable.

Ditto for twitter.

-ryan

Re: Digg 4 Preview on TWiT

[Eric Evans]

On Mon, 2010-06-28 at 07:53 -0700, Kochheiser,Todd W - TOK-DITT-1 wrote:
> On a related but separate note: While I am fairly new to Cassandra and
> have only been following the mailing lists for a few months, the
> conversation with Kevin Rose on TWiT made me curious if the versions
> of Cassandra that Digg, Twitter, and Facebook are using may end up
> being forks of the Apache project or old versions.

Facebook and Apache have diverged (technically we're the fork). To the
best of my knowledge, this has always been the case.

> As the Apache Cassandra project moves forward with new features, are
> these large and very public installations of Cassandra going to be
> able to continue contributing patches and features and/or accept
> patches and features from the Apache project?

I can't speak for Digg and Twitter, but as I understand it, that is the
Plan (both have indicated as much publically).

> While most recent commits appear to come from Eric Evans and Jonathan
> Ellis, the committers
> list for Cassandra does include, among many others, Facebook, Twitter,
> and Digg.

Work is underway to merge Digg's work on vector clocks, and Twitter has
done recent work on Hadoop support (among other things).


-- 
Eric Evans
eev...@rackspace.com

Re: Digg 4 Preview on TWiT

[Kelvin Kakugawa]

If you're interested:
https://issues.apache.org/jira/browse/CASSANDRA-1072
https://issues.apache.org/jira/browse/CASSANDRA-580

-Kelvin

On Mon, Jun 28, 2010 at 9:35 AM, Chris Goffinet  wrote:
> Digg is not forking Cassandra. We use 0.6 for production, with a few
> in-house patches (related to our infrastructure). The biggest difference
> with our branch and apache 0.6 branch is we have the work Kelvin and Twitter
> has done in regards to Vector Clocks + Distributed Counters. This will never
> go into 0.6, but should hit 0.7 hopefully soon. We will start to move to 0.7
> once it gets more stable.
> -Chris
> On Jun 28, 2010, at 7:53 AM, Kochheiser,Todd W - TOK-DITT-1 wrote:
>
> On yesterday’s “This Week in Tech” (TWiT) podcast with Leo Laporte
> (Wiki:http://wiki.twit.tv/wiki/TWiT_254), Kevin Rose of Digg fame was a
> guest.  He gave a public preview of the new Digg 4; it looks very nice and
> should be released in the next month or two.  He also mentioned that Digg 4
> is using Cassandra and that it is an Apache Open Source project.  He
> mentioned Twitter and how the Twitter and Digg engineers have been working
> closely on Cassandra related issues.  There was a passing reference to Digg
> also working with Facebook engineers, but I could be wrong on that point.
>
> On a related but separate note: While I am fairly new to Cassandra and have
> only been following the mailing lists for a few months, the conversation
> with Kevin Rose on TWiT made me curious if the versions of Cassandra that
> Digg, Twitter, and Facebook are using may end up being forks of the Apache
> project or old versions.  As the Apache Cassandra project moves forward with
> new features, are these large and very public installations of Cassandra
> going to be able to continue contributing patches and features and/or accept
> patches and features from the Apache project?  While most recent commits
> appear to come from Eric Evans and Jonathan Ellis, the committers list for
> Cassandra does include, among many others, Facebook, Twitter, and Digg.
>
> My apology if anyone feels this is an inappropriate post to this list.
>
> Todd
>
>
>
>
>
>

Re: Digg 4 Preview on TWiT

[Chris Goffinet]

Digg is not forking Cassandra. We use 0.6 for production, with a few in-house 
patches (related to our infrastructure). The biggest difference with our branch 
and apache 0.6 branch is we have the work Kelvin and Twitter has done in 
regards to Vector Clocks + Distributed Counters. This will never go into 0.6, 
but should hit 0.7 hopefully soon. We will start to move to 0.7 once it gets 
more stable.

-Chris

On Jun 28, 2010, at 7:53 AM, Kochheiser,Todd W - TOK-DITT-1 wrote:

> On yesterday’s “This Week in Tech” (TWiT) podcast with Leo Laporte 
> (Wiki:http://wiki.twit.tv/wiki/TWiT_254), Kevin Rose of Digg fame was a 
> guest.  He gave a public preview of the new Digg 4; it looks very nice and 
> should be released in the next month or two.  He also mentioned that Digg 4 
> is using Cassandra and that it is an Apache Open Source project.  He 
> mentioned Twitter and how the Twitter and Digg engineers have been working 
> closely on Cassandra related issues.  There was a passing reference to Digg 
> also working with Facebook engineers, but I could be wrong on that point.
>  
> On a related but separate note: While I am fairly new to Cassandra and have 
> only been following the mailing lists for a few months, the conversation with 
> Kevin Rose on TWiT made me curious if the versions of Cassandra that Digg, 
> Twitter, and Facebook are using may end up being forks of the Apache project 
> or old versions.  As the Apache Cassandra project moves forward with new 
> features, are these large and very public installations of Cassandra going to 
> be able to continue contributing patches and features and/or accept patches 
> and features from the Apache project?  While most recent commits appear to 
> come from Eric Evans and Jonathan Ellis, the committers list for Cassandra 
> does include, among many others, Facebook, Twitter, and Digg. 
>  
> My apology if anyone feels this is an inappropriate post to this list.
>  
> Todd 
>  
>  
>  
>  
>