OK....so the the deal is...we don't know what the deal was. It seems there could have been a network issue between the Northscale instance and my Heroku app.
In a desperate attempt to right the wrong I thought...why not try to give the whole damn Memcache add on the shake? I did a downgrade, then an upgrade. This got my app a brand new memcache server instance and things have been rolling along well since then. On Tue, Oct 5, 2010 at 10:47 AM, Brandon Casci <[email protected]>wrote: > Aye.... > > > On Tue, Oct 5, 2010 at 10:37 AM, John Norman <[email protected]> wrote: > >> Maybe you should take it up in the Northscale forums ( >> http://www.northscale.com/support/policies.html). >> >> >> On Tue, Oct 5, 2010 at 9:26 AM, Brandon Casci <[email protected]>wrote: >> >>> The problem for me isn't the exception vs miss, it's data reliability. >>> >>> >>> On Tue, Oct 5, 2010 at 10:19 AM, Keenan Brock <[email protected]>wrote: >>> >>>> Hi Brandon, >>>> >>>> Once, I had changed my memcache client library and it started raising >>>> exceptions. >>>> Turns out the previous library just returned a nil, while this one >>>> raised an exception. So I hacked the library to return nil on failures >>>> rather than throw exceptions. >>>> >>>> (The previous library just returned a nil not failing) >>>> >>>> >>>> This seems like the answer for me. >>>> >>>> Maybe alter the gem and add the option to throw an exception instead of >>>> returning a nil. >>>> >>>> >>>> But it is curious that it would fail so many times. >>>> >>>> --Keenan >>>> >>>> On Oct 5, 2010, at 9:25 AM, Brandon Casci wrote: >>>> >>>> It took a few days for the ticket to get answered, but yesterday did get >>>> confirmation from Heroku that this was a problem, with Amazon EC2 causing >>>> the pain. See this: >>>> >>>> >>>> http://getsatisfaction.com/heroku/topics/sporadic_errors_with_memcache_addon >>>> >>>> I have the same symptoms they mention, though for my app it's not >>>> sporadic, it's a state. The only way I can seem to shake it is by opening >>>> the rails console and doing Rails.cache.clear, and things go back to normal >>>> until problem kicks back up again. >>>> >>>> I'd rate this problem a 9.5 out of 10 for my app. >>>> >>>> What makes things particularly difficult for me is input comes in from >>>> outside sources and sits in memcache. In this case, what a radio station is >>>> currently playing, and it should stay there until the next time until new >>>> data comes in via the API. I suppose it's fair to say that data should be >>>> placed in the DB, which is fine. I can do that, and then do a >>>> Rails.cache.fetch from there. Any miss would mean another AR query instead >>>> of a nil. That's still a problem though, because a lot of supplementary >>>> data >>>> gets queried with the song title, artwork, photos, products and more. So >>>> all >>>> those cache misses will have a recognizable negative impact. >>>> >>>> >>>> >>>> >>>> On Tue, Oct 5, 2010 at 9:13 AM, John Norman <[email protected]> wrote: >>>> >>>>> Brandon: Regretably, I'm cheating and am using the Dalli gem without >>>>> the guidance of Heroku. :-( I couldn't get memcached-northscale to compile >>>>> on my Mac. Whatever. >>>>> >>>>> In any case: >>>>> >>>>> >> Rails.cache.write "foo", "bar" >>>>> => true >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >> Rails.cache.read "foo" >>>>> => "bar" >>>>> >>>>> >>>>> >>>>> On Mon, Oct 4, 2010 at 3:53 PM, Brandon Casci >>>>> <[email protected]>wrote: >>>>> >>>>>> Yep :) >>>>>> >>>>>> >>>>>> On Mon, Oct 4, 2010 at 4:48 PM, Teng Siong Ong >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> are you sure that you are using the supported memcache gem? >>>>>>> http://docs.heroku.com/memcache >>>>>>> >>>>>>> On Mon, Oct 4, 2010 at 3:43 PM, Brandon Casci < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> It's crazy making. >>>>>>>> >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => 1 >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => 1 >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >> Rails.cache.read "mykey" >>>>>>>> => nil >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Oct 2, 2010 at 7:58 PM, Brandon Casci < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hello >>>>>>>>> >>>>>>>>> I'm new to Heroku, so I'm not sure if this is a problem, or normal >>>>>>>>> behavior. >>>>>>>>> >>>>>>>>> I'm seeing a inconsistent memcache reads. Reading a key will >>>>>>>>> sometimes alternate between nil and the value placed in memcache. >>>>>>>>> Other >>>>>>>>> times the value just seems to vanish, though that could be a sign that >>>>>>>>> memcache is full, though I can't find a way to tell if it's full. >>>>>>>>> >>>>>>>>> What do es everyone think might be happening? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ========================================= >>>>>>>>> Brandon Casci >>>>>>>>> Loudcaster >>>>>>>>> http://loudcaster.com >>>>>>>>> ========================================= >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ========================================= >>>>>>>> Brandon Casci >>>>>>>> Loudcaster >>>>>>>> http://loudcaster.com >>>>>>>> ========================================= >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Heroku" group. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> To unsubscribe from this group, send email to >>>>>>>> [email protected]<heroku%[email protected]> >>>>>>>> . >>>>>>>> For more options, visit this group at >>>>>>>> http://groups.google.com/group/heroku?hl=en. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Heroku" group. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]<heroku%[email protected]> >>>>>>> . >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/heroku?hl=en. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ========================================= >>>>>> Brandon Casci >>>>>> Loudcaster >>>>>> http://loudcaster.com >>>>>> ========================================= >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Heroku" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]<heroku%[email protected]> >>>>>> . >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/heroku?hl=en. >>>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Heroku" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<heroku%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/heroku?hl=en. >>>>> >>>> >>>> >>>> >>>> -- >>>> ========================================= >>>> Brandon Casci >>>> Loudcaster >>>> http://loudcaster.com >>>> ========================================= >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Heroku" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]. >>>> For more options, visit this group at >>>> http://groups.google.com/group/heroku?hl=en. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Heroku" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<heroku%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/heroku?hl=en. >>>> >>> >>> >>> >>> -- >>> ========================================= >>> Brandon Casci >>> Loudcaster >>> http://loudcaster.com >>> ========================================= >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Heroku" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<heroku%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/heroku?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Heroku" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<heroku%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/heroku?hl=en. >> > > > > -- > ========================================= > Brandon Casci > Loudcaster > http://loudcaster.com > ========================================= > -- ========================================= Brandon Casci Loudcaster http://loudcaster.com ========================================= -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
