Hey Jud,
Thanks a ton for the data sample. I'll dig into it more tomorrow, but
my first glance shows:
$ grep '<entry' buzzes_ | wc -l
18756
$ grep '<title type="html">Buzz by .* from Twitter</title>' buzzes_ | wc -l
14828
$ grep '<title type="html">Buzz by .* from Google Reader</title>'
buzzes_ | wc -l
2258
So I'm pretty sure you're just seeing the morning recrawl from Google
Reader as it picks up feeds that have no real-time updates. I don't
think it's an anomaly but I'll ask around. Thanks again,
-Brett
On Thu, Mar 4, 2010 at 4:27 PM, Jud <[email protected]> wrote:
> I uploaded a sample of buzzes from near the peak of yesterday's spike
> period to this group. I didn't dig too far in, but you'll prolly have
> a better idea of what to look for. nothing jumped right out at me.
>
> Jud
>
> On Mar 4, 12:10 pm, Brett Slatkin <[email protected]> wrote:
>> Hey Jud,
>>
>> On Thu, Mar 4, 2010 at 10:36 AM, Jud Valeski <[email protected]> wrote:
>> > We see it every day... clockwork.
>>
>> Yeah it's interesting. I wonder if this is the combination of Western
>> Europe and the east coast waking up, combined with the crawling
>> schedule of Google Buzz (ie, resyndication of feeds from slower
>> providers like Twitter). Do you have any visibility into the
>> type/source of the Buzz updates that are present in that spike? I'll
>> have to dig a bit to find that.
>>
>> -Brett
>>
>> > On Wed, Mar 3, 2010 at 5:28 PM, Brett Slatkin <[email protected]> wrote:
>>
>> >> Hey Jud,
>>
>> >> On Wed, Mar 3, 2010 at 8:20 AM, Jud <[email protected]> wrote:
>> >> > We're seeing something strange with inbound POSTs from google hub
>> >> > subscribed buzz publisher
>> >> > user post topics. We have a "large" number of topics we're subscribed
>> >> > to, and everyday around 4am Eastern we see an approx. 3x spike in
>> >> > inbound POST volume that lasts for a couple of hours.
>>
>> >> > Our edge HTTP server logs are logging the flux, so we're confident
>> >> > (but not 100%) that our app isn't backing up.
>>
>> >> > As anyone who's played with queues before is aware... they can be a
>> >> > bear to chase down. I thought I'd post here just to see if anyone else
>> >> > is seeing similar behavior.
>>
>> >> > I posted this on the google buzz group with a slightly different
>> >> > flavor but haven't heard anything on that front.
>>
>> >> Thanks for the report.
>>
>> >> I do see a delivery spike on the reference hub from 1-2am PST last
>> >> night too. Not sure what the cause is. Please let me know if you see
>> >> this again. I'm trying to track down if this was overactive retries of
>> >> delivery or something else.
>>
>> >> -Brett
>
