My personal opinion is worth way less than John's, but I'd still like to
insert it here. I was dramatically affected by a software product that
I don't even subscribe to, so I'm somewhat curious why you would defend
them so readily at this juncture. Perhaps they aren't totally to
blame. But perhaps you are unaware of some of the ramifications of this
foul-up. I'm not sure. But if you were affected by a service that you
didn't have any connection to the way I was, perhaps it would be a
different story. It seems like every message that was sent to appriver
to be pattern-checked for potential spam, was then sent out to the
intended recipient, *every time it was checked against their spam
filters*. Which caused 1000 messages to be delivered per message, and
then some, which caused a crazy amount of return or bounced messages in
turn (which is where my server was hit).
Again, I don't have all the facts and I may be wrong about some of the
details, but this is what appears to have happened to me. Here is a
snippet of one of the messages that was bounced back to my server, just
FYI. This is just a snippet, and the headers were much longer, but I
just wanted to throw them out there just in case.
Received: from server128.appriver.com (HELO inbound.appriver.com)
([207.97.226.126])
by rrcs-mgw-01b.hrndva.rr.com with ESMTP; 17 May 2007 14:38:51 -0400
Received: by inbound.appriver.com (CommuniGate Pro PIPE 5.1.7)
with PIPE id 7902890; Thu, 17 May 2007 14:19:43 -0400
Received: from [207.97.230.16] (HELO server97.appriver.com)
by inbound.appriver.com (CommuniGate Pro SMTP 5.1.7)
with ESMTP id 7902398; Thu, 17 May 2007 14:18:45 -0400
Received: by server97.appriver.com (CommuniGate Pro PIPE 5.1.4)
with PIPE id 337776981; Thu, 17 May 2007 14:16:18 -0400
Received: from [74.205.4.33] (HELO inbound.appriver.com)
John T (lists) wrote:
Inserting my 2 cents here since that is all that it is worth.
In backing up what Matt said, let me relate a similar example of a
problem that occurred a year and a half ago to a major IT security
products vendor:
At about 6:15 AM PT on a week day in the middle of a normal busy week,
their content filtering servers begin to become unresponsive. At
first, it was intermittent and hard to pinpoint. But within about 45
minutes, they stopped responding completely. Well, their appliances
did what they were designed to do by default configuration, fail safe.
Block all access if the content filtering server does not respond. All
one had to do though was to log onto the appliance and change the
failsafe block to allow. But this is where the fun (not) began. There
are hundreds or more of library's, both public and private, as well as
schools, that are using those appliances and that content filtering
service. Guess what? They are bound by law to have content filtering
in place, meaning they could not turn the fail safe off. Companies and
schools and libraries started screaming bloody murder and demanded a
resolution an hour ago. The content filtering service was finally
restored about 2:30 PM if I recall correctly.
So, what happened? I mean this is a big company and it should have
things in place to prevent this. Right?
They did. As much as some one would expect them to.
They had 4 servers. The servers were fine, they were still running.
There were no software changes, and in fact their tests showed the
servers were still responding. They were located at a location with
multiple internet connections, and all tests showed the internet
connections were all up and working. Power was flowing fine and all
UPSs as well as the generator were all fine. Finally, after about 2
hours, the problem was found: My understanding is that a single module
in a enterprise router failed but in a way that was hard to find. Once
found, the hardware vendor sent a replacement part by courier to replace.
My understanding is that it cost them well over 10 grand to eliminate
that one single point of failure. And that was just for the hardware.
Just goes to prove once again that in IT, 80% of the result is 20% of
the cost. That remain 20% of result is what costs the 80%.
*John T*
*From:* Message Sniffer Community [mailto:[EMAIL PROTECTED] *On
Behalf Of *Matt
*Sent:* Friday, May 18, 2007 9:44 PM
*To:* Message Sniffer Community
*Subject:* [sniffer] Re: Appriver issue
I have something that I would also like to clear up.
When I indicated that AppRiver had removed it's contact page, it
likely just wasn't operating at the time that I was attempting to
access it. Considering their issues, it would not be a surprise to
see other issues like this caused, but it seemed suspicious since
their home page was working and not their contact page. I did note
that it was working by the time that it was pointed out that it was up.
In no way did I ever believe that Pete or Sniffer had any direct
involvement in the system that created these problems, and in no way
should this reflect badly on Pete or Sniffer as far as I am concerned.
I was slightly miffed after getting off the phone with them where
their reaction quite clearly indicated that they were aware of the
issue. I suggested that they take their servers off-line due to the
issues that were being caused, but I was probably barking up the wrong
tree. The servers weren't taken off line for another hour or so, or
maybe this is when the delivery servers caught up with the queued
E-mail destined for my client. I'm not sure why they didn't act on
this sooner. When you have a loop, it is important to stop it, and
their multi-homing made it difficult for others to block. One user
received about 500 copies of the same message (and also called them),
and there were other examples that we saw which were much more
limited. I do hope that they didn't choose to introduce new software
at 11 a.m. ET on the busiest E-mail day of the week, and that this was
only when the problems surfaced...
Everyone that deals with significant volumes of E-mail has issues from
time to time, and I wouldn't draw conclusions about AppRiver based on
just this one circumstance. I would imagine that it is hard to plan
for how to deal with a broad scale looping issue, and I'm sure this
was a learning experience for them.
Matt
#############################################################
This message is sent to you because you are subscribed to
the mailing list <[email protected]>.
To unsubscribe, E-mail to: <[EMAIL PROTECTED]>
To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]>
To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]>
Send administrative queries to <[EMAIL PROTECTED]>
