> Huh? Do you mean a PIX blade in a Cisco switch-router chassis? It
> would be very useful if you could be less vague about the
> equipment in use.
Right it's a PIX blade in Cisco chassis. The PIX is running ASA version 7.0(6)
> That depends more on your customers' networking attributes
>
Huh? Do you mean a PIX blade in a Cisco switch-router chassis? It
would be very useful if you could be less vague about the
equipment in use.
Right it's a PIX blade in Cisco chassis. The PIX is running ASA version 7.0(6)
That depends more on your customers' networking attributes
then
On Dec 20 2007 23:05, Ilpo Järvinen wrote:
>>
>> Given the fact that I've had this problem for so long, over a variety
>> of networking hardware vendors and colo-facilities, this really sounds
>> good to me. It will be challenging for me to justify a kernel core
>> dump, but a simple patch to
> I do have TCP Sequence # Randomization enabled on my router.
Huh? Do you mean a PIX blade in a Cisco switch-router chassis? It
would be very useful if you could be less vague about the
equipment in use.
> However,
> if this was causing an issue, wouldn't it always occur and cause
>
On Thu, 20 Dec 2007, James Nichols wrote:
> > You'd probably should also investigate the Linux kernel,
> > especially the size and locks of the components of the Sack data
> > structures and what happens to those data structures after Sack is
> > disabled (presumably the Sack data structure is in
James Nichols wrote
> > I still dont understand.
> >
> > "tcpdump -p -n -s 1600 -c 1" doesnt reveal User data at all.
> >
> > Without any exact data from you, I am afraid nobody can help.
>
> Oh, I didn't see that you specified specific options. I'll still have
> to anonymize 2000+ IP
On Thu, 20 Dec 2007, James Nichols wrote:
> > I still dont understand.
> >
> > "tcpdump -p -n -s 1600 -c 1" doesnt reveal User data at all.
> >
> > Without any exact data from you, I am afraid nobody can help.
>
> Oh, I didn't see that you specified specific options. I'll still have
> to
> But I'd be very surprised if the router is acting as anything more
> that a network-layer device. It might perhaps have some soft connection
> state being used for generating accounting records. Being Cisco
> it's probably a switch-router, so it might carry some per-port hard
> state for
> I still dont understand.
>
> "tcpdump -p -n -s 1600 -c 1" doesnt reveal User data at all.
>
> Without any exact data from you, I am afraid nobody can help.
Oh, I didn't see that you specified specific options. I'll still have
to anonymize 2000+ IP addresses, but I think there is an open
[speculation by network engineer -- not kernel hacker -- follows]
> The router could be sooo crappy that it drops all packets from
> TCP streams that have SACK enabled and the client has opened
> 200+ SACK connections previously... something like that?
As far as any third party is concerned the
[speculation by network engineer -- not kernel hacker -- follows]
The router could be sooo crappy that it drops all packets from
TCP streams that have SACK enabled and the client has opened
200+ SACK connections previously... something like that?
As far as any third party is concerned the
I still dont understand.
tcpdump -p -n -s 1600 -c 1 doesnt reveal User data at all.
Without any exact data from you, I am afraid nobody can help.
Oh, I didn't see that you specified specific options. I'll still have
to anonymize 2000+ IP addresses, but I think there is an open source
But I'd be very surprised if the router is acting as anything more
that a network-layer device. It might perhaps have some soft connection
state being used for generating accounting records. Being Cisco
it's probably a switch-router, so it might carry some per-port hard
state for validating
On Thu, 20 Dec 2007, James Nichols wrote:
I still dont understand.
tcpdump -p -n -s 1600 -c 1 doesnt reveal User data at all.
Without any exact data from you, I am afraid nobody can help.
Oh, I didn't see that you specified specific options. I'll still have
to anonymize 2000+
James Nichols wrote
I still dont understand.
tcpdump -p -n -s 1600 -c 1 doesnt reveal User data at all.
Without any exact data from you, I am afraid nobody can help.
Oh, I didn't see that you specified specific options. I'll still have
to anonymize 2000+ IP addresses, but I
On Thu, 20 Dec 2007, James Nichols wrote:
You'd probably should also investigate the Linux kernel,
especially the size and locks of the components of the Sack data
structures and what happens to those data structures after Sack is
disabled (presumably the Sack data structure is in some
I do have TCP Sequence # Randomization enabled on my router.
Huh? Do you mean a PIX blade in a Cisco switch-router chassis? It
would be very useful if you could be less vague about the
equipment in use.
However,
if this was causing an issue, wouldn't it always occur and cause
connection
On Dec 20 2007 23:05, Ilpo Järvinen wrote:
Given the fact that I've had this problem for so long, over a variety
of networking hardware vendors and colo-facilities, this really sounds
good to me. It will be challenging for me to justify a kernel core
dump, but a simple patch to dump the
On Wed, 19 Dec 2007, Eric Dumazet wrote:
> James Nichols a écrit :
> > On 12/19/07, Eric Dumazet <[EMAIL PROTECTED]> wrote:
> > > James Nichols a écrit :
> > > > > So you see outgoing SYN packets, but no SYN replies coming from the
> > > > > remote
> > > > > peer ? (you mention ACKS, but the
> The router could be sooo crappy that it drops all packets from
> TCP streams that have SACK enabled and the client has opened
> 200+ SACK connections previously... something like that?
I don't know, maybe. My router is a fairly new model Cisco and is
pretty major (i.e. pretty expensive), so
James Nichols a écrit :
On 12/19/07, Eric Dumazet <[EMAIL PROTECTED]> wrote:
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a SYN+ACK),
Right, I meant
On Dec 19 2007 12:43, James Nichols wrote:
>On 12/19/07, Eric Dumazet <[EMAIL PROTECTED]> wrote:
>> James Nichols a écrit :
>> >> So you see outgoing SYN packets, but no SYN replies coming from the remote
>> >> peer ? (you mention ACKS, but the first packet received from the remote
>> >> peer
On 12/19/07, Eric Dumazet <[EMAIL PROTECTED]> wrote:
> James Nichols a écrit :
> >> So you see outgoing SYN packets, but no SYN replies coming from the remote
> >> peer ? (you mention ACKS, but the first packet received from the remote
> >> peer should be a SYN+ACK),
> >
> > Right, I meant to
> So you see outgoing SYN packets, but no SYN replies coming from the remote
> peer ? (you mention ACKS, but the first packet received from the remote
> peer should be a SYN+ACK),
Right, I meant to say SYN+ACK. I don't see them coming back.
> When the problem comes, instead of restarting the
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a SYN+ACK),
Right, I meant to say SYN+ACK. I don't see them coming back.
So... Really unlikely a linux
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a SYN+ACK),
Right, I meant to say SYN+ACK. I don't see them coming back.
So... Really unlikely a linux
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a SYN+ACK),
Right, I meant to say SYN+ACK. I don't see them coming back.
When the problem comes, instead of restarting the
On 12/19/07, Eric Dumazet [EMAIL PROTECTED] wrote:
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a SYN+ACK),
Right, I meant to say SYN+ACK. I
On Dec 19 2007 12:43, James Nichols wrote:
On 12/19/07, Eric Dumazet [EMAIL PROTECTED] wrote:
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a
James Nichols a écrit :
On 12/19/07, Eric Dumazet [EMAIL PROTECTED] wrote:
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the remote
peer ? (you mention ACKS, but the first packet received from the remote
peer should be a SYN+ACK),
Right, I meant to
The router could be sooo crappy that it drops all packets from
TCP streams that have SACK enabled and the client has opened
200+ SACK connections previously... something like that?
I don't know, maybe. My router is a fairly new model Cisco and is
pretty major (i.e. pretty expensive), so it's
On Wed, 19 Dec 2007, Eric Dumazet wrote:
James Nichols a écrit :
On 12/19/07, Eric Dumazet [EMAIL PROTECTED] wrote:
James Nichols a écrit :
So you see outgoing SYN packets, but no SYN replies coming from the
remote
peer ? (you mention ACKS, but the first packet received from
On Dec 18 2007 21:37, Eric Dumazet wrote:
>
> If turning off tcp_sack makes the problem go away, why dont you
> turn it off all the time ?
>
That would just be workaround. I welcome the efforts to track this;
not all users have the time to do so.
Disabling tcp_sack also disabled it kernel-wide,
James Nichols a écrit :
Well... please dont start a flame war :(
Back to your SYN_SENT problem, I suppose the remote IP is known, so you
probably could post here the result of a tcdpump ?
tcpdump -p -n -s 1600 host IP_of_problematic_peer -c 500
Most probably remote peer received too many
On 12/18/2007 02:45 PM, James Nichols wrote:
>
> I've run tcpdump for all IPs during this problem. I haven't tried
> doing it for a single explicit IP address- due to the nature of the
> workload it's very difficult to know which IPs will be hit at any
> given moment. What I did see in the
> Well... please dont start a flame war :(
>
> Back to your SYN_SENT problem, I suppose the remote IP is known, so you
> probably could post here the result of a tcdpump ?
>
> tcpdump -p -n -s 1600 host IP_of_problematic_peer -c 500
>
> Most probably remote peer received too many attempts from
> Well... please dont start a flame war :(
>
> Back to your SYN_SENT problem, I suppose the remote IP is known, so you
> probably could post here the result of a tcdpump ?
>
> tcpdump -p -n -s 1600 host IP_of_problematic_peer -c 500
>
> Most probably remote peer received too many attempts from
On Dec 18 2007 13:21, James Nichols wrote:
>
>> Well you could still blame Java. I am sure that if you program was C,
>> the problem could be narrowed down much easier.
>
>I'm curious to know how this problem would be easier to narrow down if
>it were written in C.
>
It depends on the developers
James Nichols a écrit :
Here is a purely hypothethical (and in practice unlikely) idea:
Java opens up too many sockets (more than you really request) and the
kernel, for whatever reason, does not deliver packets to programs
which have maxed out their fds. Well it would already help if the
java
> Here is a purely hypothethical (and in practice unlikely) idea:
> Java opens up too many sockets (more than you really request) and the
> kernel, for whatever reason, does not deliver packets to programs
> which have maxed out their fds. Well it would already help if the
> java blob was split
On Dec 18 2007 13:09, James Nichols wrote:
>
>> >> Well you could still blame Java. I am sure that if you program was C,
>> >> the problem could be narrowed down much easier.
>> >
>> >That may very well be true, but I can't rewrite the whole 500K line
>> >application in C at this point. Plus,
> >> Well you could still blame Java. I am sure that if you program was C,
> >> the problem could be narrowed down much easier.
> >
> >That may very well be true, but I can't rewrite the whole 500K line
> >application in C at this point. Plus, it's a web app which would be
> >"fun" to implement
On Dec 18 2007 11:45, James Nichols wrote:
>
>> Well you could still blame Java. I am sure that if you program was C,
>> the problem could be narrowed down much easier.
>
>That may very well be true, but I can't rewrite the whole 500K line
>application in C at this point. Plus, it's a web app
> Well you could still blame Java. I am sure that if you program was C,
> the problem could be narrowed down much easier.
That may very well be true, but I can't rewrite the whole 500K line
application in C at this point. Plus, it's a web app which would be
"fun" to implement in C.
--
To
On Dec 18 2007 10:34, James Nichols wrote:
>
>It's very challenging for me to upgrade the kernel as this is a
>production system and I need to run on whatever the latest RedHat
>supports for support contract reasons. I probably could do it if
>there are specific fixes that there is reason to
> Try uploading something through rsync+ssh, or scp+ssh. If it aborts
> or hangs after a while, that may be an strong indication of a crappy
> router. Also, I'd advise to upgrade to something newer like >=
> 2.6.22. There was one of those SACK-broken routers around here too,
> but it seemed to
Try uploading something through rsync+ssh, or scp+ssh. If it aborts
or hangs after a while, that may be an strong indication of a crappy
router. Also, I'd advise to upgrade to something newer like =
2.6.22. There was one of those SACK-broken routers around here too,
but it seemed to have been
On Dec 18 2007 10:34, James Nichols wrote:
It's very challenging for me to upgrade the kernel as this is a
production system and I need to run on whatever the latest RedHat
supports for support contract reasons. I probably could do it if
there are specific fixes that there is reason to believe
Well you could still blame Java. I am sure that if you program was C,
the problem could be narrowed down much easier.
That may very well be true, but I can't rewrite the whole 500K line
application in C at this point. Plus, it's a web app which would be
fun to implement in C.
--
To unsubscribe
On Dec 18 2007 11:45, James Nichols wrote:
Well you could still blame Java. I am sure that if you program was C,
the problem could be narrowed down much easier.
That may very well be true, but I can't rewrite the whole 500K line
application in C at this point. Plus, it's a web app which
Well you could still blame Java. I am sure that if you program was C,
the problem could be narrowed down much easier.
That may very well be true, but I can't rewrite the whole 500K line
application in C at this point. Plus, it's a web app which would be
fun to implement in C.
Well I do
On Dec 18 2007 13:09, James Nichols wrote:
Well you could still blame Java. I am sure that if you program was C,
the problem could be narrowed down much easier.
That may very well be true, but I can't rewrite the whole 500K line
application in C at this point. Plus, it's a web app which
Here is a purely hypothethical (and in practice unlikely) idea:
Java opens up too many sockets (more than you really request) and the
kernel, for whatever reason, does not deliver packets to programs
which have maxed out their fds. Well it would already help if the
java blob was split into
On Dec 18 2007 13:21, James Nichols wrote:
Well you could still blame Java. I am sure that if you program was C,
the problem could be narrowed down much easier.
I'm curious to know how this problem would be easier to narrow down if
it were written in C.
It depends on the developers
James Nichols a écrit :
Here is a purely hypothethical (and in practice unlikely) idea:
Java opens up too many sockets (more than you really request) and the
kernel, for whatever reason, does not deliver packets to programs
which have maxed out their fds. Well it would already help if the
java
Well... please dont start a flame war :(
Back to your SYN_SENT problem, I suppose the remote IP is known, so you
probably could post here the result of a tcdpump ?
tcpdump -p -n -s 1600 host IP_of_problematic_peer -c 500
Most probably remote peer received too many attempts from you, and a
Well... please dont start a flame war :(
Back to your SYN_SENT problem, I suppose the remote IP is known, so you
probably could post here the result of a tcdpump ?
tcpdump -p -n -s 1600 host IP_of_problematic_peer -c 500
Most probably remote peer received too many attempts from you, and a
James Nichols a écrit :
Well... please dont start a flame war :(
Back to your SYN_SENT problem, I suppose the remote IP is known, so you
probably could post here the result of a tcdpump ?
tcpdump -p -n -s 1600 host IP_of_problematic_peer -c 500
Most probably remote peer received too many
On 12/18/2007 02:45 PM, James Nichols wrote:
I've run tcpdump for all IPs during this problem. I haven't tried
doing it for a single explicit IP address- due to the nature of the
workload it's very difficult to know which IPs will be hit at any
given moment. What I did see in the full IP
On Dec 18 2007 21:37, Eric Dumazet wrote:
If turning off tcp_sack makes the problem go away, why dont you
turn it off all the time ?
That would just be workaround. I welcome the efforts to track this;
not all users have the time to do so.
Disabling tcp_sack also disabled it kernel-wide,
On Dec 14 2007 15:39, James Nichols wrote:
>
>However, after approximately 38 hours of operation, all outbound
>connection attempts get stuck in the SYN_SENT state. It happens
>instantaneously, where I go from the baseline of about 60-80 sockets
>in SYN_SENT to a count of 200 (corresponding to
On Dec 14 2007 15:39, James Nichols wrote:
However, after approximately 38 hours of operation, all outbound
connection attempts get stuck in the SYN_SENT state. It happens
instantaneously, where I go from the baseline of about 60-80 sockets
in SYN_SENT to a count of 200 (corresponding to the #
Hello,
I have a Java application that makes a large number of outbound
webservice calls over HTTP/TCP. The hosts contacted are a fixed set
of about 2000 hosts and a web service call is made to each of them
approximately every 5 mintues by a pool of 200 Java threads. Over
time, on average a
Hello,
I have a Java application that makes a large number of outbound
webservice calls over HTTP/TCP. The hosts contacted are a fixed set
of about 2000 hosts and a web service call is made to each of them
approximately every 5 mintues by a pool of 200 Java threads. Over
time, on average a
64 matches
Mail list logo
