Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-12 Thread Ross Vandegrift
On Fri, Feb 08, 2008 at 01:07:46PM +0100, Andi Kleen wrote:
> On Thu, Feb 07, 2008 at 02:44:46PM -0500, Ross Vandegrift wrote:
> > On Wed, Feb 06, 2008 at 09:53:57AM +0100, Andi Kleen wrote:
> > My initial test is end-to-end 1000Mbps, but I've got a few different
> > packet rates.
> > 
> > > If the young/old heuristics do not work well enough anymore most
> > > likely we should try readding RED to the syn queue again. That used
> > > to be pretty effective in the early days. I don't quite remember why
> > > Linux didn't end up using it in fact.
> > 
> > I'm running juno-z with 2, 4, & 8 threads of syn flood to port 80.
> > wireshark measures 2 threads at 350pps, 4 threads at 750pps, and 8
> 
> What's the total bandwidth of the attack?

Sorry for the delay in getting back to you on this Andi.

Here's a breakdown for each attack of the pps and bandwidth:

packets/s   Mb/s

380 0.182 
715 0.343
11930.572
16460   7.896

The first three tests were done with some fixed delay inbetween syn
floods.  The last is done with all delays as small as possible.


> > threads at 1200pps.  Under no SYN flood, the server handles 750 HTTP
> > requests per second, measured via httping in flood mode.
> 
> Thanks for the tests. Could you please do an additional experiment? 
> Use sch_em or similar to add a jittering longer latency in the connection
> (as would be realistic in a real distributed DOS). Does it make a
> difference? 

Jitter on both ends makes it worse.  Jitter only on the syn flooder
end behaves approximately the same.

If I add jitter to both the flooder and the target with:
tc qdisc add dev eth0 root netem delay 50ms 100ms distribution normal
I can kill off the host with even a single thread of syn flooding,
even with a backlog queue size of 32768.


> > With a default tcp_max_syn_backlog of 1024, I can trivially prevent
> > any inbound client connections with 2 threads of syn flood.
> > Enabling tcp_syncookies brings the connection handling back up to 725
> > fetches per second.
> 
> Yes the defaults are probably too low. That's something that should
> be fixed.

Yea, with a longer queue, the server is somewhat more resiliant.  But
it's still pretty easy to overwhelm it.  syncookies is a night and day
difference.


> > At these levels the CPU impact of tcp_syncookies is nothing.  I can't
> 
> CPU impact of syncookies was never a concern. The problems are rather
> missing flow control and disabling of valuable TCP features.

Oh definitely - Willy raised the CPU issue in another mail, I was just
including my findings with a bigger CPU.

In general I agree with you for the TCP features, but in the cases where
we're enabling syncookies, it's the difference between bad connectivity and
no connectivity.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-12 Thread Ross Vandegrift
On Fri, Feb 08, 2008 at 01:07:46PM +0100, Andi Kleen wrote:
 On Thu, Feb 07, 2008 at 02:44:46PM -0500, Ross Vandegrift wrote:
  On Wed, Feb 06, 2008 at 09:53:57AM +0100, Andi Kleen wrote:
  My initial test is end-to-end 1000Mbps, but I've got a few different
  packet rates.
  
   If the young/old heuristics do not work well enough anymore most
   likely we should try readding RED to the syn queue again. That used
   to be pretty effective in the early days. I don't quite remember why
   Linux didn't end up using it in fact.
  
  I'm running juno-z with 2, 4,  8 threads of syn flood to port 80.
  wireshark measures 2 threads at 350pps, 4 threads at 750pps, and 8
 
 What's the total bandwidth of the attack?

Sorry for the delay in getting back to you on this Andi.

Here's a breakdown for each attack of the pps and bandwidth:

packets/s   Mb/s

380 0.182 
715 0.343
11930.572
16460   7.896

The first three tests were done with some fixed delay inbetween syn
floods.  The last is done with all delays as small as possible.


  threads at 1200pps.  Under no SYN flood, the server handles 750 HTTP
  requests per second, measured via httping in flood mode.
 
 Thanks for the tests. Could you please do an additional experiment? 
 Use sch_em or similar to add a jittering longer latency in the connection
 (as would be realistic in a real distributed DOS). Does it make a
 difference? 

Jitter on both ends makes it worse.  Jitter only on the syn flooder
end behaves approximately the same.

If I add jitter to both the flooder and the target with:
tc qdisc add dev eth0 root netem delay 50ms 100ms distribution normal
I can kill off the host with even a single thread of syn flooding,
even with a backlog queue size of 32768.


  With a default tcp_max_syn_backlog of 1024, I can trivially prevent
  any inbound client connections with 2 threads of syn flood.
  Enabling tcp_syncookies brings the connection handling back up to 725
  fetches per second.
 
 Yes the defaults are probably too low. That's something that should
 be fixed.

Yea, with a longer queue, the server is somewhat more resiliant.  But
it's still pretty easy to overwhelm it.  syncookies is a night and day
difference.


  At these levels the CPU impact of tcp_syncookies is nothing.  I can't
 
 CPU impact of syncookies was never a concern. The problems are rather
 missing flow control and disabling of valuable TCP features.

Oh definitely - Willy raised the CPU issue in another mail, I was just
including my findings with a bigger CPU.

In general I agree with you for the TCP features, but in the cases where
we're enabling syncookies, it's the difference between bad connectivity and
no connectivity.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-07 Thread Ross Vandegrift
On Wed, Feb 06, 2008 at 09:53:57AM +0100, Andi Kleen wrote:
> That would be useful yes -- for different bandwidths.

My initial test is end-to-end 1000Mbps, but I've got a few different
packet rates.

> If the young/old heuristics do not work well enough anymore most
> likely we should try readding RED to the syn queue again. That used
> to be pretty effective in the early days. I don't quite remember why
> Linux didn't end up using it in fact.

I'm running juno-z with 2, 4, & 8 threads of syn flood to port 80.
wireshark measures 2 threads at 350pps, 4 threads at 750pps, and 8
threads at 1200pps.  Under no SYN flood, the server handles 750 HTTP
requests per second, measured via httping in flood mode.

With a default tcp_max_syn_backlog of 1024, I can trivially prevent
any inbound client connections with 2 threads of syn flood.
Enabling tcp_syncookies brings the connection handling back up to 725
fetches per second.

If I raise the backlog to 16384, 4 threads gives me about 650 legit
requests per sec.  Going to 8 threads makes connections very unreliable - a
handful will get through every 15 to 20 seconds.  Again,
tcp_syncookies returns performance almost totally back to normal.

Cranking juno-z to the max generates me about 16kpps.  Any syn backlog
is easily overwhelmed and nothing gets through.  tcp_syncookies gets
me back to 650 requests per second.

At these levels the CPU impact of tcp_syncookies is nothing.  I can't
measure a difference.  In the real world, a 16kpps syn flood is small.
People with a distributed botnet can easily get to the hundreds of
thousands, and I have seen over million packets per second of SYN flood.


BTW, I can trigger a soft lockup BUG when I restart apache to change the
backlog during the 16kpps test-case:

BUG: soft lockup detected on CPU#1!
 [] softlockup_tick+0x96/0xa4
 [] update_process_times+0x39/0x5c
 [] smp_apic_timer_interrupt+0x5b/0x6c
 [] apic_timer_interrupt+0x1f/0x24
 [] taskstats_exit_send+0x152/0x371
 [] netlink_kernel_create+0x5/0x11c
 [] reqsk_queue_alloc+0x32/0x81
 [] lock_sock+0x8e/0x96
 [] inet_csk_listen_start+0x17/0x106
 [] inet_listen+0x3c/0x5f
 [] sys_listen+0x4a/0x66
 [] sys_socketcall+0x98/0x19e
 [] do_syscall_trace+0xab/0xb1
 [] syscall_call+0x7/0xb
 ===
BUG: soft lockup detected on CPU#3!
 [] softlockup_tick+0x96/0xa4
 [] update_process_times+0x39/0x5c
 [] smp_apic_timer_interrupt+0x5b/0x6c
 [] apic_timer_interrupt+0x1f/0x24
 [] taskstats_exit_send+0x152/0x371
 [] netlink_kernel_create+0x5/0x11c
 [] reqsk_queue_alloc+0x32/0x81
 [] lock_sock+0x8e/0x96
 [] inet_csk_listen_start+0x17/0x106
 [] inet_listen+0x3c/0x5f
 [] sys_listen+0x4a/0x66
 [] sys_socketcall+0x98/0x19e
 [] do_syscall_trace+0xab/0xb1
 [] syscall_call+0x7/0xb
 ===







-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-07 Thread Ross Vandegrift
On Wed, Feb 06, 2008 at 09:53:57AM +0100, Andi Kleen wrote:
 That would be useful yes -- for different bandwidths.

My initial test is end-to-end 1000Mbps, but I've got a few different
packet rates.

 If the young/old heuristics do not work well enough anymore most
 likely we should try readding RED to the syn queue again. That used
 to be pretty effective in the early days. I don't quite remember why
 Linux didn't end up using it in fact.

I'm running juno-z with 2, 4,  8 threads of syn flood to port 80.
wireshark measures 2 threads at 350pps, 4 threads at 750pps, and 8
threads at 1200pps.  Under no SYN flood, the server handles 750 HTTP
requests per second, measured via httping in flood mode.

With a default tcp_max_syn_backlog of 1024, I can trivially prevent
any inbound client connections with 2 threads of syn flood.
Enabling tcp_syncookies brings the connection handling back up to 725
fetches per second.

If I raise the backlog to 16384, 4 threads gives me about 650 legit
requests per sec.  Going to 8 threads makes connections very unreliable - a
handful will get through every 15 to 20 seconds.  Again,
tcp_syncookies returns performance almost totally back to normal.

Cranking juno-z to the max generates me about 16kpps.  Any syn backlog
is easily overwhelmed and nothing gets through.  tcp_syncookies gets
me back to 650 requests per second.

At these levels the CPU impact of tcp_syncookies is nothing.  I can't
measure a difference.  In the real world, a 16kpps syn flood is small.
People with a distributed botnet can easily get to the hundreds of
thousands, and I have seen over million packets per second of SYN flood.


BTW, I can trigger a soft lockup BUG when I restart apache to change the
backlog during the 16kpps test-case:

BUG: soft lockup detected on CPU#1!
 [c044d1ec] softlockup_tick+0x96/0xa4
 [c042ddb0] update_process_times+0x39/0x5c
 [c04196f7] smp_apic_timer_interrupt+0x5b/0x6c
 [c04059bf] apic_timer_interrupt+0x1f/0x24
 [c045007b] taskstats_exit_send+0x152/0x371
 [c05c007b] netlink_kernel_create+0x5/0x11c
 [c05a7415] reqsk_queue_alloc+0x32/0x81
 [c05a5aca] lock_sock+0x8e/0x96
 [c05ce8c4] inet_csk_listen_start+0x17/0x106
 [c05e720f] inet_listen+0x3c/0x5f
 [c05a3e55] sys_listen+0x4a/0x66
 [c05a4f4d] sys_socketcall+0x98/0x19e
 [c0407ef7] do_syscall_trace+0xab/0xb1
 [c0404eff] syscall_call+0x7/0xb
 ===
BUG: soft lockup detected on CPU#3!
 [c044d1ec] softlockup_tick+0x96/0xa4
 [c042ddb0] update_process_times+0x39/0x5c
 [c04196f7] smp_apic_timer_interrupt+0x5b/0x6c
 [c04059bf] apic_timer_interrupt+0x1f/0x24
 [c045007b] taskstats_exit_send+0x152/0x371
 [c05c007b] netlink_kernel_create+0x5/0x11c
 [c05a7415] reqsk_queue_alloc+0x32/0x81
 [c05a5aca] lock_sock+0x8e/0x96
 [c05ce8c4] inet_csk_listen_start+0x17/0x106
 [c05e720f] inet_listen+0x3c/0x5f
 [c05a3e55] sys_listen+0x4a/0x66
 [c05a4f4d] sys_socketcall+0x98/0x19e
 [c0407ef7] do_syscall_trace+0xab/0xb1
 [c0404eff] syscall_call+0x7/0xb
 ===







-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-05 Thread Ross Vandegrift
On Tue, Feb 05, 2008 at 09:11:06PM +0100, Andi Kleen wrote:
> > The problem is that any reasonably recent PC can generate enough
> > forged SYN packets to overwhelm reasonable SYN queues on a much more
> > powerful server.
> 
> Have you actually seen this with a recent kernel in the wild or are
> you just talking theoretically?
> 
> Linux uses some heuristics to manage the syn queue that should
> still ensure reasonable service even without cookies under attack.
> Also SYN-RECV sockets are stored in a special data structure optimized
> to use minimal resources.
> 
> It is far from the classical head drop method that was so vunerable
> to syn flooding.

I work at a hosting company and we see these kinds of issues in the
real world fairly frequently.  I would guess maybe a monthly basis.
The servers where we have seen this are typically running RHEL 4 or 5
kernels, so I can't really speak to how recent the kernel is in this
specific term.

If I can find a box that we could temporary get a kernel.org kernel
on, I'll see if I can get a real comparison together.  We have
collected a few of the more effective attack tools that have been left
on compromised systems, so it wouldn't be too difficult to get some
numbers.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-05 Thread Ross Vandegrift
On Tue, Feb 05, 2008 at 10:29:28AM -0800, Glenn Griffin wrote:
> > Syncookies are discouraged these days. They disable too many
> > valuable TCP features (window scaling, SACK) and even without them
> > the kernel is usually strong enough to defend against syn floods
> > and systems have much more memory than they used to be.
> >
> > So I don't think it makes much sense to add more code to it, sorry.
> 
> As you say the kernel is usually strong enough to defend against syn flood
> attacks, but what about the situations where it isn't?  As valuable as the TCP
> features are I would give them up if it means I'm able to connect to my sshd
> port when I otherwise would be unable to.  While increased synq sizes, better
> dropping algorithms, and minisocks are a great way to mitigate the attacks and
> in most cases are enough, there are situations where syncookies are nice.
> 
> Regardless, I would say as long as ipv4 has syncookie support it will
> accurately be viewed as a deficiency of ipv6 if it lacks support.  So perhaps
> the discussion should be we whether all the other defenses are enough to
> warrant the removal of syncookie support from ipv4.  That topic may bring in
> more opinions.

Yes, syncookies, while presenting some tradeoffs, are a necessary tool
to have.

The problem is that any reasonably recent PC can generate enough
forged SYN packets to overwhelm reasonable SYN queues on a much more
powerful server.

Imagine a server with a few hundres Apache virtual hosts.  One website
pisses off the wrong person and it impacts service for everyone.
While syncookies isn't always enough, enabling it often helps
make the server more resiliant during attacks.  And for web service, most
of the connections are short-lived connections for small pieces of data -
so I'm not really convinced that window scaling and selective ACK are all
that important.


-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-05 Thread Ross Vandegrift
On Tue, Feb 05, 2008 at 10:29:28AM -0800, Glenn Griffin wrote:
  Syncookies are discouraged these days. They disable too many
  valuable TCP features (window scaling, SACK) and even without them
  the kernel is usually strong enough to defend against syn floods
  and systems have much more memory than they used to be.
 
  So I don't think it makes much sense to add more code to it, sorry.
 
 As you say the kernel is usually strong enough to defend against syn flood
 attacks, but what about the situations where it isn't?  As valuable as the TCP
 features are I would give them up if it means I'm able to connect to my sshd
 port when I otherwise would be unable to.  While increased synq sizes, better
 dropping algorithms, and minisocks are a great way to mitigate the attacks and
 in most cases are enough, there are situations where syncookies are nice.
 
 Regardless, I would say as long as ipv4 has syncookie support it will
 accurately be viewed as a deficiency of ipv6 if it lacks support.  So perhaps
 the discussion should be we whether all the other defenses are enough to
 warrant the removal of syncookie support from ipv4.  That topic may bring in
 more opinions.

Yes, syncookies, while presenting some tradeoffs, are a necessary tool
to have.

The problem is that any reasonably recent PC can generate enough
forged SYN packets to overwhelm reasonable SYN queues on a much more
powerful server.

Imagine a server with a few hundres Apache virtual hosts.  One website
pisses off the wrong person and it impacts service for everyone.
While syncookies isn't always enough, enabling it often helps
make the server more resiliant during attacks.  And for web service, most
of the connections are short-lived connections for small pieces of data -
so I'm not really convinced that window scaling and selective ACK are all
that important.


-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Add IPv6 support to TCP SYN cookies

2008-02-05 Thread Ross Vandegrift
On Tue, Feb 05, 2008 at 09:11:06PM +0100, Andi Kleen wrote:
  The problem is that any reasonably recent PC can generate enough
  forged SYN packets to overwhelm reasonable SYN queues on a much more
  powerful server.
 
 Have you actually seen this with a recent kernel in the wild or are
 you just talking theoretically?
 
 Linux uses some heuristics to manage the syn queue that should
 still ensure reasonable service even without cookies under attack.
 Also SYN-RECV sockets are stored in a special data structure optimized
 to use minimal resources.
 
 It is far from the classical head drop method that was so vunerable
 to syn flooding.

I work at a hosting company and we see these kinds of issues in the
real world fairly frequently.  I would guess maybe a monthly basis.
The servers where we have seen this are typically running RHEL 4 or 5
kernels, so I can't really speak to how recent the kernel is in this
specific term.

If I can find a box that we could temporary get a kernel.org kernel
on, I'll see if I can get a real comparison together.  We have
collected a few of the more effective attack tools that have been left
on compromised systems, so it wouldn't be too difficult to get some
numbers.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: ifconfig changing a routing table?

2007-09-01 Thread Ross Vandegrift
On Fri, Aug 31, 2007 at 03:55:53AM +, Dan Stromberg wrote:
> Is anyone on the list familiar with ifconfig up and/or ip link up 
> changing a routing table?  How does the kernel decide what route to add 
> in such a circumstance?

By the parameteres you supplied for the interface configuration.

So, if you're getting a connected route for 10.3.0.0/16 out of dev
eth0, it's because you assigned some IP in that subnet to that device,
with a netmask of 255.255.0.0

Check that your netmask is correct.


-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: ifconfig changing a routing table?

2007-09-01 Thread Ross Vandegrift
On Fri, Aug 31, 2007 at 03:55:53AM +, Dan Stromberg wrote:
 Is anyone on the list familiar with ifconfig up and/or ip link up 
 changing a routing table?  How does the kernel decide what route to add 
 in such a circumstance?

By the parameteres you supplied for the interface configuration.

So, if you're getting a connected route for 10.3.0.0/16 out of dev
eth0, it's because you assigned some IP in that subnet to that device,
with a netmask of 255.255.0.0

Check that your netmask is correct.


-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


dump of ext3 very slow from dm LV

2007-06-30 Thread Ross Vandegrift
Hello everyone,

A while back I asked about this on the LVM mailing list, but didn't
receive a response.  Am revisiting it now, so would like a broader
audience.

I've run into some substantial read slowness when using dump or tar
to backup filesystems that are stored on LVM2 LVs.  All of my tests
are dumping ext3 filesystems to /dev/null.

I'm seeing speeds in the range of 3-4MiB/s.  If I increase dump's
blocksize to 128KiB records, speed jumps up to about 20KiB/s, which
is better but still frustratingly slow. Going to 1024KiB gets me
about 28MiB/s.

Today I loop mounted a file on my LVM, filled it up with some crap
from urandom and dumped it out.  I was able to get 50-60MiB/s.  

Is this amount of performance loss due to dm expected? My LV is just two
SATA hard disks that I put into a VG. Stupid benchmarks with dd gives me
a raw read speed from the LV of about 60MiB/sec.

I'm using kernel 2.6.18-4-686 from debian.  Any ideas on what's
slowing down dump? 
-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


dump of ext3 very slow from dm LV

2007-06-30 Thread Ross Vandegrift
Hello everyone,

A while back I asked about this on the LVM mailing list, but didn't
receive a response.  Am revisiting it now, so would like a broader
audience.

I've run into some substantial read slowness when using dump or tar
to backup filesystems that are stored on LVM2 LVs.  All of my tests
are dumping ext3 filesystems to /dev/null.

I'm seeing speeds in the range of 3-4MiB/s.  If I increase dump's
blocksize to 128KiB records, speed jumps up to about 20KiB/s, which
is better but still frustratingly slow. Going to 1024KiB gets me
about 28MiB/s.

Today I loop mounted a file on my LVM, filled it up with some crap
from urandom and dumped it out.  I was able to get 50-60MiB/s.  

Is this amount of performance loss due to dm expected? My LV is just two
SATA hard disks that I put into a VG. Stupid benchmarks with dd gives me
a raw read speed from the LV of about 60MiB/sec.

I'm using kernel 2.6.18-4-686 from debian.  Any ideas on what's
slowing down dump? 
-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Unidentified Intel wifi network card

2007-01-31 Thread Ross Vandegrift
On Wed, Jan 31, 2007 at 03:54:55PM -0500, [EMAIL PROTECTED] wrote:
> Only gotcha I know of:  Sometimes one of the ipw3945d kernel threads will
> hang in a loop if the RFKill switch is set to "kill" when the system comes
> up.  I also don't know how it handles suspend, I don't use that on my D820.

I have a D820 and I had a horrible time with the wireless for a long
time.  Every five minutes or so, the card would flip out and burn 100%
CPU making the box unusable for 15-45 seconds.

The most recent version of the firmware fixed this issue and now
things are much better.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Unidentified Intel wifi network card

2007-01-31 Thread Ross Vandegrift
On Wed, Jan 31, 2007 at 03:54:55PM -0500, [EMAIL PROTECTED] wrote:
 Only gotcha I know of:  Sometimes one of the ipw3945d kernel threads will
 hang in a loop if the RFKill switch is set to kill when the system comes
 up.  I also don't know how it handles suspend, I don't use that on my D820.

I have a D820 and I had a horrible time with the wireless for a long
time.  Every five minutes or so, the card would flip out and burn 100%
CPU making the box unusable for 15-45 seconds.

The most recent version of the firmware fixed this issue and now
things are much better.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: why can't I remove a kernel module (or: what uses a given module)?

2006-12-03 Thread Ross Vandegrift
On Sun, Dec 03, 2006 at 12:58:24PM +0100, Tomasz Chmielewski wrote:
> You mean the "Used by" column? No, it's not used by any other module 
> according to lsmod output.
> 
> Any other methods of checking what uses /dev/sda*?

There's a good chance that if it was loaded at system boot, hald or
udev may be doing something with it.

When you loaded it manually, you didn't have udev rescan for devices
so they didn't notice that you had loaded up a new disk.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: why can't I remove a kernel module (or: what uses a given module)?

2006-12-03 Thread Ross Vandegrift
On Sun, Dec 03, 2006 at 12:58:24PM +0100, Tomasz Chmielewski wrote:
 You mean the Used by column? No, it's not used by any other module 
 according to lsmod output.
 
 Any other methods of checking what uses /dev/sda*?

There's a good chance that if it was loaded at system boot, hald or
udev may be doing something with it.

When you loaded it manually, you didn't have udev rescan for devices
so they didn't notice that you had loaded up a new disk.

-- 
Ross Vandegrift
[EMAIL PROTECTED]

The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell.
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/