RE: sendfile to nonblocking socket
> David Schwartz пишет: > > You have a misunderstanding about the semantics of 'sendfile'. > The 'sendfile' function is just a more efficient version of a > read followed by a write. If you did a read followed by a write, > it would block as well (in the read). > > > > DS > sendfile function is not just a more efficient version of a read > followed by a write. It reads from one fd and write to another at tha > same time. Please try to read 2G, and then write 2G - and how much > memory you will be need and how much time you will loose while reading > 2G from disk, but not writing them to socket. You are correct. What I meant to say was that it's just a more efficient version of 'mmap'ing a file and then 'write'ing from the 'mmap'. The 'write' to a non-blocking socket can still 'block' on disk I/O. > If you know more > efficient method to transfer file from disk to network - please advise. > Now all I want is really non-blocking sendfile. Currently sendfile is > non-blocking on network, but not on disk i/o. And when I have network > faster than disk - I get block. There are many different techniques and which is correct depends on what direction you want to go. POSIX asynchronous I/O is one possibility. Threads plus epoll is another. It really depends upon how much performance you need, how much complexity you can tolerate, and how portable you need to be. DS - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: sendfile to nonblocking socket
David Schwartz пишет: You have a misunderstanding about the semantics of 'sendfile'. The 'sendfile' function is just a more efficient version of a read followed by a write. If you did a read followed by a write, it would block as well (in the read). DS sendfile function is not just a more efficient version of a read followed by a write. It reads from one fd and write to another at tha same time. Please try to read 2G, and then write 2G - and how much memory you will be need and how much time you will loose while reading 2G from disk, but not writing them to socket. You are correct. What I meant to say was that it's just a more efficient version of 'mmap'ing a file and then 'write'ing from the 'mmap'. The 'write' to a non-blocking socket can still 'block' on disk I/O. If you know more efficient method to transfer file from disk to network - please advise. Now all I want is really non-blocking sendfile. Currently sendfile is non-blocking on network, but not on disk i/o. And when I have network faster than disk - I get block. There are many different techniques and which is correct depends on what direction you want to go. POSIX asynchronous I/O is one possibility. Threads plus epoll is another. It really depends upon how much performance you need, how much complexity you can tolerate, and how portable you need to be. DS - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: sendfile to nonblocking socket
David Schwartz пишет: You have a misunderstanding about the semantics of 'sendfile'. The 'sendfile' function is just a more efficient version of a read followed by a write. If you did a read followed by a write, it would block as well (in the read). DS sendfile function is not just a more efficient version of a read followed by a write. It reads from one fd and write to another at tha same time. Please try to read 2G, and then write 2G - and how much memory you will be need and how much time you will loose while reading 2G from disk, but not writing them to socket. If you know more efficient method to transfer file from disk to network - please advise. Now all I want is really non-blocking sendfile. Currently sendfile is non-blocking on network, but not on disk i/o. And when I have network faster than disk - I get block. Thank you, Alex - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: sendfile to nonblocking socket
David Miller wrote:
From: voron <[EMAIL PROTECTED]>
Date: Tue, 24 Apr 2007 00:13:27 +0300
As I see, nonblocking mode is enabled - sendfile sends less than asked.
The socket is marked as non-blocking, but the disk I/O is not.
It's blocking on the disk I/O not the socket part of the operation.
How can I told kernel to not block on disk I/O? I tried non-blocking on
disk i/o fd, but it seems to be ignored. Strace with both nonblocing
disk fd and socket fd in attach
#non-blocking on socket fd enabled
04:34:07 ioctl(9, FIONBIO, [1]) = 0
#non-blocking on disk fd enabled
04:34:11 ioctl(12, FIONBIO, [1])= 0
#normal sendfile
04:34:07 sendfile(9, 12, [444282], 2147477638) = 812682
#32 seconds sendfile
04:34:11 sendfile(9, 12, [261474962], 2147476846) = 2144612230
Thank you,
Alex
04:34:04 write(3, "2007/04/24 04:34:04 [info] 32390"..., 85) = 85
04:34:04 epoll_wait(11, {{EPOLLIN, {u32=3618562064, u64=61614924423184}}}, 512,
4294967295) = 1
04:34:07 accept(8, {sa_family=AF_INET, sin_port=htons(52673),
sin_addr=inet_addr("192.168.78.1")}, [5056848310527066128]) = 9
04:34:07 ioctl(9, FIONBIO, [1]) = 0
04:34:07 epoll_ctl(11, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLET, {u32=3618562369,
u64=61614924423489}}) = 0
04:34:07 epoll_wait(11, {{EPOLLIN, {u32=3618562369, u64=61614924423489}}}, 512,
60) = 1
04:34:07 recvfrom(9, "GET /3.tmp HTTP/1.0\r\nUser-Agent:"..., 1024, 0, NULL,
NULL) = 147
04:34:07 open("/var/www/cacti/htdocs/3.tmp", O_RDONLY) = 12
04:34:07 fstat(12, {st_mode=S_IFREG|0644, st_size=4294967296, ...}) = 0
04:34:07 setsockopt(9, SOL_TCP, TCP_CORK, [1], 4) = 0
04:34:07 writev(9, [{"HTTP/1.1 200 OK\r\nServer: nginx/0"..., 262}], 1) = 262
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [0], 2147475456) = 20576
04:34:07 epoll_ctl(11, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLOUT|EPOLLET,
{u32=3618562369, u64=61614924423489}}) = 0
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [20576], 2147479456) = 55568
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599975) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [76144], 2147477136) = 41676
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599973) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [117820], 2147476420) = 83352
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599972) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [201172], 2147479084) = 243110
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599971) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [444282], 2147477638) = 812682
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599968) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [1256964], 2147475964) = 7501680
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599950) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [8758644], 2147478156) = 234274688
04:34:10 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599846) = 1
04:34:10 ioctl(12, FIONBIO, [1])= 0
04:34:10 sendfile(9, 12, [24302], 2147478284) = 583464
04:34:10 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:10 ioctl(12, FIONBIO, [1])= 0
04:34:10 sendfile(9, 12, [243616796], 2147476452) = 16712076
04:34:11 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 50) = 1
04:34:11 ioctl(12, FIONBIO, [1])= 0
04:34:11 sendfile(9, 12, [260328872], 2147476056) = 1146090
04:34:11 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599766) = 1
04:34:11 ioctl(12, FIONBIO, [1])= 0
04:34:11 sendfile(9, 12, [261474962], 2147476846) = 2144612230
04:34:43 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599754) = 1
04:34:43 ioctl(12, FIONBIO, [1])= 0
04:34:43 sendfile(9, 12, [2406087192], 180104) = 611248
04:34:43 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:43 ioctl(12, FIONBIO, [1])= 0
04:34:43 sendfile(9, 12, [2406698440], 1888268856) = 1000890816
04:34:57 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 51) = 1
04:34:57 ioctl(12, FIONBIO, [1])= 0
04:34:57 sendfile(9, 12, [3407589256], 887378040) = 972440
04:34:58 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:58 ioctl(12, FIONBIO, [1])= 0
04:34:58 sendfile(9, 12, [3408561696], 886405600) = 861304
04:34:58 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599977) = 1
04:34:58 ioctl(12, FIONBIO, [1])= 0
RE: sendfile to nonblocking socket
> As I see, nonblocking mode is enabled - sendfile sends less than asked. > But 2G via single 30 seconds sendfile call - this is blocking call. How > can I avoid that? I prefer sendfile as fastest way to send file > content to network socket. The problem with sendfile block on > nonblocking socket has place only when I'm using network connection, > that is faster than my hard disk, for example gigabit NIC or localhost. > When I use 100Mbit NIC, which is slower, than my hard disk, I got small > and fast sendfile calls without blocking. > > My kernel is Linux 2.6.20.3-grsec x86_64. I verified that also on > 2.6.18 - same results. Please advise. You have a misunderstanding about the semantics of 'sendfile'. The 'sendfile' function is just a more efficient version of a read followed by a write. If you did a read followed by a write, it would block as well (in the read). DS - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: sendfile to nonblocking socket
From: voron <[EMAIL PROTECTED]> Date: Tue, 24 Apr 2007 00:13:27 +0300 > As I see, nonblocking mode is enabled - sendfile sends less than asked. The socket is marked as non-blocking, but the disk I/O is not. It's blocking on the disk I/O not the socket part of the operation. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: sendfile to nonblocking socket
From: voron [EMAIL PROTECTED] Date: Tue, 24 Apr 2007 00:13:27 +0300 As I see, nonblocking mode is enabled - sendfile sends less than asked. The socket is marked as non-blocking, but the disk I/O is not. It's blocking on the disk I/O not the socket part of the operation. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: sendfile to nonblocking socket
As I see, nonblocking mode is enabled - sendfile sends less than asked. But 2G via single 30 seconds sendfile call - this is blocking call. How can I avoid that? I prefer sendfile as fastest way to send file content to network socket. The problem with sendfile block on nonblocking socket has place only when I'm using network connection, that is faster than my hard disk, for example gigabit NIC or localhost. When I use 100Mbit NIC, which is slower, than my hard disk, I got small and fast sendfile calls without blocking. My kernel is Linux 2.6.20.3-grsec x86_64. I verified that also on 2.6.18 - same results. Please advise. You have a misunderstanding about the semantics of 'sendfile'. The 'sendfile' function is just a more efficient version of a read followed by a write. If you did a read followed by a write, it would block as well (in the read). DS - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: sendfile to nonblocking socket
David Miller wrote:
From: voron [EMAIL PROTECTED]
Date: Tue, 24 Apr 2007 00:13:27 +0300
As I see, nonblocking mode is enabled - sendfile sends less than asked.
The socket is marked as non-blocking, but the disk I/O is not.
It's blocking on the disk I/O not the socket part of the operation.
How can I told kernel to not block on disk I/O? I tried non-blocking on
disk i/o fd, but it seems to be ignored. Strace with both nonblocing
disk fd and socket fd in attach
#non-blocking on socket fd enabled
04:34:07 ioctl(9, FIONBIO, [1]) = 0
#non-blocking on disk fd enabled
04:34:11 ioctl(12, FIONBIO, [1])= 0
#normal sendfile
04:34:07 sendfile(9, 12, [444282], 2147477638) = 812682
#32 seconds sendfile
04:34:11 sendfile(9, 12, [261474962], 2147476846) = 2144612230
Thank you,
Alex
04:34:04 write(3, 2007/04/24 04:34:04 [info] 32390..., 85) = 85
04:34:04 epoll_wait(11, {{EPOLLIN, {u32=3618562064, u64=61614924423184}}}, 512,
4294967295) = 1
04:34:07 accept(8, {sa_family=AF_INET, sin_port=htons(52673),
sin_addr=inet_addr(192.168.78.1)}, [5056848310527066128]) = 9
04:34:07 ioctl(9, FIONBIO, [1]) = 0
04:34:07 epoll_ctl(11, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLET, {u32=3618562369,
u64=61614924423489}}) = 0
04:34:07 epoll_wait(11, {{EPOLLIN, {u32=3618562369, u64=61614924423489}}}, 512,
60) = 1
04:34:07 recvfrom(9, GET /3.tmp HTTP/1.0\r\nUser-Agent:..., 1024, 0, NULL,
NULL) = 147
04:34:07 open(/var/www/cacti/htdocs/3.tmp, O_RDONLY) = 12
04:34:07 fstat(12, {st_mode=S_IFREG|0644, st_size=4294967296, ...}) = 0
04:34:07 setsockopt(9, SOL_TCP, TCP_CORK, [1], 4) = 0
04:34:07 writev(9, [{HTTP/1.1 200 OK\r\nServer: nginx/0..., 262}], 1) = 262
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [0], 2147475456) = 20576
04:34:07 epoll_ctl(11, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLOUT|EPOLLET,
{u32=3618562369, u64=61614924423489}}) = 0
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [20576], 2147479456) = 55568
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599975) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [76144], 2147477136) = 41676
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599973) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [117820], 2147476420) = 83352
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599972) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [201172], 2147479084) = 243110
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599971) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [444282], 2147477638) = 812682
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599968) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [1256964], 2147475964) = 7501680
04:34:07 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599950) = 1
04:34:07 ioctl(12, FIONBIO, [1])= 0
04:34:07 sendfile(9, 12, [8758644], 2147478156) = 234274688
04:34:10 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599846) = 1
04:34:10 ioctl(12, FIONBIO, [1])= 0
04:34:10 sendfile(9, 12, [24302], 2147478284) = 583464
04:34:10 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:10 ioctl(12, FIONBIO, [1])= 0
04:34:10 sendfile(9, 12, [243616796], 2147476452) = 16712076
04:34:11 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 50) = 1
04:34:11 ioctl(12, FIONBIO, [1])= 0
04:34:11 sendfile(9, 12, [260328872], 2147476056) = 1146090
04:34:11 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599766) = 1
04:34:11 ioctl(12, FIONBIO, [1])= 0
04:34:11 sendfile(9, 12, [261474962], 2147476846) = 2144612230
04:34:43 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599754) = 1
04:34:43 ioctl(12, FIONBIO, [1])= 0
04:34:43 sendfile(9, 12, [2406087192], 180104) = 611248
04:34:43 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:43 ioctl(12, FIONBIO, [1])= 0
04:34:43 sendfile(9, 12, [2406698440], 1888268856) = 1000890816
04:34:57 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 51) = 1
04:34:57 ioctl(12, FIONBIO, [1])= 0
04:34:57 sendfile(9, 12, [3407589256], 887378040) = 972440
04:34:58 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 60) = 1
04:34:58 ioctl(12, FIONBIO, [1])= 0
04:34:58 sendfile(9, 12, [3408561696], 886405600) = 861304
04:34:58 epoll_wait(11, {{EPOLLOUT, {u32=3618562369, u64=61614924423489}}},
512, 599977) = 1
04:34:58 ioctl(12, FIONBIO, [1])= 0
04:34:58
Re: sendfile to nonblocking socket
David Schwartz пишет: You have a misunderstanding about the semantics of 'sendfile'. The 'sendfile' function is just a more efficient version of a read followed by a write. If you did a read followed by a write, it would block as well (in the read). DS sendfile function is not just a more efficient version of a read followed by a write. It reads from one fd and write to another at tha same time. Please try to read 2G, and then write 2G - and how much memory you will be need and how much time you will loose while reading 2G from disk, but not writing them to socket. If you know more efficient method to transfer file from disk to network - please advise. Now all I want is really non-blocking sendfile. Currently sendfile is non-blocking on network, but not on disk i/o. And when I have network faster than disk - I get block. Thank you, Alex - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
