Ian,
Thanks for the strace tips. By performing an strace on the 4th thread I do see
several 'poll' calls:
[pid 3858] 11:38:47.332883 poll([{fd=4, events=POLLIN, revents=POLLIN}], 1,
3000) = 1
[pid 3858] 11:38:47.335494 poll([{fd=40, events=POLLOUT, revents=POLLOUT}], 1,
0) = 1
[pid 3858] 11:38:47.335704 poll([{fd=40, events=POLLIN, revents=POLLIN}], 1,
5000) = 1
[pid 3858] 11:38:47.339133 poll([{fd=40, events=POLLIN, revents=POLLIN}], 1,
5000) = 1
I did not see any of the "restart_syscall".
One interesting fact is that it opened /etc/hosts 290 times:
[pid 3858] 11:38:45.008533 open("/etc/hosts", O_RDONLY) = 4
# cat autofs.debug.out | grep "/etc/hosts" | wc
290 2030 17882
I have determined that this issue started occurring since we upgraded to kernel
2.6.21 and autofs5. Using the stock CentOS 4.2 kernel (2.6.9-22.EL) and
autofs-4.1.3-155 this behavior is not exhibited.
---
James T. Richardson, Jr.
[EMAIL PROTECTED]
eXcellence in IS Solutions, Inc.
713-862-9200 x226
-----Original Message-----
From: Ian Kent [mailto:[EMAIL PROTECTED]
Sent: Thursday, December 06, 2007 12:17 AM
To: James Richardson
Cc: [email protected]
Subject: Re: [autofs] Slow mounts when using large round robin sets
On Wed, 2007-12-05 at 11:35 -0600, James Richardson wrote:
> Hello all,
>
>
>
> I’ve been trying to perform a stack trace on the automount process
> using “strace –p `pidof automount`”. However, when strace attaches to
> this PID nothing happens. Not even when a mount is accessed. Is there
> a better way to get a stackdump of the automount process?
Try:
ps -efL|grep autom
and look at the output.
For a configuration with a single master map entry you would see
something like:
[EMAIL PROTECTED] RHEL-5]# ps -efL|grep autom
root 2089 1 2089 0 4 10:15 ? 00:00:00 automount
root 2089 1 2090 0 4 10:15 ? 00:00:00 automount
root 2089 1 2091 0 4 10:15 ? 00:00:00 automount
root 2089 1 2094 0 4 10:15 ? 00:00:00 automount
root 17284 17067 17284 0 1 15:10 pts/4 00:00:00 grep autom
The first thread is the main thread, it just sits waiting for signals.
The second and third threads are service threads and the fourth is the
thread that does the work.
You may get what's needed then by stracing the fourth process (or other
mount handling thread). The "poll" call is the givaway:
[EMAIL PROTECTED] RHEL-5]# strace -f -p 2094
Process 2091 attached with 4 threads - interrupt to quit
[pid 2094] poll( <unfinished ...>
[pid 2091] restart_syscall(<... resuming interrupted call ...> <unfinished ...>
[pid 2090] restart_syscall(<... resuming interrupted call ...> <unfinished ...>
[pid 2089] rt_sigtimedwait(~[CHLD CONT RTMIN RT_1], <unfinished ...>
[pid 2090] <... restart_syscall resumed> ) = -1 ETIMEDOUT (Connection timed
out)
and so on.
Fact is that automount doesn't do DNS lookups much so this may be
something odd in glibc.
>
>
>
> Thanks,
>
>
>
> ---
>
> James T. Richardson, Jr.
>
> [EMAIL PROTECTED]
>
> eXcellence in IS Solutions, Inc.
>
> 713-862-9200 x226
>
>
>
>
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of James Richardson
> Sent: Tuesday, November 27, 2007 10:11 AM
> To: [email protected]
> Subject: [autofs] Slow mounts when using large round robin sets
>
>
>
>
> All,
>
>
>
> I have stumbled upon an interesting situation when using AutoFS to
> mount nfs shares where the the NFS server is served from a large (over
> 20) round robin set.
>
>
>
> For example, I use NIS to distribute my auto.home NIS map. Each of my
> users has their home directory located on a central NFS server with
> multiple network interfaces (about 50). Those 50 IP’s are in a DNS
> round robin set. When using autofs to mount one of their home
> directories dynamically it can take anywhere from one to 7-8 seconds
> (usually about 2 seconds).
>
>
>
> When the DNS RR set is scaled down to only have 15-20 entries the
> mounting time is around half of one second - (.5 seconds). Mounting
> directly using the mount command is the fastest and always mounts in
> under .5 seconds.
>
>
>
> There are no errors in the syslog (messages or daemon.log). I’m using
> kernel 2.6.21 and autofs 5.0.1`-9 (and have tested this with 5.0.2-1).
>
>
>
> One interesting side note, I did some packet captures with the larger
> DNS RR set, and found that for a single autofs mount, there could be
> anywhere from 3,000-5,000 DNS queries for the NFS server.
>
>
>
> I’m puzzled as to why this is happening or how to further troubleshoot
> the issue.
>
>
>
> Thanks,
>
>
>
> ---
>
> James T. Richardson, Jr.
>
> [EMAIL PROTECTED]
>
> eXcellence in IS Solutions, Inc.
>
> 713-862-9200 x226
>
>
>
>
>
> NOTICE:
> This message may contain privileged or otherwise confidential information.
> If you are not the intended recipient, please immediately advise the sender
> by reply email and delete the message and any attachments without using,
> copying or disclosing the contents.
>
> NOTICE:
> This message may contain privileged or otherwise confidential information.
> If you are not the intended recipient, please immediately advise the sender
> by reply email and delete the message and any attachments without using,
> copying or disclosing the contents.
> _______________________________________________
> autofs mailing list
> [email protected]
> http://linux.kernel.org/mailman/listinfo/autofs
NOTICE:
This message may contain privileged or otherwise confidential information.
If you are not the intended recipient, please immediately advise the sender
by reply email and delete the message and any attachments without using,
copying or disclosing the contents.
_______________________________________________
autofs mailing list
[email protected]
http://linux.kernel.org/mailman/listinfo/autofs
