Re: [Imap-uw] Problem with mix folders...
On Tue, 19 May 2009, Oscar del Rio wrote: Solaris 9 and earlier had scandir() in the SunOS/BSD Compatibility Library Functions, and you had to compile with /usr/ucb/cc or link with the libucb libraries, which could caused other linking problems. The usual way to compile software that uses scandir() was to have a private scandir() function which is available with many programs. Ah, yes, it's coming back to me now. I remembered that scandir() was broken, but I wasn't sure if it was due to libucb or other brokenness. For example of the latter, for the longest time Solaris had an unbelievably slow implementation of strstr(). That whole /usr/ucb/cc and libucb thing was a complete nightmare, and caused me no end of problems for many years. libucb could be a textbook example of how NOT to do a compatibility API, as it created incompatibility after incompatibility. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem with mix folders...
Gary R. Schmidt wrote: Mark Crispin wrote: The whole reason why a private Scandir is used is that, for the longest time, Solaris didn't have a scandir() call at all and when it did it was broken. I forget why. Maybe SUN finally figured out how The fix (for my problem) is to use the system-supplied scandir() for current Solaris variants. Thanks for the info! Solaris 9 and earlier had scandir() in the SunOS/BSD Compatibility Library Functions, and you had to compile with /usr/ucb/cc or link with the libucb libraries, which could caused other linking problems. The usual way to compile software that uses scandir() was to have a private scandir() function which is available with many programs. Solaris 10 (since 2005) has scandir() in the Standard C Library Functions. Solaris 9: SunOS/BSD Compatibility Library Functions scandir(3UCB) NAME scandir, alphasort - scan a directory SYNOPSIS /usr/ucb/cc [ flag... ] file... #include #include Solaris 10: Standard C Library Functions scandir(3C) NAME scandir, alphasort - scan a directory SYNOPSIS #include #include ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem with mix folders...
Timo Sirainen wrote: On Sun, 2009-05-17 at 01:05 +1000, Gary R. Schmidt wrote: Is there an IMAP test suite that doesn't cost 15K somewhere? http://imapwiki.org/ImapTest Doesn't test everything yet, feel free to add more tests :) Thanks for that. Cheers, GaryB-) ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem with mix folders...
Mark Crispin wrote: If I had to guess, it would be that the DIR_SIZE macro is miscalculating the size of the direct struct that's assigned to p in line 53, leading to a buffer overflow in line 55. Try adding an assert that verifies that DIR_SIZE(d) is greater than, or equal to, ((d->d_name + strlen(d->d_name) + 1) - d). If that assert bites, that's the cause of the problem. The whole reason why a private Scandir is used is that, for the longest time, Solaris didn't have a scandir() call at all and when it did it was broken. I forget why. Maybe SUN finally figured out how to do a working scandir() call, but if they've broken DIR_SIZE() that is bad news. Well, it's none of the above... It's lines 45 and 46 in scandir.c: 45: if ((!dirp) || (fstat (dirp->dd_fd,&stb) < 0)) return -1; 46: nlmax = stb.st_size / 24; /* guesstimate at number of files */ It relies on undefined behaviour, calling stat() on the fd inside a DIR structure. On a ZFS file system, st_size returns the number of entries in the directory, not the number of bytes allocated to names and inodes, like it did/does for an older file system type. (Who else remember 2 bytes for inode, and 14 bytes for the file name?) (On an XFS file system it returns the number of blocks allocated for directory entries * block size. I don't have any other *NIX file systems within easy reach at the moment.) The directory that caused the problem had 8 entries, and 8 / 24 == 0 ==> SEGV. The fix (for my problem) is to use the system-supplied scandir() for current Solaris variants. The code in scandir should be modified, the call to fstat() should be removed and the initial guess at nlmax set to something reasonable, say, 16. Cheers, GaryB-) ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem with mix folders...
On Sun, 2009-05-17 at 01:05 +1000, Gary R. Schmidt wrote: > Is there an IMAP test suite that doesn't cost 15K somewhere? http://imapwiki.org/ImapTest Doesn't test everything yet, feel free to add more tests :) signature.asc Description: This is a digitally signed message part ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem with mix folders...
If I had to guess, it would be that the DIR_SIZE macro is miscalculating the size of the direct struct that's assigned to p in line 53, leading to a buffer overflow in line 55. Try adding an assert that verifies that DIR_SIZE(d) is greater than, or equal to, ((d->d_name + strlen(d->d_name) + 1) - d). If that assert bites, that's the cause of the problem. The whole reason why a private Scandir is used is that, for the longest time, Solaris didn't have a scandir() call at all and when it did it was broken. I forget why. Maybe SUN finally figured out how to do a working scandir() call, but if they've broken DIR_SIZE() that is bad news. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
[Imap-uw] Problem with mix folders...
Hi, I've found a problem with mix folders, where an EXPUNGE cause a core dump. (This has also been posted on comp.mail.imap in similar form.) System: Solaris 10 x64, built with Sun Studio Compiler 12, OpenCSW SSL libraries, SSLTYPE=none, CREATEPROTO=mixproto, and mailsubdir=".MailDir". sh-3.00$ imapd * PREAUTH [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ IDLE UIDPLUS NAMESPACE CHILDREN MAILBOX-REFERRALS BINARY UNSELECT ESEARCH WITHIN SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND] Pre-authenticated user grs paranoia.mcleod-schmidt.id.au IMAP4rev1 2007e.404 at Fri, 15 May 2009 01:53:21 +1000 (EST) 1 select me2 * 61 EXISTS * 4 RECENT * OK [UIDVALIDITY 1240838034] UID validity status * OK [UIDNEXT 176] Predicted next UID * FLAGS (NonJunk \Answered \Flagged \Deleted \Draft \Seen) * OK [PERMANENTFLAGS (NonJunk \* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags * OK [UNSEEN 12] first unseen message in /home/grs/.MailDir/me2 1 OK [READ-WRITE] SELECT completed 2 expunge Segmentation Fault (core dumped) sh-3.00$ file core core:ELF 32-bit LSB core file 80386 Version 1, from 'imapd' sh-3.00$ dbx /opt/local/sbin/imapd core Reading imapd core file header read successfully Reading ld.so.1 Reading libsocket.so.1 Reading libnsl.so.1 Reading libgen.so.1 Reading libc.so.1 program terminated by signal SEGV (no mapping at the fault address) 0xfede4d01: realfree+0x0043:movl (%ecx),%eax Current function is fs_get (optimized) 38 void *block = malloc (size ? size : (size_t) 1); (dbx 1) where [1] realfree(0x8173060), at 0xfede4d01 [2] cleanfree(0x0), at 0xfede531f [3] _malloc_unlocked(0x18, 0xfefa40b8, 0x8073620, 0x0, 0x8046a68, 0x806c804), at 0xfede483b [4] malloc(0x18, 0x0, 0x0, 0x80d867e), at 0xfede4764 =>[5] fs_get(size = ???) (optimized), at 0x806c804 (line ~38) in "fs_unix.c" [6] Scandir(dirname = ???, namelist = ???, select = ???, compar = ???) (optimized), at 0x8078201 (line ~53) in "scandir.c" [7] mix_expunge(stream = ???, sequence = ???, options = ???) (optimized), at 0x80d8213 (line ~1054) in "mix.c" [8] mail_expunge_full(stream = ???, sequence = ???, options = ???) (optimized), at 0x807eb71 (line ~2507) in "mail.c" [9] main(argc = ???, argv = ???) (optimized), at 0x8061b32 (line ~660) in "imapd.c" (dbx 2) q sh-3.00$ The problem seems to be with Scandir(), and removing from the equation and using the system-supplied version fixes this problem, but I am not confident that I haven't created another problem somewhere. Is there an IMAP test suite that doesn't cost 15K somewhere? Cheers, GaryB-) ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
