Hello David, thanks for the response. > it may not be worth it
I agree. I think it is wise to give up to simulate dirread() rigorously. plan9 is not unix. > Ori's suggestion to use Getdirentries64 on OSX might be better I cannot find getdirentries64() on my OSX. I examined david code and found the code have still some problems. (1) some directory entries are lost. (2) uid and gid is wrong. the test program is attached here.
test_dirread.c
Description: Binary data
the result is shown below.
#
# ls (official ls in osx)
#
-bash$ ls /usr/bin|wc
1108 1108 9719
-bash$ ls /dev|wc
352 352 2428
#
# david
#
-bash$ mk -f mkfile_david
9c -D__DARWIN_64_BIT_INO_T test_dirread.c
9c -D__DARWIN_64_BIT_INO_T dirread_david2.c
9l -o o.test_dirread test_dirread.o dirread_david2.o
-bash$ o.test_dirread .
.DS_Store root 501 416 0
_mp9dir.c root 501 200000320 0
_p9dir.c root 501 20000000106 0
a.out root 501 710 0
a1.c root 501 572 0
b1.c root 501 243 0
dirread.c.orig root 501 743 0
dirread1.c root 501 137 0
dirread_david.c root 501 473 0
...
-bash$ o.test_dirread /usr/bin |wc
770 3850 20936
-bash$ o.test_dirread -a /usr/bin |wc
770 3850 20936
-bash$ o.test_dirread /dev |wc
320 1600 7739
-bash$ o.test_dirread -a /dev |wc
320 1600 7739
-bash$
#
# mine
#
-bash$ mk
9c -D__DARWIN_64_BIT_INO_T test_dirread.c
9c -D__DARWIN_64_BIT_INO_T mdirread2.c
9c -D__DARWIN_64_BIT_INO_T _mp9dir.c
9l -o o.test_dirread test_dirread.o mdirread2.o _mp9dir.o
-bash$ o.test_dirread .
.DS_Store arisawa staff 644 6148
_mp9dir.c arisawa staff 644 4948
_mp9dir.o arisawa staff 644 10284
_p9dir.c arisawa staff 444 4733
a.out arisawa staff 755 9552
a1.c arisawa staff 644 741
b1.c arisawa staff 644 2390
dirread.c.orig arisawa staff 444 3699
dirread1.c arisawa staff 644 3790
dirread_david.c arisawa staff 644 3768
...
-bash$ o.test_dirread /usr/bin |wc
1108 5540 33034
-bash$ o.test_dirread -a /usr/bin |wc
1108 5540 33034
-bash$ o.test_dirread /dev |wc
352 1760 10308
-bash$ o.test_dirread -a /dev |wc
352 1760 10308
-bash$
David idea is nice. I borrowed the idea:
dir = fdopendir(dup(fd));
in mdirread2.c
mkfile
Description: Binary data
mdirread2.c
Description: Binary data
_mp9dir.c
Description: Binary data
NOTE on _p9dir.c
(A) d->length
I observed opening character device can make a problem.
I think length of those device is allowed to assign 0.
if(S_ISBLK(lst->st_mode) || S_ISCHR(lst->st_mode)){
if((fd = open(name, O_RDONLY)) >= 0){
d->length = disksize(fd, st);
close(fd);
}
}
(B) DMDEVICE
you will find the code in _p9dir.c
else if(S_ISBLK(st->st_mode)){
d->mode |= DMDEVICE;
d->qid.path = ('b'<<16)|st->st_rdev;
}
else if(S_ISCHR(st->st_mode)){
d->mode |= DMDEVICE;
d->qid.path = ('c'<<16)|st->st_rdev;
}
but I prefer
else if(S_ISBLK(st->st_mode)){
d->mode |= DMBLOCK;
d->qid.path = ('b'<<16)|st->st_rdev;
}
else if(S_ISCHR(st->st_mode)){
d->mode |= DMCHAR;
d->qid.path = ('c'<<16)|st->st_rdev;
}
(C) I fixed some codes that produce wrong uid and gid.
comments welcome
> 2017/04/12 16:05、David Arroyo <[email protected]> のメール：
>
> The problem is twofold;
>
> * The function exits early if it can't read a max-length directory
> entry, so that entry is "skipped" in subsequent calls to mygetdents.
> * As you said, we also lose any buffered dirents that haven't been
> returned from readdir yet.
>
> I think these are both fixable problems, but it may not be worth it.
> Ori's suggestion to use Getdirentries64 on OSX might be better.
>
> David
>
> On Tue, Apr 11, 2017, at 10:33 PM, arisawa wrote:
>> I did more test on david code and found a problem.
>>
>> -bash$ mk -f mkfile_david
>> -bash$ o.test_dirread -a /usr/bin |wc
>> 1084 4336 27266
>> -bash$ o.test_dirread /usr/bin |wc
>> 1084 4336 27266
>> -bash$ ls /usr/bin |wc
>> 1108 1108 9719
>>
>> option -a is for dirreadall.
>> 1108 - 1084 entries are lost.
>>
>> they are:
>> o.test_dirread /usr/bin | awk '{print $1}' | sort >/tmp/a
>> ls /usr/bin | sort >/tmp/b
>> diff /tmp/a /tmp/b
>>
>> -bash$ diff /tmp/a /tmp/b
>> 21a22
>>> SplitForks
>> 240a242,246
>>> easy_install-2.6
>>> easy_install-2.7
>>> efax
>>> efix
>>> egrep
>> 461a468,473
>>> kcc
>>> kdestroy
>>> kextutil
>>> keytool
>>> kgetcred
>>> kill.d
>> 675a688,694
>>> piconv
>>> piconv5.16
>>> piconv5.18
>>> pidpersec.d
>>> pkgbuild
>>> pkill
>>> pl
>> 880a900,904
>>> spfquery5.18
>>> splain
>>> splain5.16
>>> splain5.18
>>> split
>>
>> sorry if I make a mistake, but I suspect readdir_r() has a buffer,
>> which can make a problem in using dup().
>>
>> Kenji Arisawa.
>>
>>> 2017/04/09 13:18、arisawa <[email protected]> のメール：
>>>
>>> thanks david.
>>>
>>> using dup() is very nice idea!
>>> your code works with
>>> CFLAGS=-D__DARWIN_64_BIT_INO_T # manual is wrong
>>> and a fix:
>>> // buf = ((char*)buf) + d_reclen(buf);
>>> buf = (struct dirent *)(((char*)buf) + d_reclen(buf));
>>> and adding
>>> #define NAME_MAX 256
>>> in somewhere.
>>>
>>> now /dev is readable.
>>>
>>> one problem is left.
>>>
>>> my test code:
>>> fd = open(dirname,OREAD);
>>> if(fd < 0)
>>> fatal("%s open error",dirname);
>>> while((n = dirread(fd, &db)) > 0){
>>> print("#DBG n=%d\n",n);
>>> for(i = 0; i < n; i++)
>>> print("%s %s %s \n", db[i].name, db[i].uid, db[i].gid);
>>> }
>>> close(fd);
>>> shows for dirname=$HOME
>>> ...
>>> arch root 501
>>> bin root 501
>>> ...
>>> but they should be
>>> arch arisawa staff
>>> bin arisawa staff
>>> this problem comes from _p9dir() that is used in dirpackage().
>>>
>>> Kenji Arisawa
>>>
>>>
>>>> 2017/04/08 18:07、David Arroyo <[email protected]> のメール：
>>>>
>>>> Ignore my previous post, I was tired and forgot about dup(). How about
>>>> something like this? (attached)
>>>>
>>>> I only tested this on Ubuntu, I don't have an OS X machine. I still went
>>>> with readdir_r because the AIX and Solaris man pages for readdir were
>>>> vague about its behavior when called from multiple threads (glibc, musl,
>>>> FreeBSD look pretty safe).
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>> On Sat, Apr 8, 2017, at 03:27 AM, David Arroyo wrote:
>>>>> This should be doable with some combination of fdopendir(3) and
>>>>> readdir(3). I'm not sure how to avoid leaking memory through the
>>>>> returned DIR pointer and any memory allocated with by readdir(3).
>>>>> This is usually free'd by closedir(3), which we can't use without
>>>>> closing the underlying file.
>>>>>
>>>>> It should be OK to use free() on the return value of fdopendir, and
>>>>> stick to the uglier readdir_r(3) interface. I can definitely see why
>>>>> Russ went with the simpler system-specific interfaces on this.
>>>>>
>>>>> David
>>>>>
>>>>> On Sat, Apr 8, 2017, at 02:46 AM, Ori Bernstein wrote:
>>>>>> On Sat, 8 Apr 2017 15:21:47 +0900, arisawa <[email protected]> wrote:
>>>>>>
>>>>>>> but how to?
>>>>>>>
>>>>>>> unix doesn’t have something like fdreaddir(int fd).
>>>>>>> my guess: russ unwillingly used a low level function such as
>>>>>>> int getdirentries(int fd, char *buf, int nbytes, long *basep).
>>>>>>>
>>>>>>> readdirall() might be OK in regular usage.
>>>>>>
>>>>>> I don't use OSX regularly, although I do maintain the syscall
>>>>>> layer for Myrddin on it.
>>>>>>
>>>>>> Getdirentries64 exists, and rudimentary testing doesn't show
>>>>>> any difficulties with using it.
>>>>>>
>>>>>> --
>>>>>> Ori Bernstein
>>>>>>
>>>>>
>>>> <posix-dirread.patch>
>>>
>>
>>
> <posix-dirread2.patch>
