Re: [Toybox] Valid range for stat userID and groupID?

[darken]

>
> B) Current toybox should, yes. Older ones had the API wrong:
>

>   https://github.com/landley/toybox/commit/3d64b0cc95c5


My toybox build is from April (https://github.com/d4rken/toybox/tree/sdmaid
).
But if %u and %g is used I think it doesn't try lookup in /etc/ anyways,
right?

(...)
>
Which came from the kernel.


That makes sense, so there isn't really anything to be done from this end.
Should have looked more into the toybox code.
Sorry for the false alarm and thanks for the insights guys!

If I can ever reproduce this on a device I have access to, I'll open a
ticket on the AOSP bugtracker.


2017-05-14 21:54 GMT+02:00 Rob Landley :

>
>
> On 05/14/2017 02:16 PM, darken wrote:
> >> which will try to read /etc/passwd and /etc/group which don't exist on
> > android.
> >
> > And then fall back to just use the numeric ids? Which are stored in the
> > filesystem? Inodes I think?
>
> A) Some filsystems do. Fat won't. (The whole filesystem belongs to the
> same user, you can set it at mount time.)
>
> B) Current toybox should, yes. Older ones had the API wrong:
>
>   https://github.com/landley/toybox/commit/3d64b0cc95c5
>
> > Isn't reading uids/guids from inodes be independent from bionic/musl
> etc.?
>
> What's important is what the "stat" system call returns. How that's
> implemented under the covers is hidden from userspace.
>
> > On second thought, if this data is stored and read from the filesystem,
> > couldn't a filesystem corruption cause these values?
>
> Depends on the filesystem. How many of them store 32 bit uids? (ext2 did
> this weird low 16 bits/high 16 bits straddle thing because the inode
> format got extended over the years and they could only append, not
> insert, so...)
>
> >> Does it reliably happen on those files, or is it a race condition?
> >
> > If there is such a file, it's read everytime like this.
> > I don't think it's a new issue either, I just usually don't log this.
> > Could have been the same with busybox.
>
> Whether it happens with busybox would be good to know, but I'm guessing
> it does. We're reporting what's in stat:
>
>   else if (type == 'u') out('u', stat->st_uid);
>
> Which came from either stat or lstat:
>
> if (flagf && !statfs(TT.file, (void *)));
> else if (flagf || (L ? stat : lstat)(TT.file, (void *))) {
>   perror_msg("'%s'", TT.file);
>   continue;
> }
>
> Which came from the kernel.
>
> > I've looked a bit more through the reports.
> > 4294967295 is most common but there is also: 3912069456 or 3800903683 or
> > 3780276225 or 3851690820 etc. but with those I can see that the user
> > used some root tools to mount extra partitions (e.g. apps2sd ->
> > /data/sdext2), so I would write those ones off as unrelated.
> >
> > 4294967295 accounts for 80% of the reports and 99% are from devices on
> > Android 6.0.1, 93% of those are Samsung and mostly SM-N930F or SM-G900F.
> > While I'm the first to chalk an issue up to a Samsung specific bug, this
> > could also just be coincidence due to Samsungs device popularity...
>
> Usually -1 means "failure", although how that gets put into a stat
> structure field I couldn't tell you.
>
> >> Send it to me if you find it. Bionic doesn't have a conventional
> > makefile you can use to build it outside the AOSP hairball.
> >
> > Will do if I find one, first need to see about resizing my VM though if
> > I want to download AOSP :)
>
> I plead the third.
>
> Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[Rob Landley]

On 05/14/2017 02:16 PM, darken wrote:
>> which will try to read /etc/passwd and /etc/group which don't exist on
> android.
> 
> And then fall back to just use the numeric ids? Which are stored in the
> filesystem? Inodes I think?

A) Some filsystems do. Fat won't. (The whole filesystem belongs to the
same user, you can set it at mount time.)

B) Current toybox should, yes. Older ones had the API wrong:

  https://github.com/landley/toybox/commit/3d64b0cc95c5

> Isn't reading uids/guids from inodes be independent from bionic/musl etc.?

What's important is what the "stat" system call returns. How that's
implemented under the covers is hidden from userspace.

> On second thought, if this data is stored and read from the filesystem,
> couldn't a filesystem corruption cause these values?

Depends on the filesystem. How many of them store 32 bit uids? (ext2 did
this weird low 16 bits/high 16 bits straddle thing because the inode
format got extended over the years and they could only append, not
insert, so...)

>> Does it reliably happen on those files, or is it a race condition?
> 
> If there is such a file, it's read everytime like this.
> I don't think it's a new issue either, I just usually don't log this.
> Could have been the same with busybox.

Whether it happens with busybox would be good to know, but I'm guessing
it does. We're reporting what's in stat:

  else if (type == 'u') out('u', stat->st_uid);

Which came from either stat or lstat:

if (flagf && !statfs(TT.file, (void *)));
else if (flagf || (L ? stat : lstat)(TT.file, (void *))) {
  perror_msg("'%s'", TT.file);
  continue;
}

Which came from the kernel.

> I've looked a bit more through the reports.
> 4294967295 is most common but there is also: 3912069456 or 3800903683 or
> 3780276225 or 3851690820 etc. but with those I can see that the user
> used some root tools to mount extra partitions (e.g. apps2sd ->
> /data/sdext2), so I would write those ones off as unrelated.
> 
> 4294967295 accounts for 80% of the reports and 99% are from devices on
> Android 6.0.1, 93% of those are Samsung and mostly SM-N930F or SM-G900F.
> While I'm the first to chalk an issue up to a Samsung specific bug, this
> could also just be coincidence due to Samsungs device popularity...

Usually -1 means "failure", although how that gets put into a stat
structure field I couldn't tell you.

>> Send it to me if you find it. Bionic doesn't have a conventional
> makefile you can use to build it outside the AOSP hairball.
> 
> Will do if I find one, first need to see about resizing my VM though if
> I want to download AOSP :)

I plead the third.

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[darken]

> which will try to read /etc/passwd and /etc/group which don't exist on
android.

And then fall back to just use the numeric ids? Which are stored in the
filesystem? Inodes I think?
Isn't reading uids/guids from inodes be independent from bionic/musl etc.?
On second thought, if this data is stored and read from the filesystem,
couldn't a filesystem corruption cause these values?

> Does it reliably happen on those files, or is it a race condition?

If there is such a file, it's read everytime like this.
I don't think it's a new issue either, I just usually don't log this.
Could have been the same with busybox.

I've looked a bit more through the reports.
4294967295 is most common but there is also: 3912069456 or 3800903683
or 3780276225
or 3851690820 etc. but with those I can see that the user used some root
tools to mount extra partitions (e.g. apps2sd -> /data/sdext2), so I would
write those ones off as unrelated.

4294967295 accounts for 80% of the reports and 99% are from devices on
Android 6.0.1, 93% of those are Samsung and mostly SM-N930F or SM-G900F.
While I'm the first to chalk an issue up to a Samsung specific bug, this
could also just be coincidence due to Samsungs device popularity...

> Send it to me if you find it. Bionic doesn't have a conventional makefile
you can use to build it outside the AOSP hairball.

Will do if I find one, first need to see about resizing my VM though if I
want to download AOSP :)


2017-05-14 20:21 GMT+02:00 Rob Landley :

> On 05/13/2017 12:00 PM, darken wrote:
> >> are you building with bionic? glibc won't know what android does with
> >
> > I'm building against musl libc.
>
> which will try to read /etc/passwd and /etc/group which don't exist on
> android.
>
> > I couldn't get toybox with bionic
> > (though the last time i tried was a few month ago).
>
> I'm poking at the NDK, in theory that should build against bionic. In
> practice, it's a work in progress:
>
> http://lists.landley.net/pipermail/toybox-landley.net/2017-May/008976.html
>
> >> glibc won't know what android does with uid/gid for apps.
> >
> > Doesn't bionic/3rd party only make a difference when trying to resolve
> > the ids to names?
> >
> >> (also, 4294967295 is -1.)
> >
> > Hm, -1 is not a valid guid though right? They all have to be positive
> > ranging from 0 to X?
>
> They should, yes. (The two magic values to the kernel are 0 and 65534.
> The second means "nobody" and there's actually some effort to make
> multiple processes belonging to nobody not able to access each _other_.
> See https://lwn.net/Articles/695478/ )
>
> >> i couldn't find any files on a Pixel XL where the built-in toybox
> > reported unusual uids or gids.
> >
> > It's not very common, a few files, among a few devices out of thousands.
> > I never saw this in person either, just the occasional reports.
>
> Does it reliably happen on those files, or is it a race condition?
>
> >> as usual, not using Android's libc on an Android device is likely to
> > lead to confusing results.
> >
> > I'll look into setting up a bionic toolchain again.
> > You don't happen to have an edition of "Crosscompiling toybox with
> > bionic for dummies" at hand :)?
>
> Send it to me if you find it. Bionic doesn't have a conventional
> makefile you can use to build it outside the AOSP hairball.
>
> Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[Rob Landley]

On 05/13/2017 12:00 PM, darken wrote:
>> are you building with bionic? glibc won't know what android does with
> 
> I'm building against musl libc.

which will try to read /etc/passwd and /etc/group which don't exist on
android.

> I couldn't get toybox with bionic
> (though the last time i tried was a few month ago).

I'm poking at the NDK, in theory that should build against bionic. In
practice, it's a work in progress:

http://lists.landley.net/pipermail/toybox-landley.net/2017-May/008976.html

>> glibc won't know what android does with uid/gid for apps.
> 
> Doesn't bionic/3rd party only make a difference when trying to resolve
> the ids to names?
> 
>> (also, 4294967295 is -1.)
> 
> Hm, -1 is not a valid guid though right? They all have to be positive
> ranging from 0 to X?

They should, yes. (The two magic values to the kernel are 0 and 65534.
The second means "nobody" and there's actually some effort to make
multiple processes belonging to nobody not able to access each _other_.
See https://lwn.net/Articles/695478/ )

>> i couldn't find any files on a Pixel XL where the built-in toybox
> reported unusual uids or gids.
> 
> It's not very common, a few files, among a few devices out of thousands.
> I never saw this in person either, just the occasional reports.

Does it reliably happen on those files, or is it a race condition?

>> as usual, not using Android's libc on an Android device is likely to
> lead to confusing results.
> 
> I'll look into setting up a bionic toolchain again.
> You don't happen to have an edition of "Crosscompiling toybox with
> bionic for dummies" at hand :)?

Send it to me if you find it. Bionic doesn't have a conventional
makefile you can use to build it outside the AOSP hairball.

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[darken]

Unfortunately the output is from devices I don't have access to, via
automatic (anonymous) bug report.

Am 13.05.2017 19:38 schrieb "Rob Landley" :

> On 05/13/2017 08:29 AM, darken wrote:
> > I stumbled upon the following "stat -c
> > "%a:%F:%d:%i:%u:%g:%s:%B:%b:%Z:%N" output from a GT-N5110@7.1.2
> >
> >> 5665:FIFO (named
> > pipe):45836:456097:1171143868:4160815103:0:512:0:18446744073
> 709551615:`/data/user/0/de.andip71.boeffla_config_v2/app_
> webview/webview_data.lock'
> >
> > The a UID of "1171143868" and GUID of "4160815103" seems way too large.
> > The device in question is rooted and that specific file with sticky &
> > suid bit seems to be a bit special anyways, but I've also seen this from
> > more vanilla files like this from a SM-N9300@6.0.1:
> >
> >> 600:regular
> > file:100d:116024:1001:4294967295:16384:4096:32:1494489310:`/
> data/user/0/com.android.providers.telephony/opname.db'
> >
> > GUID of "4294967295" seems large too.
>
> That's -1 typecast to unsigned int.
>
> Your previous UID and GID are:
>
> 1171143868: 0b010001011100111000001000
> 4160815103: 0b1000
>
> Other than the first 8 bits of the first one being a capital "E" in
> ascii this makes no sense to me.
>
> Try running it under strace to see what stat() is returning?
>
> Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[Rob Landley]

On 05/13/2017 08:29 AM, darken wrote:
> I stumbled upon the following "stat -c
> "%a:%F:%d:%i:%u:%g:%s:%B:%b:%Z:%N" output from a GT-N5110@7.1.2
> 
>> 5665:FIFO (named
> pipe):45836:456097:1171143868:4160815103:0:512:0:18446744073709551615:`/data/user/0/de.andip71.boeffla_config_v2/app_webview/webview_data.lock'
> 
> The a UID of "1171143868" and GUID of "4160815103" seems way too large.
> The device in question is rooted and that specific file with sticky &
> suid bit seems to be a bit special anyways, but I've also seen this from
> more vanilla files like this from a SM-N9300@6.0.1:
> 
>> 600:regular
> file:100d:116024:1001:4294967295:16384:4096:32:1494489310:`/data/user/0/com.android.providers.telephony/opname.db'
> 
> GUID of "4294967295" seems large too.

That's -1 typecast to unsigned int.

Your previous UID and GID are:

1171143868: 0b010001011100111000001000
4160815103: 0b1000

Other than the first 8 bits of the first one being a capital "E" in
ascii this makes no sense to me.

Try running it under strace to see what stat() is returning?

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[darken]

> are you building with bionic? glibc won't know what android does with

I'm building against musl libc. I couldn't get toybox with bionic (though
the last time i tried was a few month ago).

> glibc won't know what android does with uid/gid for apps.

Doesn't bionic/3rd party only make a difference when trying to resolve the
ids to names?

> (also, 4294967295 is -1.)

Hm, -1 is not a valid guid though right? They all have to be positive
ranging from 0 to X?

> i couldn't find any files on a Pixel XL where the built-in toybox
reported unusual uids or gids.

It's not very common, a few files, among a few devices out of thousands.
I never saw this in person either, just the occasional reports.

> as usual, not using Android's libc on an Android device is likely to lead
to confusing results.

I'll look into setting up a bionic toolchain again.
You don't happen to have an edition of "Crosscompiling toybox with bionic
for dummies" at hand :)?

2017-05-13 18:45 GMT+02:00 enh :

> are you building with bionic? glibc won't know what android does with
> uid/gid for apps. (also, 4294967295 is -1.)
>
> i couldn't find any files on a Pixel XL where the built-in toybox
> reported unusual uids or gids.
>
> as usual, not using Android's libc on an Android device is likely to
> lead to confusing results.
>
> On Sat, May 13, 2017 at 6:29 AM, darken  wrote:
> > I stumbled upon the following "stat -c "%a:%F:%d:%i:%u:%g:%s:%B:%b:%
> Z:%N"
> > output from a GT-N5110@7.1.2
> >
> >> 5665:FIFO (named
> >> pipe):45836:456097:1171143868:4160815103:0:512:0:
> 18446744073709551615:`/data/user/0/de.andip71.boeffla_
> config_v2/app_webview/webview_data.lock'
> >
> > The a UID of "1171143868" and GUID of "4160815103" seems way too large.
> The
> > device in question is rooted and that specific file with sticky & suid
> bit
> > seems to be a bit special anyways, but I've also seen this from more
> vanilla
> > files like this from a SM-N9300@6.0.1:
> >
> >> 600:regular
> >> file:100d:116024:1001:4294967295:16384:4096:32:
> 1494489310:`/data/user/0/com.android.providers.telephony/opname.db'
> >
> > GUID of "4294967295" seems large too.
> >
> > Is it normal and I should expect these kind of values or is this some
> weird
> > root related stuff one should ignore or is it maybe a bug in toybox or
> the
> > Android kernel?
> >
> > ~Matthias
> >
> > ___
> > Toybox mailing list
> > Toybox@lists.landley.net
> > http://lists.landley.net/listinfo.cgi/toybox-landley.net
> >
>
>
>
> --
> Elliott Hughes - http://who/enh - http://jessies.org/~enh/
> Android native code/tools questions? Mail me/drop by/add me as a reviewer.
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Valid range for stat userID and groupID?

[enh]

are you building with bionic? glibc won't know what android does with
uid/gid for apps. (also, 4294967295 is -1.)

i couldn't find any files on a Pixel XL where the built-in toybox
reported unusual uids or gids.

as usual, not using Android's libc on an Android device is likely to
lead to confusing results.

On Sat, May 13, 2017 at 6:29 AM, darken  wrote:
> I stumbled upon the following "stat -c "%a:%F:%d:%i:%u:%g:%s:%B:%b:%Z:%N"
> output from a GT-N5110@7.1.2
>
>> 5665:FIFO (named
>> pipe):45836:456097:1171143868:4160815103:0:512:0:18446744073709551615:`/data/user/0/de.andip71.boeffla_config_v2/app_webview/webview_data.lock'
>
> The a UID of "1171143868" and GUID of "4160815103" seems way too large. The
> device in question is rooted and that specific file with sticky & suid bit
> seems to be a bit special anyways, but I've also seen this from more vanilla
> files like this from a SM-N9300@6.0.1:
>
>> 600:regular
>> file:100d:116024:1001:4294967295:16384:4096:32:1494489310:`/data/user/0/com.android.providers.telephony/opname.db'
>
> GUID of "4294967295" seems large too.
>
> Is it normal and I should expect these kind of values or is this some weird
> root related stuff one should ignore or is it maybe a bug in toybox or the
> Android kernel?
>
> ~Matthias
>
> ___
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
>



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net