Re: CVS commit: src/tests/syscall

2011-07-03 Thread Jukka Ruohonen
On Sun, Jul 03, 2011 at 11:45:24PM +, David Holland wrote:
> On Sun, Jul 03, 2011 at 08:22:51PM +, Jukka Ruohonen wrote:
>  > Modified Files:
>  >src/tests/syscall: t_mknod.c
>  > 
>  > Log Message:
>  > Remove wrong check.
>  > 
>  > XXX: Why does this fail on tmpfs but not ffs?
> 
> That *should* fail. Probably ffs is failing to check.

Can you file a PR?

- Jukka.


Re: CVS commit: src/tests/syscall

2011-07-03 Thread David Holland
On Sun, Jul 03, 2011 at 08:22:51PM +, Jukka Ruohonen wrote:
 > Modified Files:
 >  src/tests/syscall: t_mknod.c
 > 
 > Log Message:
 > Remove wrong check.
 > 
 > XXX: Why does this fail on tmpfs but not ffs?

That *should* fail. Probably ffs is failing to check.

-- 
David A. Holland
dholl...@netbsd.org


Re: CVS commit: src/tests/syscall

2011-05-29 Thread Matthias Scheler
On Sun, May 29, 2011 at 02:57:14PM +0100, Julio M. Merino Vidal wrote:
> On 5/29/11 1:57 PM, Matthias Scheler wrote:
> >Modified Files:
> > src/tests/syscall: t_pollts.c
> >
> >Log Message:
> >[...]
> >Also use ATF_REQUIRE_EQ_MSG() instead of ATF_REQUIRE_MSG() to avoid
> >crashes if one of the required conditions isn't met.
> 
> It could be a bug in the macro.  I tried changing the last
> ATF_REQUIRE_EQ_MSG call in the program to ATF_REQUIRE_MSG but it did
> not fail.  And I tried adding an explicitly-failing ATF_REQUIRE_MSG
> and did not see a crash either...

I don't think it's a bug in the macro. The problem was that it use the
0 value as the format string which caused a crash. I'm not sure
why I haven't seen this before.

Kind regards

-- 
Matthias Scheler  http://zhadum.org.uk/


Re: CVS commit: src/tests/syscall

2011-05-29 Thread Julio Merino

On 5/29/11 1:57 PM, Matthias Scheler wrote:

Modified Files:
src/tests/syscall: t_pollts.c

Log Message:
[...]
Also use ATF_REQUIRE_EQ_MSG() instead of ATF_REQUIRE_MSG() to avoid
crashes if one of the required conditions isn't met.


It could be a bug in the macro.  I tried changing the last 
ATF_REQUIRE_EQ_MSG call in the program to ATF_REQUIRE_MSG but it did not 
fail.  And I tried adding an explicitly-failing ATF_REQUIRE_MSG and did 
not see a crash either...


Could you provide some more details and/or a stack trace please?

--
Julio Merino / @jmmv


Re: CVS commit: src/tests/syscall

2011-05-28 Thread Christos Zoulas
In article <20110528161614.ga71...@colwyn.zhadum.org.uk>,
Matthias Scheler   wrote:
>On Sat, May 28, 2011 at 11:34:49AM -0400, Christos Zoulas wrote:
>> Module Name: src
>> Committed By:christos
>> Date:Sat May 28 15:34:49 UTC 2011
>> 
>> Modified Files:
>>  src/tests/syscall: t_pselect.c
>> 
>> Log Message:
>> add a timeout test
>
>Your tests don't pass on my setup:
>
>t_pselect (17/21): 2 test cases
>pselect_signal_mask_with_signal: Failed: pselect() did not receive signal
>pselect_signal_mask_with_timeout: Failed: pselect() did not receive signal

I am puzzled too. I will take a look.

christos



Re: CVS commit: src/tests/syscall

2011-05-28 Thread Matthias Scheler
On Sat, May 28, 2011 at 11:34:49AM -0400, Christos Zoulas wrote:
> Module Name:  src
> Committed By: christos
> Date: Sat May 28 15:34:49 UTC 2011
> 
> Modified Files:
>   src/tests/syscall: t_pselect.c
> 
> Log Message:
> add a timeout test

Your tests don't pass on my setup:

t_pselect (17/21): 2 test cases
pselect_signal_mask_with_signal: Failed: pselect() did not receive signal
pselect_signal_mask_with_timeout: Failed: pselect() did not receive signal

Kind regards

-- 
Matthias Scheler  http://zhadum.org.uk/


Re: CVS commit: src/tests/syscall

2011-05-22 Thread Julio Merino

On 5/22/11 12:48 AM, Christos Zoulas wrote:

On May 22, 12:28am, j...@julipedia.org (Julio Merino) wrote:
-- Subject: Re: CVS commit: src/tests/syscall
| That's what was done.  But in order to enforce the timeouts et. al. this
| library had to fork() on every test case.  This was really, really ugly
| (and slow) in shell.  It was also hard to keep the 3 implementations (c,
| c++, shell) in sync.

You should not have to fork if you don't set a timeout. Or you can fork
just one killer and have it timeout in 2x the last registered timeout and exit.
It listens to a socket for pid's and timeouts; if the socket is still
useable after the timeout it kills.


One of the original design goals of atf was to explicitly run every test 
case in a subprocess.  The idea was to prevent broken test cases from 
corrupting the results of other tests.  So yes, you have to fork (if 
only because it's in the requirements of the design) ;-)


I had actually thought about the alternative you mention not too long 
ago, but if you do that, then a misbehaving test case will result in the 
termination of the whole test program (you need to kill a particular 
process after all).  Maybe it's not a big deal since that's what other 
very-popular frameworks do, but it's a complete shift in how atf 
currently works.


Anyway.  We are getting side-tracked here; I was just trying to see if 
we had a bug in atf.  If you want to discuss this further, the topic 
needs to be moved to atf-devel.


re: CVS commit: src/tests/syscall

2011-05-21 Thread matthew green


this isn't directly relevant to the discussion at hand, but more
about timeouts in general.


one problem that timeouts can have is with a broken clock, the
timeout will never fire.

i don't think that atf can handle this itself, it needs to be in
the testcase.  there was one src/regress test case that i had
patched to have a hard limit on iterations to avoid this, but that
change got lost.

of course, this case should be considered a failure, but to me the
important part was to avoid the testsuite itself hanging forever.


.mrg.


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Christos Zoulas
On May 22, 12:28am, j...@julipedia.org (Julio Merino) wrote:
-- Subject: Re: CVS commit: src/tests/syscall

| atf-test-case(4) contains a description of these.  (Yeah, it could be 
| improved.)

Good.

| > I don't see why. You just put it all in a shared library written in c and
| > used from c++. Shell you have already.
| 
| That's what was done.  But in order to enforce the timeouts et. al. this 
| library had to fork() on every test case.  This was really, really ugly 
| (and slow) in shell.  It was also hard to keep the 3 implementations (c, 
| c++, shell) in sync.

You should not have to fork if you don't set a timeout. Or you can fork
just one killer and have it timeout in 2x the last registered timeout and exit.
It listens to a socket for pid's and timeouts; if the socket is still
useable after the timeout it kills.

| I'm not saying it's not possible, just that it was ugly _in the past_. 
| We can revisit this later.  The renewed runtime I'm working on (atf v2, 
| aka kyua) will support different test program styles, so we should be 
| able to easily re-experiment with this approach.

Yes, the implementation needs to be done carefully.

christos


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Julio Merino

On 5/22/11 12:18 AM, Christos Zoulas wrote:

On May 21, 10:31pm, j...@julipedia.org (Julio Merino) wrote:
-- Subject: Re: CVS commit: src/tests/syscall

| Sounds like a good idea.  Note that there is *many* more things that
| will not work.  The test case will dump garbage in the current directory
| and it won't be cleaned, the environment won't be sanitized, child
| processes won't be killed...

It could mention all of them.


atf-test-case(4) contains a description of these.  (Yeah, it could be 
improved.)



| Tests used to do exactly that more than a year ago, but it turned out to
| be a bad idea...
|
| The test programs were extremely complex: they had to implement all the
| test case isolation (work directory, timeouts, etc.) themselves, and
| that meant having to implement the same thing in C, C++ and shell.

I don't see why. You just put it all in a shared library written in c and
used from c++. Shell you have already.


That's what was done.  But in order to enforce the timeouts et. al. this 
library had to fork() on every test case.  This was really, really ugly 
(and slow) in shell.  It was also hard to keep the 3 implementations (c, 
c++, shell) in sync.


I'm not saying it's not possible, just that it was ugly _in the past_. 
We can revisit this later.  The renewed runtime I'm working on (atf v2, 
aka kyua) will support different test program styles, so we should be 
able to easily re-experiment with this approach.


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Christos Zoulas
On May 21, 10:31pm, j...@julipedia.org (Julio Merino) wrote:
-- Subject: Re: CVS commit: src/tests/syscall

| Sounds like a good idea.  Note that there is *many* more things that
| will not work.  The test case will dump garbage in the current directory
| and it won't be cleaned, the environment won't be sanitized, child
| processes won't be killed...

It could mention all of them.

| 
| > Also it would be nice if programs
| > when invoked with -l or something they would list all their tests, and if
| 
| That already works :-)  The output format is gly though

Yes, perhaps use a different flag to just list the names.

| > invoked with -a they would run all the tests.
| 
| Tests used to do exactly that more than a year ago, but it turned out to
| be a bad idea...
| 
| The test programs were extremely complex: they had to implement all the
| test case isolation (work directory, timeouts, etc.) themselves, and
| that meant having to implement the same thing in C, C++ and shell.

I don't see why. You just put it all in a shared library written in c and
used from c++. Shell you have already.

| Also, and this is what bugged most people: debugging the test cases was
| almost impossible due to all the black magic that was involved in them.
|  The test cases would delete temporary files on failure and gdb was
| unusable.

You can have a flag to disable the setup/cleanup.

| As a result, I moved all the isolation logic into atf-run and turned the
| test programs into a 'dumb' frontend for the test cases.  Test programs
| now don't do much more than just running test case bodies and exposing a
| consistent interface, so they really should not be used directly unless
| you need to debug a particular test case.  Oh, and the whole system
| works much, much faster ;-P

Well, I've never ran atf-run :-) and I've ran individual tests multiple times.

christos



Re: CVS commit: src/tests/syscall

2011-05-21 Thread Julio Merino
On 05/21/11 21:58, Christos Zoulas wrote:
> On May 21,  9:26pm, j...@julipedia.org (Julio Merino) wrote:
> -- Subject: Re: CVS commit: src/tests/syscall
> 
> | Just to make sure we are talking about the same: you were running the
> | test program with atf-run; correct?  If yes, what platform was this in?
> |  (I'm testing with amd64.)  Otherwise, this is expected behavior: tests
> | are not supposed to be executed without atf-run.
> 
> No, I am running it directly. Programs could warn if they figured out
> that they are not running under atf-run. Print something like: not running
> under atf-run, timeouts will not work.

Sounds like a good idea.  Note that there is *many* more things that
will not work.  The test case will dump garbage in the current directory
and it won't be cleaned, the environment won't be sanitized, child
processes won't be killed...

> Also it would be nice if programs
> when invoked with -l or something they would list all their tests, and if

That already works :-)  The output format is gly though

> invoked with -a they would run all the tests.

Tests used to do exactly that more than a year ago, but it turned out to
be a bad idea...

The test programs were extremely complex: they had to implement all the
test case isolation (work directory, timeouts, etc.) themselves, and
that meant having to implement the same thing in C, C++ and shell.

Also, and this is what bugged most people: debugging the test cases was
almost impossible due to all the black magic that was involved in them.
 The test cases would delete temporary files on failure and gdb was
unusable.

As a result, I moved all the isolation logic into atf-run and turned the
test programs into a 'dumb' frontend for the test cases.  Test programs
now don't do much more than just running test case bodies and exposing a
consistent interface, so they really should not be used directly unless
you need to debug a particular test case.  Oh, and the whole system
works much, much faster ;-P


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Christos Zoulas
On May 21,  9:26pm, j...@julipedia.org (Julio Merino) wrote:
-- Subject: Re: CVS commit: src/tests/syscall

| On 05/21/11 15:44, Christos Zoulas wrote:
| > On May 21,  3:34pm, j...@netbsd.org (Julio Merino) wrote:
| > -- Subject: Re: CVS commit: src/tests/syscall
| > 
| > | On 5/21/11 3:19 PM, Christos Zoulas wrote:
| > | > Revert the latest revision in the test and boot a kernel before the 
pselect
| > | > changes, and see it getting stuck.
| > | 
| > | Any particular revision?  I have a machine that hasn't been updated for 
| > | at least two weeks; will that be enough?
| > 
| > Oh sure. revision 1.1
| 
| Alright, so I just tried this.  Downgraded the test to 1.1 and ensured
| that my kernel does not have the fix (sys_select.c:1.30 and sys_sig.c:1.32).
| 
| And the timeout of 2 seconds works just fine.
| 
| Just to make sure we are talking about the same: you were running the
| test program with atf-run; correct?  If yes, what platform was this in?
|  (I'm testing with amd64.)  Otherwise, this is expected behavior: tests
| are not supposed to be executed without atf-run.

No, I am running it directly. Programs could warn if they figured out
that they are not running under atf-run. Print something like: not running
under atf-run, timeouts will not work. Also it would be nice if programs
when invoked with -l or something they would list all their tests, and if
invoked with -a they would run all the tests.

christos


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Julio Merino
On 05/21/11 15:44, Christos Zoulas wrote:
> On May 21,  3:34pm, j...@netbsd.org (Julio Merino) wrote:
> -- Subject: Re: CVS commit: src/tests/syscall
> 
> | On 5/21/11 3:19 PM, Christos Zoulas wrote:
> | > Revert the latest revision in the test and boot a kernel before the 
> pselect
> | > changes, and see it getting stuck.
> | 
> | Any particular revision?  I have a machine that hasn't been updated for 
> | at least two weeks; will that be enough?
> 
> Oh sure. revision 1.1

Alright, so I just tried this.  Downgraded the test to 1.1 and ensured
that my kernel does not have the fix (sys_select.c:1.30 and sys_sig.c:1.32).

And the timeout of 2 seconds works just fine.

Just to make sure we are talking about the same: you were running the
test program with atf-run; correct?  If yes, what platform was this in?
 (I'm testing with amd64.)  Otherwise, this is expected behavior: tests
are not supposed to be executed without atf-run.


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Julio Merino

On 5/21/11 5:25 PM, David Laight wrote:

On Sat, May 21, 2011 at 05:14:07PM +0200, Manuel Bouyer wrote:

On Sat, May 21, 2011 at 03:34:22PM +0100, Julio Merino wrote:

Yes, the timeout thing is broken.  It should really be a
specification of the test case size (e.g. 'small', 'large') and
allow the user to define his timeout preferences for every class,
because they will vary from machine to machine.


Sure. I ran ATF on a machine with a 1Mhz CPU clock (this is a
simulator, the real hardware is expected to run faster :).
Lots of tests timed out just because on such hardware they can't
complete in less than 5mn.


Perhaps the timeout(s) should be settable units that default
to 1 second.
Them the timeouts can be scaled for very slow (or fast) systems.


Perhaps.  But timeouts in a measure of time do not make sense in this 
context.  There won't be any way to scale them in a reliable manner, so 
this will lead to flaky tests.  And anyway, the user will be required to 
have control over how long these deadlines are.


Again, there is no point in setting test timeouts to 1 second (except 
for very very rare cases).  Timeouts are only used to kill tests that 
get stuck; nothing should rely on them.  If there is many timeouts in 
your test suite (which will result in failed tests), something is broken 
and needs fixing.


--
Julio Merino / @jmmv


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Christos Zoulas
On May 21,  5:25pm, da...@l8s.co.uk (David Laight) wrote:
-- Subject: Re: CVS commit: src/tests/syscall

| On Sat, May 21, 2011 at 05:14:07PM +0200, Manuel Bouyer wrote:
| > On Sat, May 21, 2011 at 03:34:22PM +0100, Julio Merino wrote:
| > > Yes, the timeout thing is broken.  It should really be a
| > > specification of the test case size (e.g. 'small', 'large') and
| > > allow the user to define his timeout preferences for every class,
| > > because they will vary from machine to machine.
| > 
| > Sure. I ran ATF on a machine with a 1Mhz CPU clock (this is a
| > simulator, the real hardware is expected to run faster :).
| > Lots of tests timed out just because on such hardware they can't
| > complete in less than 5mn.
| 
| Perhaps the timeout(s) should be settable units that default
| to 1 second.
| Them the timeouts can be scaled for very slow (or fast) systems.

That should be easy to do since they are already strings.
1sec, 1msec, 1usec, 1nsec

christos


Re: CVS commit: src/tests/syscall

2011-05-21 Thread David Laight
On Sat, May 21, 2011 at 05:14:07PM +0200, Manuel Bouyer wrote:
> On Sat, May 21, 2011 at 03:34:22PM +0100, Julio Merino wrote:
> > Yes, the timeout thing is broken.  It should really be a
> > specification of the test case size (e.g. 'small', 'large') and
> > allow the user to define his timeout preferences for every class,
> > because they will vary from machine to machine.
> 
> Sure. I ran ATF on a machine with a 1Mhz CPU clock (this is a
> simulator, the real hardware is expected to run faster :).
> Lots of tests timed out just because on such hardware they can't
> complete in less than 5mn.

Perhaps the timeout(s) should be settable units that default
to 1 second.
Them the timeouts can be scaled for very slow (or fast) systems.

David

-- 
David Laight: da...@l8s.co.uk


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Manuel Bouyer
On Sat, May 21, 2011 at 03:34:22PM +0100, Julio Merino wrote:
> Yes, the timeout thing is broken.  It should really be a
> specification of the test case size (e.g. 'small', 'large') and
> allow the user to define his timeout preferences for every class,
> because they will vary from machine to machine.

Sure. I ran ATF on a machine with a 1Mhz CPU clock (this is a
simulator, the real hardware is expected to run faster :).
Lots of tests timed out just because on such hardware they can't
complete in less than 5mn.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Christos Zoulas
On May 21,  3:34pm, j...@netbsd.org (Julio Merino) wrote:
-- Subject: Re: CVS commit: src/tests/syscall

| On 5/21/11 3:19 PM, Christos Zoulas wrote:
| > In article<20110521083602.ga24...@netbsd.org>,
| > Julio Merino  wrote:
| >>> 1. It did not work. What units is the timeout anyway? I waited 2 seconds
| >>> and 2 minutes and it did not fire.
| >>
| >> It's in seconds.  The default is 300.
| >
| > I think the unit is too long for modern hardware. Imaging having 300 tests
| > needing 1 second timeouts.
| 
| Yes, the timeout thing is broken.  It should really be a specification 
| of the test case size (e.g. 'small', 'large') and allow the user to 
| define his timeout preferences for every class, because they will vary 
| from machine to machine.
| 
| That said, I don't see why you'd want to have 300 tests needing 1-second 
| timeouts.  If a test times out, it's broken, so the only thing I can 
| deduce is that you'd have 300 broken tests ;-)
| 
| >>> 3. No matter how the timeout is done (unless you start a watcher process 
and
| >>> kill -KILL the test process) it can break (masking signal mask, 
changing
| >>> timers, catching signal).
| >>
| >> The timeout is enforced from atf-run, not from inside the test case; it is
| >> already using a helper process in a sense, so it should work.  If it 
doesn't,
| >> it is a bug that I'd like to debug.
| >
| > Revert the latest revision in the test and boot a kernel before the pselect
| > changes, and see it getting stuck.
| 
| Any particular revision?  I have a machine that hasn't been updated for 
| at least two weeks; will that be enough?

Oh sure. revision 1.1

christos


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Julio Merino

On 5/21/11 3:19 PM, Christos Zoulas wrote:

In article<20110521083602.ga24...@netbsd.org>,
Julio Merino  wrote:

1. It did not work. What units is the timeout anyway? I waited 2 seconds
and 2 minutes and it did not fire.


It's in seconds.  The default is 300.


I think the unit is too long for modern hardware. Imaging having 300 tests
needing 1 second timeouts.


Yes, the timeout thing is broken.  It should really be a specification 
of the test case size (e.g. 'small', 'large') and allow the user to 
define his timeout preferences for every class, because they will vary 
from machine to machine.


That said, I don't see why you'd want to have 300 tests needing 1-second 
timeouts.  If a test times out, it's broken, so the only thing I can 
deduce is that you'd have 300 broken tests ;-)



3. No matter how the timeout is done (unless you start a watcher process and
kill -KILL the test process) it can break (masking signal mask, changing
timers, catching signal).


The timeout is enforced from atf-run, not from inside the test case; it is
already using a helper process in a sense, so it should work.  If it doesn't,
it is a bug that I'd like to debug.


Revert the latest revision in the test and boot a kernel before the pselect
changes, and see it getting stuck.


Any particular revision?  I have a machine that hasn't been updated for 
at least two weeks; will that be enough?


Thanks.

--
Julio Merino / @jmmv


Re: CVS commit: src/tests/syscall

2011-05-21 Thread Christos Zoulas
In article <20110521083602.ga24...@netbsd.org>,
Julio Merino   wrote:
>On Wed, May 18, 2011 at 12:30:24PM +, Christos Zoulas wrote:
>> In article <4dd39547.9010...@julipedia.org>,
>> Julio Merino   wrote:
>> >On 5/18/11 4:15 AM, Christos Zoulas wrote:
>> >> Module Name:  src
>> >> Committed By: christos
>> >> Date: Wed May 18 03:15:12 UTC 2011
>> >>
>> >> Modified Files:
>> >>   src/tests/syscall: t_pselect.c
>> >>
>> >> Log Message:
>> >> Don't depend on the atf timeout stuff, do it ourselves.
>> >
>> >Why?  Could atf do something better here?
>> 
>> 1. It did not work. What units is the timeout anyway? I waited 2 seconds
>>and 2 minutes and it did not fire.
>
>It's in seconds.  The default is 300.

I think the unit is too long for modern hardware. Imaging having 300 tests
needing 1 second timeouts.

>
>You can override the timeout by setting the 'timeout' metadata property
>to a different value.
>
>> 2. I need to do some cleanup if there is a timeout (kill the stuck process).
>>How do I register a cleanup function?
>
>Use ATF_TC_WITH_CLEANUP and then provide an ATF_TC_CLEANUP function for
>your test case.  Be aware that cleanup runs in a different process than
>the body, so the only way to transfer state is through a file in the
>current directory.
>
>> 3. No matter how the timeout is done (unless you start a watcher process and
>>kill -KILL the test process) it can break (masking signal mask, changing
>>timers, catching signal).
>
>The timeout is enforced from atf-run, not from inside the test case; it is
>already using a helper process in a sense, so it should work.  If it doesn't,
>it is a bug that I'd like to debug.

Revert the latest revision in the test and boot a kernel before the pselect
changes, and see it getting stuck.

christos



Re: CVS commit: src/tests/syscall

2011-05-21 Thread Julio Merino
On Wed, May 18, 2011 at 12:30:24PM +, Christos Zoulas wrote:
> In article <4dd39547.9010...@julipedia.org>,
> Julio Merino   wrote:
> >On 5/18/11 4:15 AM, Christos Zoulas wrote:
> >> Module Name:   src
> >> Committed By:  christos
> >> Date:  Wed May 18 03:15:12 UTC 2011
> >>
> >> Modified Files:
> >>src/tests/syscall: t_pselect.c
> >>
> >> Log Message:
> >> Don't depend on the atf timeout stuff, do it ourselves.
> >
> >Why?  Could atf do something better here?
> 
> 1. It did not work. What units is the timeout anyway? I waited 2 seconds
>and 2 minutes and it did not fire.

It's in seconds.  The default is 300.

You can override the timeout by setting the 'timeout' metadata property
to a different value.

> 2. I need to do some cleanup if there is a timeout (kill the stuck process).
>How do I register a cleanup function?

Use ATF_TC_WITH_CLEANUP and then provide an ATF_TC_CLEANUP function for
your test case.  Be aware that cleanup runs in a different process than
the body, so the only way to transfer state is through a file in the
current directory.

> 3. No matter how the timeout is done (unless you start a watcher process and
>kill -KILL the test process) it can break (masking signal mask, changing
>timers, catching signal).

The timeout is enforced from atf-run, not from inside the test case; it is
already using a helper process in a sense, so it should work.  If it doesn't,
it is a bug that I'd like to debug.

-- 
Julio Merino


Re: CVS commit: src/tests/syscall

2011-05-18 Thread Christos Zoulas
In article <4dd39547.9010...@julipedia.org>,
Julio Merino   wrote:
>On 5/18/11 4:15 AM, Christos Zoulas wrote:
>> Module Name: src
>> Committed By:christos
>> Date:Wed May 18 03:15:12 UTC 2011
>>
>> Modified Files:
>>  src/tests/syscall: t_pselect.c
>>
>> Log Message:
>> Don't depend on the atf timeout stuff, do it ourselves.
>
>Why?  Could atf do something better here?

1. It did not work. What units is the timeout anyway? I waited 2 seconds
   and 2 minutes and it did not fire.
2. I need to do some cleanup if there is a timeout (kill the stuck process).
   How do I register a cleanup function?
3. No matter how the timeout is done (unless you start a watcher process and
   kill -KILL the test process) it can break (masking signal mask, changing
   timers, catching signal).

christos



Re: CVS commit: src/tests/syscall

2011-05-18 Thread Julio Merino

On 5/18/11 4:15 AM, Christos Zoulas wrote:

Module Name:src
Committed By:   christos
Date:   Wed May 18 03:15:12 UTC 2011

Modified Files:
src/tests/syscall: t_pselect.c

Log Message:
Don't depend on the atf timeout stuff, do it ourselves.


Why?  Could atf do something better here?


Re: CVS commit: src/tests/syscall

2011-04-04 Thread Jukka Ruohonen
On Mon, Apr 04, 2011 at 09:05:19AM +0200, Nicolas Joly wrote:
> Adding the following to the testcase header should do the trick:
> 
>atf_tc_set_md_var(tc,"require.user","unprivileged");
> 
> In that case, if run as root, it will lower its privileges to the
> unprivileged user _atf before running the test.

I wonder should there be also

atf_tc_set_md_var(tc, "require.config", "rump");

so that those architectures where rump(3) does not work (?) also benefit
from the regression tests without many false positives. Cf. jun@'s test
runs in [1].

- Jukka.

[1] http://ftp.netbsd.org/pub/NetBSD/misc/jun/atf/


Re: CVS commit: src/tests/syscall

2011-04-04 Thread Nicolas Joly
On Mon, Apr 04, 2011 at 10:18:52AM +0300, Jukka Ruohonen wrote:
> On Mon, Apr 04, 2011 at 09:05:19AM +0200, Nicolas Joly wrote:
> > Adding the following to the testcase header should do the trick:
> > 
> >atf_tc_set_md_var(tc,"require.user","unprivileged");
> > 
> > In that case, if run as root, it will lower its privileges to the
> > unprivileged user _atf before running the test.
> 
> Thanks, that indeed does the job.
> 
> If I may express my humble opinion, there seems to be currently way too much
> weird macro-magic and undocumented calls in atf(7)...

Meta-data properties, like this one, are already described in
atf-test-case(4) man page.

-- 
Nicolas Joly

Projects and Developments in Bioinformatics
Institut Pasteur, Paris.


Re: CVS commit: src/tests/syscall

2011-04-04 Thread Jukka Ruohonen
On Mon, Apr 04, 2011 at 09:05:19AM +0200, Nicolas Joly wrote:
> Adding the following to the testcase header should do the trick:
> 
>atf_tc_set_md_var(tc,"require.user","unprivileged");
> 
> In that case, if run as root, it will lower its privileges to the
> unprivileged user _atf before running the test.

Thanks, that indeed does the job.

If I may express my humble opinion, there seems to be currently way too much
weird macro-magic and undocumented calls in atf(7)...

- Jukka.


Re: CVS commit: src/tests/syscall

2011-04-04 Thread Nicolas Joly
On Mon, Apr 04, 2011 at 08:32:14AM +0300, Jukka Ruohonen wrote:
> On Mon, Apr 04, 2011 at 01:59:50AM +, Christos Zoulas wrote:
> > >Check for getuid(2) == 0.
> > 
> > Why not setuid() to fix it?
> 
> That would indeed be the right thing to do. However, I am not quite sure how
> atf(7) actually works. It seems to create some temporary files such that
> simply changing the real and effective UID in the middle of a test gives
> 
>tc-se:FATAL ERROR: Cannot create results file '/tmp/atf-run.21878a/tcr':
>Permission denied
> 
> I am not sure whether this particular test deserves a more complex solution.

Adding the following to the testcase header should do the trick:

   atf_tc_set_md_var(tc,"require.user","unprivileged");

In that case, if run as root, it will lower its privileges to the
unprivileged user _atf before running the test.

-- 
Nicolas Joly

Projects and Developments in Bioinformatics
Institut Pasteur, Paris.


Re: CVS commit: src/tests/syscall

2011-04-03 Thread Jukka Ruohonen
On Mon, Apr 04, 2011 at 01:59:50AM +, Christos Zoulas wrote:
> >Check for getuid(2) == 0.
> 
> Why not setuid() to fix it?

That would indeed be the right thing to do. However, I am not quite sure how
atf(7) actually works. It seems to create some temporary files such that
simply changing the real and effective UID in the middle of a test gives

   tc-se:FATAL ERROR: Cannot create results file '/tmp/atf-run.21878a/tcr':
   Permission denied

I am not sure whether this particular test deserves a more complex solution.

- Jukka.


Re: CVS commit: src/tests/syscall

2011-04-03 Thread Christos Zoulas
In article <20110404014946.4b9e917...@cvs.netbsd.org>,
Jukka Ruohonen  wrote:
>-=-=-=-=-=-
>
>Module Name:   src
>Committed By:  jruoho
>Date:  Mon Apr  4 01:49:45 UTC 2011
>
>Modified Files:
>   src/tests/syscall: t_access.c
>
>Log Message:
>Check for getuid(2) == 0.

Why not setuid() to fix it?

christos