2007/3/27, Alexei Zakharov <[EMAIL PROTECTED]>:
> Do you think it is a permanent failure? :)
No :)
But my intention was at least to wait until CC sends failure alerts
about these tests. My initial question was about a formal criteria to
be used when placing a test to intermittent exclude list. The current
criteria is periodical CC failure alerts, right?
I don't remember any agreement on a formal criteria.
I think that we exclude [using common sense] those tests
that may produce false alarms of CCs.
I think that the threshold (whether it fails often enough to exclude it)
should depend on the number of configurations running regularly:
if we have just on CC project that runs once a week then it's OK to
have it intermittently failing for example every 10 runs. Currently we
have dozens of CC projects and it would be good to have a false alarm
not often than once per week. Basing on the desired false alarm
frequency, number of CC projects, number of intermittently failing tests
we can derive "target" failure frequency that is enough for inclusion
to the "intermittent" exclude list :)
So...
Once per million runs is probably too rare for exclusion, but once
per 50 times is probably enough
Thanks,
Mikhail
Thanks,
2007/3/27, Mikhail Loenko <[EMAIL PROTECTED]>:
> 2007/3/23, Alexei Zakharov <[EMAIL PROTECTED]>:
> > Hi all,
> >
> > In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> > some tests fail very rarely. Say, one failure per 50 tries. So the
> > question to all is: should this be treated as an intermittent failure?
> > Thoughts?
>
> Do you think it is a permanent failure? :)
>
> if it's still a failure, we should not run the tests under CCs to avoid
> false alarms
>
> Thanks,
> Mikhail
>
>
> >
> > Thanks,
> >
> > 2007/3/22, tatyana doubtsova (JIRA) <[EMAIL PROTECTED]>:
> > >
> > > [
https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130
]
> > >
> > > tatyana doubtsova commented on HARMONY-1979:
> > > --------------------------------------------
> > >
> > > Alexei, the behavior of awt module tests is unstable on linux ia32, ia
64. New runs show new failures.
> > >
> > > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> > >
> > > 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of
50
> > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but
was:<java.awt.Point[x=461,y=334]> at
java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at
java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > >
> > > 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
> > >
> > > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out
of 50
> > > org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2
out of 50
> > > java.lang.OutOfMemoryError <no stack trace available>
> > >
> > > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> > >
> > > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> > >
> > > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> > >
> > > 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> > > 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> > > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of
30
> > > 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> > > 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but
was:<java.awt.Point[x=461,y=334]> at
java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at
java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString
failes 1 time out of 30
> > > junit.framework.AssertionFailedError at
java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152)
at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > >
> > >
> > >
> > > > [classlib][awt][test] Many excluded AWT tests pass
> > > > --------------------------------------------------
> > > >
> > > > Key: HARMONY-1979
> > > > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > > Project: Harmony
> > > > Issue Type: Bug
> > > > Components: build - test - ci
> > > > Reporter: Alexey A. Ivanov
> > > > Assigned To: Alexei Zakharov
> > > > Attachments: H1979-awt-unexclude-tests.patch
> > > >
> > > >
> > > > Many excluded AWT tests pass successfully without any modification. I
tried both Windows and Linux.
> > >
> > > --
> > > This message is automatically generated by JIRA.
> > > -
> > > You can reply to this email to add a comment to the issue online.
--
Alexei Zakharov,
Intel ESSD
