Hi (moved to main list), On Tue, 2006-07-25 at 17:56 -0400, Tom Tromey wrote: > If you had a build of 0.91 + the corresponding jamvm you could make a > Mauve baseline and run a comparison quite easily. Ideally we would do > this after each release and simply reset the baseline file on > builder. Then we could see regression reports versus the previous > release automatically.
I cleaned up the mauve Harness a bit more to make sure that all result lines were really only the PASS/FAIL and testname and fixed an issue so we can handle the early return of Object.wait(timeout) on cacao and I now have result files for 0.91 and current CVS. When builder picks up the changes I can install them there and then hopefully builder starts sending regressions since the 0.91 baseline. The results aren't that. bad (all generated against cacao on fc5 x86-64) Lots of new PASSes. One new fail: New fails: FAIL: javax.swing.JComponent.getInputMap And the following regressions: Regressions: FAIL: java.awt.Canvas.PaintTest FAIL: java.awt.Component.clickModifiers FAIL: java.awt.Component.keyPressTest FAIL: java.awt.FileDialog.TestGraphics FAIL: java.awt.Robot.keyPress FAIL: java.awt.Robot.keyRelease FAIL: java.awt.Robot.mouseMove FAIL: java.awt.Robot.mousePress FAIL: java.awt.Robot.mouseRelease FAIL: java.awt.Robot.mouseWheel FAIL: java.awt.Scrollbar.ScrollbarPaintTest FAIL: java.awt.TextArea.ScrollbarPaintTest FAIL: java.awt.TextComponent.ignoreOldMouseEvents FAIL: java.awt.image.ComponentSampleModel.getDataElements FAIL: java.lang.InheritableThreadLocal.simple FAIL: java.math.BigDecimal.DiagBigDecimal FAIL: java.math.BigDecimal.construct FAIL: java.math.BigDecimal.setScale FAIL: javax.swing.JEditorPane.getScrollableTracks FAIL: javax.swing.JMenu.getInputMap FAIL: javax.swing.RepaintManager.addDirtyRegion FAIL: javax.swing.text.AbstractDocument.BranchElement.getStartOffset FAIL: javax.swing.text.AbstractDocument.LeafElement.getStartOffset FAIL: javax.swing.text.AbstractDocument.filterTest Some of the awt tests might be spurious failures. I have not investigated any of them yet. But any help going through these would be appreaciated. Full report attached. Cheers, mark
Baseline from: Wed Jul 26 15:28:12 CEST 2006 Regressions: FAIL: java.awt.Canvas.PaintTest FAIL: java.awt.Component.clickModifiers FAIL: java.awt.Component.keyPressTest FAIL: java.awt.FileDialog.TestGraphics FAIL: java.awt.Robot.keyPress FAIL: java.awt.Robot.keyRelease FAIL: java.awt.Robot.mouseMove FAIL: java.awt.Robot.mousePress FAIL: java.awt.Robot.mouseRelease FAIL: java.awt.Robot.mouseWheel FAIL: java.awt.Scrollbar.ScrollbarPaintTest FAIL: java.awt.TextArea.ScrollbarPaintTest FAIL: java.awt.TextComponent.ignoreOldMouseEvents FAIL: java.awt.image.ComponentSampleModel.getDataElements FAIL: java.lang.InheritableThreadLocal.simple FAIL: java.math.BigDecimal.DiagBigDecimal FAIL: java.math.BigDecimal.construct FAIL: java.math.BigDecimal.setScale FAIL: javax.swing.JEditorPane.getScrollableTracks FAIL: javax.swing.JMenu.getInputMap FAIL: javax.swing.RepaintManager.addDirtyRegion FAIL: javax.swing.text.AbstractDocument.BranchElement.getStartOffset FAIL: javax.swing.text.AbstractDocument.LeafElement.getStartOffset FAIL: javax.swing.text.AbstractDocument.filterTest Improvements: PASS: gnu.java.security.util.TestOfIntegerUtil PASS: gnu.javax.crypto.cipher.TestOfTripleDES PASS: gnu.javax.crypto.jce.TestOfCipherEngineInit PASS: gnu.javax.crypto.jce.TestOfCipherWrapUnwrap PASS: gnu.javax.crypto.jce.TestOfDHKeyAgreement2 PASS: gnu.javax.crypto.jce.TestOfPR27849 PASS: gnu.javax.crypto.jce.TestOfPR27853 PASS: gnu.javax.crypto.jce.TestOfTripleDESParityAdjustment PASS: gnu.javax.crypto.jce.keyring.TestOfKeystore PASS: gnu.javax.crypto.keyring.TestOfGnuPrivateKeyring PASS: gnu.javax.crypto.kwa.TestOfAESKeyWrap PASS: gnu.javax.crypto.kwa.TestOfTripleDESKeyWrap PASS: gnu.javax.crypto.pad.TestOfISO10126 PASS: java.awt.Component.invalidate PASS: java.awt.Component.isValid PASS: java.awt.Component.properties PASS: java.awt.Component.setComponentOrientation PASS: java.awt.Component.setFont PASS: java.awt.Component.setMaximumSize PASS: java.awt.Component.setMinimumSize PASS: java.awt.Component.setPreferredSize PASS: java.awt.Container.applyComponentOrientation PASS: java.awt.Container.setLayout PASS: java.awt.Graphics2D.security PASS: java.awt.Label.PaintTest PASS: java.awt.Point.setLocation PASS: java.awt.Robot.getPixelColor PASS: java.awt.TextArea.constructors PASS: java.awt.TextArea.testAppendText PASS: java.awt.TextArea.testInsertText PASS: java.awt.TextArea.testInvalidConstructorValues PASS: java.awt.TextArea.testReplaceText PASS: java.awt.TextField.constructors PASS: java.awt.Toolkit.security PASS: java.awt.font.ImageGraphicAttribute.ImageGraphicAttributeTest PASS: java.awt.font.ShapeGraphicAttribute.ShapeGraphicAttributeTest PASS: java.awt.image.BandedSampleModel.createCompatibleSampleModel PASS: java.awt.image.BandedSampleModel.createDataBuffer PASS: java.awt.image.BufferedImage.constructors PASS: java.awt.image.ByteLookupTable.constructors PASS: java.awt.image.ByteLookupTable.getTable PASS: java.awt.image.ComponentSampleModel.getPixel PASS: java.awt.image.ComponentSampleModel.getSample PASS: java.awt.image.ComponentSampleModel.getSampleDouble PASS: java.awt.image.ComponentSampleModel.getSampleFloat PASS: java.awt.image.ConvolveOp.filter PASS: java.awt.image.MultiPixelPackedSampleModel.constructors PASS: java.awt.image.MultiPixelPackedSampleModel.createCompatibleSampleModel PASS: java.awt.image.MultiPixelPackedSampleModel.createDataBuffer PASS: java.awt.image.MultiPixelPackedSampleModel.createSubsetSampleModel PASS: java.awt.image.MultiPixelPackedSampleModel.equals PASS: java.awt.image.MultiPixelPackedSampleModel.getDataElements PASS: java.awt.image.MultiPixelPackedSampleModel.getOffset PASS: java.awt.image.MultiPixelPackedSampleModel.getPixel PASS: java.awt.image.MultiPixelPackedSampleModel.getSample PASS: java.awt.image.MultiPixelPackedSampleModel.getSampleSize PASS: java.awt.image.MultiPixelPackedSampleModel.getScanlineStride PASS: java.awt.image.MultiPixelPackedSampleModel.getTransferType PASS: java.awt.image.MultiPixelPackedSampleModel.hashCode PASS: java.awt.image.MultiPixelPackedSampleModel.setDataElements PASS: java.awt.image.MultiPixelPackedSampleModel.setPixel PASS: java.awt.image.MultiPixelPackedSampleModel.setSample PASS: java.awt.image.PixelGrabber.testNullProducer PASS: java.awt.image.SampleModel.getPixel PASS: java.awt.image.SampleModel.getPixels PASS: java.awt.image.ShortLookupTable.constructors PASS: java.awt.image.ShortLookupTable.getTable PASS: java.awt.image.SinglePixelPackedSampleModel.createSubsetSampleModel PASS: java.awt.image.SinglePixelPackedSampleModel.equals PASS: java.awt.image.SinglePixelPackedSampleModel.getDataElements PASS: java.awt.image.SinglePixelPackedSampleModel.getSampleSize PASS: java.awt.image.SinglePixelPackedSampleModel.hashCode PASS: java.awt.testName PASS: java.beans.VetoableChangeSupport.addVetoableChangeListener PASS: java.beans.beancontext.Add PASS: java.beans.beancontext.InstantiateChild PASS: java.beans.beancontext.Remove PASS: java.lang.ClassLoader.loadClass PASS: java.lang.Thread.insecurity PASS: java.lang.Thread.security PASS: java.lang.ThreadGroup.insecurity PASS: java.lang.management.ClassLoadingMXBeanTest PASS: java.lang.management.OperatingSystemMXBeanTest PASS: java.lang.management.RuntimeMXBeanTest PASS: java.math.BigInteger.modPow PASS: java.net.URLStreamHandler.Except PASS: java.nio.channels.FileChannel.multidirectbufferIO PASS: java.rmi.server.Uniqueness PASS: java.util.Calendar.add PASS: java.util.Collections.unmodifiableMap PASS: java.util.logging.LoggingMXBean.Test PASS: javax.crypto.spec.TestOfPBEKeySpec PASS: javax.crypto.spec.TestOfSecretKeySpec PASS: javax.imageio.stream.MemoryCacheImageInputStream.TestImageInputStreamImpl PASS: javax.swing.AbstractButton.setHorizontalAlignment PASS: javax.swing.AbstractButton.setHorizontalTextPosition PASS: javax.swing.AbstractButton.setRolloverIcon PASS: javax.swing.AbstractButton.setRolloverSelectedIcon PASS: javax.swing.AbstractButton.setVerticalAlignment PASS: javax.swing.AbstractButton.setVerticalTextPosition PASS: javax.swing.ButtonGroup.add PASS: javax.swing.ButtonGroup.remove PASS: javax.swing.DefaultBoundedRangeModel.serialization PASS: javax.swing.DefaultButtonModel.setArmed PASS: javax.swing.DefaultButtonModel.setEnabled PASS: javax.swing.DefaultButtonModel.setGroup PASS: javax.swing.DefaultButtonModel.setPressed PASS: javax.swing.DefaultButtonModel.setSelected PASS: javax.swing.DefaultComboBoxModel.removeElement PASS: javax.swing.DefaultListSelectionModel.setAnchorSelectionIndex PASS: javax.swing.InputMap.put PASS: javax.swing.JCheckBoxMenuItem.constructors PASS: javax.swing.JComboBox.ComboRobot PASS: javax.swing.JComboBox.basic PASS: javax.swing.JComponent.getActionForKeyStroke PASS: javax.swing.JComponent.getComponentPopupMenu PASS: javax.swing.JComponent.getRegisteredKeyStrokes PASS: javax.swing.JComponent.getVerifyInputWhenFocusTarget PASS: javax.swing.JComponent.registerKeyboardAction PASS: javax.swing.JComponent.setComponentPopupMenu PASS: javax.swing.JComponent.setDefaultLocale PASS: javax.swing.JComponent.setInheritsPopupMenu PASS: javax.swing.JComponent.setVerifyInputWhenFocusTarget PASS: javax.swing.JFileChooser.getFileSelectionMode PASS: javax.swing.JFileChooser.setFileSelectionMode PASS: javax.swing.JFrame.constructors PASS: javax.swing.JInternalFrame.AccessibleJInternalFrame.getCurrentAccessibleValue PASS: javax.swing.JInternalFrame.AccessibleJInternalFrame.setCurrentAccessibleValue PASS: javax.swing.JInternalFrame.getDesktopIcon PASS: javax.swing.JInternalFrame.getNormalBounds PASS: javax.swing.JInternalFrame.setDesktopIcon PASS: javax.swing.JInternalFrame.setNormalBounds PASS: javax.swing.JInternalFrame.setSelected2 PASS: javax.swing.JInternalFrame.setTitle PASS: javax.swing.JLabel.AccessibleJLabel.getAccessibleName PASS: javax.swing.JLabel.Mnemonic PASS: javax.swing.JLabel.getAccessibleContext PASS: javax.swing.JLabel.paramString PASS: javax.swing.JLabel.setDisplayedMnemonic PASS: javax.swing.JLabel.setDisplayedMnemonicIndex PASS: javax.swing.JLabel.setLabelFor PASS: javax.swing.JList.getNextMatch PASS: javax.swing.JList.setLayoutOrientation PASS: javax.swing.JList.setValueIsAdjusting PASS: javax.swing.JList.setVisibleRowCount PASS: javax.swing.JMenu.constructors PASS: javax.swing.JMenu.getPopUpMenu PASS: javax.swing.JMenu.remove PASS: javax.swing.JMenuBar.getActionMap PASS: javax.swing.JMenuBar.getInputMap PASS: javax.swing.JMenuBar.getSubElements PASS: javax.swing.JRadioButtonMenuItem.isFocusable PASS: javax.swing.JScrollBar.getActionMap PASS: javax.swing.JScrollBar.getInputMap PASS: javax.swing.JScrollBar.paramString PASS: javax.swing.JScrollPane.getActionMap PASS: javax.swing.JScrollPane.getInputMap PASS: javax.swing.JSplitPane.getActionMap PASS: javax.swing.JSplitPane.getInputMap PASS: javax.swing.JTable.AccessibleJTable.AccessibleJTableCell.getAccessibleRole PASS: javax.swing.JTable.AccessibleJTable.AccessibleJTableHeaderCell.getAccessibleRole PASS: javax.swing.JTable.AccessibleJTable.getAccessibleColumnHeader PASS: javax.swing.JTable.getAccessibleContext PASS: javax.swing.JTable.setColumnSelectionAllowed PASS: javax.swing.JTable.setRowSelectionAllowed PASS: javax.swing.JToolBar.buttonInsets PASS: javax.swing.JToolBar.getInputMap PASS: javax.swing.JTree.setModel PASS: javax.swing.Timer.setDelay PASS: javax.swing.Timer.setInitialDelay PASS: javax.swing.ToolTipManager.setDismissDelay PASS: javax.swing.ToolTipManager.setInitialDelay PASS: javax.swing.ToolTipManager.setReshowDelay PASS: javax.swing.event.ListDataEvent.constructor PASS: javax.swing.event.ListDataEvent.getIndex0 PASS: javax.swing.event.ListDataEvent.getIndex1 PASS: javax.swing.event.ListDataEvent.toString PASS: javax.swing.plaf.IconUIResource.constructor PASS: javax.swing.plaf.basic.BasicArrowButton.getMaximumSize PASS: javax.swing.plaf.basic.BasicArrowButton.getMinimumSize PASS: javax.swing.plaf.basic.BasicArrowButton.getPreferredSize PASS: javax.swing.plaf.basic.BasicArrowButton.isFocusable PASS: javax.swing.plaf.basic.BasicScrollBarUI.constructor PASS: javax.swing.plaf.basic.BasicScrollBarUI.getMinimumSize PASS: javax.swing.plaf.basic.BasicScrollBarUI.getPreferredSize PASS: javax.swing.plaf.basic.BasicScrollBarUI.layoutContainer PASS: javax.swing.plaf.metal.MetalIconFactory.getCheckBoxMenuItemIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getFileChooserDetailViewIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getFileChooserHomeFolderIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getFileChooserListViewIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getFileChooserNewFolderIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getFileChooserUpFolderIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getHorizontalSliderThumbIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getInternalFrameAltMaximizeIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getInternalFrameCloseIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getInternalFrameDefaultMenuIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getInternalFrameMaximizeIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getInternalFrameMinimizeIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getRadioButtonMenuItemIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getTreeComputerIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getTreeFloppyDriveIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getTreeHardDriveIcon PASS: javax.swing.plaf.metal.MetalIconFactory.getVerticalSliderThumbIcon PASS: javax.swing.plaf.metal.MetalScrollButton.isFocusable PASS: javax.swing.table.DefaultTableColumnModel.constructor PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleChild PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleChildrenCount PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleComponent PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleContext PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleDescription PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleIndexInParent PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleName PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleRole PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getAccessibleValue PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getLocale PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.isFocusTraversable PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.isVisible PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.setAccessibleName PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.getAccessibleChild PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.getAccessibleChildrenCount PASS: javax.swing.table.JTableHeader.AccessibleJTableHeader.getAccessibleRole PASS: javax.swing.text.AbstractDocument.getDocumentProperties PASS: javax.swing.text.PlainDocument.getDocumentProperties PASS: javax.swing.text.PlainDocument.insertUpdate PASS: javax.swing.text.PlainDocument.remove PASS: javax.swing.text.TabSet.constructor PASS: javax.swing.text.TabSet.equals PASS: javax.swing.text.TabSet.getTab PASS: javax.swing.text.TabSet.getTabCount PASS: javax.swing.text.TabSet.getTabIndexAfter PASS: javax.swing.text.TabSet.toString PASS: javax.swing.text.TabStop.toString PASS: javax.swing.text.html.HTMLDocument.FindById PASS: javax.swing.tree.DefaultTreeCellRenderer.getBackgroundNonSelectionColor PASS: javax.xml.parsers.DocumentBuilder.PR27864 New fails: FAIL: javax.swing.JComponent.getInputMap Totals: PASS: 2655 XPASS: 0 FAIL: 220 XFAIL: 0
