On 02 Feb 2014, at 14:31, [email protected] wrote:
> For diligence and curiosity leading up to the Pharo3 release, I downloaded
> build image 30733 (with PharoLauncher)
Getting an image green is a lot of little, trivial, hard work… and green always
means “Green in this context”. Just headless vs. non-headless
changes a lot, for example.
Another wonderful thing is that tests are fine when run individually, but fail
when running all tests.
We need to think about automatic UI testing…
Loading an external package changes *everything*. We do not have a a release
criterium that all tests are green after loading external packages.
> and immediately ran all tests, which reported 7 failures and 1 error (see
> attached snapshot). Now presumably these did not occur in the CI build
> since otherwise it would not have been available to PharoLauncher, so this
> raises a few questions/ideas:
>
> 1. Why the discrepancy / how did these slip through ? Perhaps some difference
> in local environment / difference in VM? Can anyone else reproduce this for
> 30733? I've logged an new case [1] for this where I plan to either link to
> existing cases or add a subcase for each failure. Please add your results
> there, particularly if you see a known issue.
>
> Here are my failures...
> BlockClosureTest>>#testTrace
Not failing for me
> JobTest>>#testCurrent
Not failing for me
> MetacelloRepositorySqueakCommonTestCase>>#testAsRepositorySpecFor
> MetacelloRepositorySqueakCommonTestCase>>#testDirectoryRepository
> MetacelloRepositorySqueakCommonTestCase>>#testFileTreeRepository
Not failing fo rme
> ReleaseTest>>#testObsoleteClasses
Not failing for me
> SimmulateMouseSpecification>>#testSimulateRightClick
yes, that is failing
> and errors...
> OpenToolTest>>#testOpenBrowseOnInstalledTraitMethod
>
yes.
The last two are examples of UI tests that are not run on the Build server in a
way that they could fail.
> 3. Assuming Pharo3 will go through a Release Candidate phase,
Normally the idea is that we will just release. In the past we used an
elaborate process, but the reality is that nobody looks at release candidates.
You can tell 5 times that the release candidate will be released unchanged and
people should check: they will not. They will download
the release, though, and then complain that the “obvious” thing X is not fixed
that that the people who managed the release process did
everything wrong. Because “release” means bug free. By magic.
Marcus
