I think there is confusion about what -createImages does. It creates new baselines based on whatever the code and player on your computer generates and says it passes if it can successfully create the bitmap. That option should only be used to create a reference image, and nobody else running that test should use that option unless they have a good reason to change the reference images.
So changing swf.version and running with -createImages doesn't really tell you anything. If you accidentally made every text invisible, -createImages would not catch that, it would just generate new reference images without any text in them. Therefore, we need to decide what we want to use as the reference, and set up the scripts so that they have generate the same image on everyone's system without extra configuration on their part. This was easy when Flex only certified against one version of the player. I thought it would be relatively easy when we opened up to many player versions, but I wasn't aware that the player changed antialiasing in FP11. That will make things harder. On 10/27/12 1:26 PM, "Frédéric THOMAS" <webdoubl...@hotmail.com> wrote: > Well, from what I can see, the update of the build.properties like that > solve the problem: > > #10.3 > swf.version = 12 > #11.0 > #swf.version = 13 > #11.1 > #swf.version = 14 > #11.3 > #swf.version = 15 > #11.4 > #swf.version = 16 > > building the framework against a particular playerglobal and swf-version and > running the tests like that : > sh mini_run.sh -all -createImages > > It builds the Mustella test swfs with the related playerglobal and > swf-version and valid images (I can guess that from the fact that if I build > them against the 11.1 / 14, that creates identical images than the one we've > got under svn and gives different images compiling against an other > playerglobal and swf-version where only the edges of the glyphs change), so > for me that solves the problem, anyone can then builds a fresh checkouted > version of the sdk with a particular playerglobal and swf-version, generates > the images before he modifies the framework and then run the normal Mustella > tests without regenerate the images. > > Please tell me if I missed something ? > > -----Message d'origine----- > From: Frédéric THOMAS > Sent: Saturday, October 27, 2012 8:32 PM > To: flex-dev@incubator.apache.org > Subject: Re: About Flex runtime > > Alex, > > Could you please enlighten me, I'm a bit confuse now, looking at the build > of the Mustella tests, I found it uses the flex-config.xml wherein I > defined the playerglobal and swf version thru the build.properties, so, the > mustella tests are also compile with those arguments or I'm wrong ? > > -----Message d'origine----- > From: Frédéric THOMAS > Sent: Saturday, October 27, 2012 7:34 PM > To: flex-dev@incubator.apache.org > Subject: Re: About Flex runtime > > Ok, I've got it. > I thought building the framework with the good swf-version was enough :$ > > So, I'm gonna wait for your thoughts doing some experiments thru time. > > -----Message d'origine----- > From: Alex Harui > Sent: Saturday, October 27, 2012 6:58 PM > To: flex-dev@incubator.apache.org > Subject: Re: About Flex runtime > > I don't understand how your change to build.properties helps. > > Running -createImages creates new baselines, so you would expect all tests > to pass. But if you check those in, then it will fail for other > swf-versions. I think the swf-version needs to be set at the SWF level in > the .compile file. > > But before we go off and do this, I want to ponder the implications, such > as, then we'd be running fewer SWFs in the newer swf-versions, and we could > miss something important. > > > On 10/27/12 9:29 AM, "Frédéric THOMAS" <webdoubl...@hotmail.com> wrote: > >> If we can be sure that the swf-version drives how flash player draws >> fonts, >> the correction is done. >> >> Do you want me to attach a patch to the issue ? >> >> -----Message d'origine----- >> From: Frédéric THOMAS >> Sent: Saturday, October 27, 2012 6:13 PM >> To: flex-dev@incubator.apache.org >> Subject: Re: About Flex runtime >> >> I updated the build.properties like that : >> >> #10.3 >> swf.version = 12 >> #11.0 >> #swf.version = 13 >> #11.1 >> #swf.version = 14 >> #11.3 >> #swf.version = 15 >> #11.4 >> #swf.version = 16 >> >> building the framework against 10.3 and running the tests like that : sh >> mini_run.sh tests/components/Label -createImages >> >> The result was : >> >> >> [java] ===================================================== >> [java] Failed: >> [java] ===================================================== >> [java] >> [java] >> [java] ===================================================== >> [java] Passes: 255 >> [java] Fails: 0 >> [java] ===================================================== >> [java] >> [java] >> [java] Wrote summary to results.txt >> [java] Wrote failures to failures.txt >> >> call_runners: >> >> run: >> >> BUILD SUCCESSFUL >> Total time: 1 minute 24 seconds >> >> -----Message d'origine----- >> From: Frédéric THOMAS >> Sent: Saturday, October 27, 2012 5:09 PM >> To: flex-dev@incubator.apache.org >> Subject: Re: About Flex runtime >> >>> We could try changing the swf version on every test that doesn't require >>> it >> to a lower number and re-create all the baseline images. >> >> is the createImages option enough to build the baseline images? >> >>> Explictly setting swf version might be the easiest to implement, and will >> help expose where the SDK has dependencies on higher swf versions (if >> anywhere). >> >> Sounds good. >> swf-version from flex-config.xml could be replace by a variable set in >> build.properties >> >> >> -----Message d'origine----- >> From: Alex Harui >> Sent: Saturday, October 27, 2012 4:51 PM >> To: flex-dev@incubator.apache.org >> Subject: Re: About Flex runtime >> >> >> >> >> On 10/27/12 5:02 AM, "Frédéric THOMAS" <webdoubl...@hotmail.com> wrote: >> >>> Back on 10.3, I did some adjustments: setting >>> DEFAULT_NUM_COLOR_VARIANCES:int = 255 and the default value of >>> ignoreMaxColorVariance to true in the CompareBitmap class, I've been able >>> to >>> reduce the number of failed tests to 1 (only >>> text_long_property_Label_Spark.png doesn't pass and I can't get why >>> because >>> whatever the number of color variance I allowed, it always said they >>> exeeded >>> by 4). >>> >>> Well, anyway, it's not a solution, that's only to hide the failures, if >>> we >>> go anyway that way, what I'm not sure, we'll need to have a conditional >>> compilation to say we're in a lower version of the player and the tests >>> should be more tolerants. >>> >> Using ignoreMaxColorVariance is not a valid option as you say. >> >> We could see if we can set maxColorVariances to a small number like 10. >> Then the odds of missing a mistake is much lower. >> >> We could try changing the swf version on every test that doesn't require >> it >> to a lower number and re-create all the baseline images. >> >> We could try to install a new system that compares certain display list >> properties instead of the actual bitmap output. I've argued for years >> that >> we should do that since really, that's all our code can control. What the >> player does with the display list properties at render time is not in our >> control. But it is sensitive to different kinds of changes in our code, >> like changing the z-order of two things that do not overlap or have >> transparency. >> >> Explictly setting swf version might be the easiest to implement, and will >> help expose where the SDK has dependencies on higher swf versions (if >> anywhere). -- Alex Harui Flex SDK Team Adobe Systems, Inc. http://blogs.adobe.com/aharui
