On 29 March 2012 06:27, Paul Sokolovsky <[email protected]> wrote: > Hello Zach, > > On Wed, 28 Mar 2012 12:05:46 -0500 > Zach Pfeffer <[email protected]> wrote: > >> On 27 March 2012 06:32, Paul Sokolovsky <[email protected]> >> wrote: >> > Hello, >> > >> > Yong Qin is working on the blueprint >> > https://blueprints.launchpad.net/lava-android-test/+spec/modify-android-build >> > to add arbitrary custom client-side scripts to Android Build. He >> > submitted first implementation of it as >> > https://code.launchpad.net/~liuyq0307/linaro-android-build-tools/run-custom/+merge/98825 >> > and documented as >> > https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration . >> > >> > Unfortunately, I'm not thrilled with that implementation, more >> > specifically, its "user interface" (i.e. any parts which user >> > directly faces) by the following reasons: >> > >> > 1. The idea behind Android Build's build config was that they're >> > short and easy for human to parse, essentially one glance-over >> > would enough to get a good idea what is built here, even for >> > outsider. Consequently, the configs should not be overloaded with >> > details not related to building. If there's a need for integration >> > with other systems, we have good pattern of externalizing such >> > details and then just referring to them with a single variable in a >> > build config. >> > >> > 2. The whole approach in >> > https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration >> > seems like trying to encode hierarchical structure in the shell >> > syntax, which is not much supporting of that. The end result looks >> > pretty much like representation of graph structure in raw assembler >> > - it's spaghetti mix of data pieces and labels, requiring long time >> > to wrap hand around to understand it, and cumbersome and >> > error-prone to write. >> > >> > >> > So, I would like to propose alternative syntax solving the issues >> > above. I probably should start with saying that if the talk is about >> > LAVA, then using native LAVA JSON request immediately comes to mind. >> > Well, I guess human-writability wasn't a design goal for that, so I >> > skip it. It still makes sense to stick to general-purpose >> > hierarchical structure syntax though. Except that JSON has 2 >> > problems: a) it doesn't support comments natively, so we'll need to >> > pre-process it; b) error reporting/localization may be still no >> > ideal. >> > >> > Anyway, here's my try, it is presented as a comment to >> > https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration >> > and then full example at >> > https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration/pfalcon >> > >> > >> > Let's discuss if that covers our needs and constraints. >> >> Paul, thanks for the alternative proposal. >> >> The request was: >> >> Provide a easy way to pass a test into LAVA from an android-build >> configuration. >> >> Yong Qin's approach seems reasonable because its of the form: >> >> MY_DEFINE="runmycommand --paramone 1 --paramtwo 2" > > Well, not quite, example at > https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration > shows 20 cross-referencing lines for defining tests (with build > configuration per se taking 5).
Right, so this is simpler. >> >> I don't like Paul's proposal because if you need something that >> complicated you should just use the LAVA python interface. I just want >> something simple. >> >> Something like: >> >> LAVA_TEST_1="runmycommand --paramone 1 --paramtwo 2" >> LAVA_TEST_PARSE_1="some regex" >> >> LAVA_TEST_2="runmyothercommand --paramone 1" >> LAVA_TEST_PARSE_2="some other regex" >> >> ..and that's it. > > Well, if it's about "parsing", it's never that simple. Yong Qin's > example shows what typical parsing regexp looks like. Sure. >> LAVA would then run LAVA_TEST_1 and LAVA_TEST_2 >> etc... and use LAVA_TEST_PARSE_1 and LAVA_TEST_PARSE_2 to parse the >> results nad send them back to the android-build. >> >> All the tests we run on Android will be baked into the build, so we >> don't have to worry about installing them. >> >> I don't even want to have to do this: >> >> LAVA_TESTS="LAVA_TEST_1 LAVA_TEST_2" >> >> ...because that's adding too much structure. > > But if you have too many things without structure, mess ensues. In > particular, it's unclear when these tests run in respect to > LAVA_TEST_PLAN. At a higher level all your going to have is a flat list: LAVA_TEST_1="runmycommand --paramone 1 --paramtwo 2" LAVA_TEST_PARSE_1="some regex" LAVA_TEST_2="runmyothercommand --paramone 1" LAVA_TEST_PARSE_2="some other regex" LAVA_TEST_3="runmycommand --paramone 1 --paramtwo 2" LAVA_TEST_PARSE_3="some regex" LAVA_TEST_4="runmyothercommand --paramone 1" LAVA_TEST_PARSE_4="some other regex" LAVA_TEST_5="runmycommand --paramone 1 --paramtwo 2" LAVA_TEST_PARSE_5="some regex" LAVA_TEST_6="runmyothercommand --paramone 1" LAVA_TEST_PARSE_6="some other regex" etc... ...that's pretty simple structure wise. What I'm advocating for is just this simple solution, which is simpler even than Yong Qin's. >> >> This is really just a way to get a test in without needing to add a >> bunch of python code to LAVA. The flat list of commands is easy to >> see, and there's no tree structure to contend with. If you need more >> structure you'd write some python or write the structure into a >> script. > > Well, all in all, what you described is quite different from what is > presented at > https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration > (and implemented in 100 lines (50% of existing code size) at > https://code.launchpad.net/~liuyq0307/linaro-android-build-tools/run-custom/+merge/98825) > , > so probably lack of proper spec to start with was the issue here. These are just sketches. The spec is: Provide a simple way to pass a test to LAVA from Android build and we have a couple of examples now: 1. Youg Qins 2. Yours 3. Mine I presented mine because its extremely simple and if you want anything more complicated you'll put it in the test repo. > If we reduce the requirements to just supporting textual (no external > URL references) content in +1 > LAVA_TEST_${N} > LAVA_TEST_PARSE_${N} > > and well defined order to run them (for example, after LAVA_TEST_PLAN), > then it certainly will be good enough to put into build configs(though > parsing regexps would still look hairy there imho). Cool. Lets do it. > > -- > Best Regards, > Paul > > Linaro.org | Open source software for ARM SoCs > Follow Linaro: http://www.facebook.com/pages/Linaro > http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog -- Zach Pfeffer Android Platform Team Lead, Linaro Platform Teams Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog _______________________________________________ linaro-validation mailing list [email protected] http://lists.linaro.org/mailman/listinfo/linaro-validation
