Mike Williams wrote on 10/06/2005 08:45:21 AM:
> Well, now that 0.6 is out, let's try to work out: what next?
Well, firstly... Thank you for getting 0.6 out! :-)
> There are already a few things on "the plan" for 0.7 - mostly things
> I've added because I (personally) think are important, or would provide
> a quick-win. See:
>
> http://jira.public.thoughtworks.org/browse/SEL?report=com.atlassian.
> jira.plugin.system.project:roadmap-panel
I agree with everything on the "soon" roadmap.
> But let's think longer-term, too.
>
> In no longer particular order or arrangement, here are some of the
> "bigger" things that are on my radar for the months ahead ...
>
> We desperately need some form of automated, cross-platform testing
> instructure (SEL-151) to help prevent regressions. My idea was to
> leverage the browsers of Selenium users/enthusiasts - just click a link
> to run tests and submit results back to the project. Probably a bit a
> work, but it should be interested, and will certainly be useful.
Yes, I think this would be a great way to distribute the testing load... Perhaps we could develop a screen saver for people that simply goes to selenium.thoughtworks.com and runs the self tests. The server could easily keep track the user agents used to run the tests (unless they user-agent spoof, of course). But this sounds lots more complicated than just getting a Mac and a few licenses of VMWare.
> Support for testing framed applications (SEL-70) is an oft-requested
> feature (it's the only JIRA issue with >1 vote). I think it's about
> time we had a go at it.
Yes, this is top on my list... We are upgrading our internal PeopleSoft applications (which uses frames) internally here at ThoughtWorks and we plan to use Selenium to functional test all of the HR department's core business processes. In the meantime, we are writing the test cases as Selenium tests with the hope that can run successfully at some point in the future. An example of TDD driving improvements to the testing framework and the business process at the same time.
> Speaking of JIRA Votes, I think we should use them more extensively to
> gauge how important issues are to the user-community.
Sure. +1 Although it would be nice to find a way to "advertise" the voting better.
> Having to crank up a Java Servlet container to test the web-site is kind
> of a PITA. Let's ditch it and just generate a static site. I'm
> thinking content in a Wiki-esque markup, with some simple templates and
> a script to skin things up. I have something like this already - a
> Rake+RedCloth+Erb mash-up that I use to generate another site - which
> could be easily adapted.
Yes, it may be controversial, but I'd say using Python or Ruby is preferred.
Hmmm, actually, tinyWebServer.py was added to the repository as a "light webserver"
to server and allow you to test the static files. But I'm fine with Python or Ruby for this.
Although, Ruby seams to light the fire of kids' imagination these days. :-)
> While we're on the subject, I think we need a build-step for the
> "_javascript_" core, which would allow us to (1) concatenate multiple
> version-controlled files into a smaller number of distributable
> artifacts, and (2) remove duplication e.g. between TestRunner.html and
> SeleneseRunner.html, and (2) inject version-numbers. I'd be keen to use
> Ruby/Rake there, too.
Yes, a "core" is needed that doesn't include the self tests or the jsunit tests... It's just the minimum you need to install on your server for "document literal / Fit-style" Selenium. There was a post a few months ago from some ThoughtWorkers who had to do something similar (removing everything but the "core" files needed to run Selenium). I don't know whether these core files need to be concatenated ino fewer files, though.
> I'd like to see the "driven-mode" code rationalized: let's concentrate
> on one, or perhaps two, driver-languages Is anyone using the .Net
> driver? Or the Java driver? Do we need 'em? Can we ditch 'em?
> (Sorry, I hope no-one is taking this personally). My preference (in
> case it's not blindingly obvious) would to be to concentrate on the Ruby
> driver, especially as it seems the most tolerant of changes to the
> Selenium core (new commands, etc).
I'd like to see the "standalone server" become the official way to do "RPC-style" (aka driven mode) Selenium. The key advantage of the standalone server is that you wouldn't need Selenium drivers for all other languages... Those languages would simply need an XML-RPC or SOAP library (which almost all do).
The basic idea is this:
1) Test source code is written in your favorite language and that code makes an XML-RPC or SOAP call to the standalone server.
2) The standalone server knows how to:
a) Proxy the command to the browser in 'Selenese' and retrieve results.
b) Since the standalone server is a "desktop application" running on the client next to the test scripts, we can have more control over the browser, like clicking away browser pop-up windows that can't be dispatched away just by _javascript_ (like download pop-ups or security pop-ups.) This is one advantage that Watir has over Selenium at the moment.
Yes, the Ruby driver is the most tolerant of API changes, but there's no reason the Python driver couldn't be refactored to do the same.
However, there's one thing that really piques my interest in Ruby on Rails as a candidate for the standalone server. Currently, the way the selenese drivers are implemented, they require multi-threading and "blocking queues" to handle the selenese requests and responses between the server and browser. And when the browser is blocked waiting for a command from the server, the whole browser is frozen... As an alternative mechanism, Ruby on Rails provides a nice little hook to ask the browser to "call back" to the server after some small interval-- no blocking queues are required. This is just a hunch, but frequent polling will be less buggy and less error prone than multi-threading and blocking queues.
> I really don't know much about the driven stuff, but I need to find out.
> With people asking for things like conditionals and (*gasp*) "goto" in
> the TestRunner, it's become obvious to me that we need to concentrate
> more on driven-mode, and make it easier to use.
A compromise is to allow "Document literal" Selenium to be _javascript_ source files, instead of requiring that they only be HTML tables (which today we're simply translating into _javascript_ calls anyway)... Allowing the removal of the "middleman" HTML table layer would allow folks to use _javascript_ for all their if/while/goto needs. I don't want to see "if, while, or goto" added as a core Selenium command. But I also wouldn't remove HTML-style tests as an option, either.
I've heard _javascript_ described as the "Switzerland" of programming languages. Even if we disagree over Ruby vs Python or .Net vs Java, suggesting _javascript_ as a supported "source test format" is not controversial.
> DavidK is part-way
> through a refactoring which should make it possible for drivers to
> actually retrieve useful data from the browser, rather than just lobbing
> it "assert" commands. It would be super-cool if we could support a
> Watir-compatibile API.
+1 (I'm sure the Watir folks would like this, too.)
>
> The "Selenium recorder" sounds pretty promising: I haven't had a chance
> to play with it myself, but the buzz has been good. Is there anything
> we should do to improve integration between the recorder and Selenium
> itself?
Allowing wiki table syntax or even "DSL-style" (without the table or wiki tags). Even better would be for the Record to generate raw _javascript_ test scripts.
To follow in the footsteps of the Rails, Django, and TurboGears projects of the world... It would be nice if we had a screencast that begins where Rails' famous screencast ends, and shows how to write (or record) Selenium tests for their app, and wire them up and run them in the TestRunner. Basically, we need more unabashed self-promotion and a "Web 2.0" hipster attitude. ;-)
With all the hype about AJAX, I hear very little about how folks are functional testing this stuff. Here's a post that asks the same question of AJAX:
http://www.lastcraft.com/blog/index.php?p=19
Here's the important quote:
"I haven’t yet seen an AJAX demo tested with Selenium. If anyone tries it, I’d like to know."
Besides supporting frames, next on the list is to see if Selenium can be the answer for functional testing of AJAX apps. At the moment, this is an open field, I think. And Selenium has the advantage of being specifically coded to support the testing of apps that use _javascript_.
So, in a nutshell... for 0.7 and beyond:
1) Frames support
2) _javascript_ source code allowed as a test format, along with wiki tables and HTML tables
3) AJAX support (can we test it? Let's show people how... Do we need to tweak anything internally? I imagine the "andWait logic needs to smart enough to wait for responses from asynch'd calls to the server)
4) Consoldating all the language drivers into the standalone server, which is written in Python or Ruby.
-Jason
_______________________________________________ Selenium-devel mailing list Selenium-devel@lists.public.thoughtworks.org http://lists.public.thoughtworks.org/mailman/listinfo/selenium-devel
