se, the it is completely non-intuitive.
--
Michael Peters
Developer
Plus Three, LP
we broke this out into a formal logical proof, the only way
that x,x != y,z would would is if x != y or x != z, or both.
--
Michael Peters
Developer
Plus Three, LP
Fergal Daly wrote:
On 7/2/05, Michael Peters <[EMAIL PROTECTED]> wrote:
But if we say
x=y and x=z can we then say that x,x != y,z
If say
$x = [];
$y = [];
$z = [];
is_deeply($x, $y); # passes
is_deeply($x, $z): # passes
is_deeply([$x,$x], [$y, $z]); # fails for some reas
o use something wrapped around TAP would be to collect other
information that TAP does not provide, like platform, date and time run, time to
completely run, and if applicable developer name.
Does any of this seem reasonable? Overkill? Already done?
Thanks,
--
Michael Peters
Developer
Plus Three, LP
to have to do anything special with
their dev machines, or smoke machines, just send their reports to my app. This
makes it a lot simpler.
But I am interested in the XML schema that you have so far. Is it tied heavily
to PITA, or mainly focused around the data output from TAP tests?
--
Michael Peters
Developer
Plus Three, LP
prove, etc)?
Thanks for the links. It'll give me more to think about.
--
Michael Peters
Developer
Plus Three, LP
Mark Stosberg wrote:
> On 2005-11-22, Michael Peters <[EMAIL PROTECTED]> wrote:
>
>>
>>Stevan Little wrote:
>>
>>>Michael,
>>>
>>>You might want to look at some of the work on the Pugs test suite.
>>>
>>>http://m19s28.v
Mark Stosberg wrote:
> On Tue, Dec 13, 2005 at 02:01:18PM -0500, Michael Peters wrote:
>
>>>>>It uses (among other things) Test::TAP::Model and
>>>>>Test::TAP::HTMLMatrix, and uses YAML as an intermediate test-run format.
>>>>
>>>>
ease. I know I've caught myself several times when my tests
run just fine, I upload to CPAN and then differences in the way the CPAN shell
runs my tests cause it to fail. It would have been nice to catche these failures
before others start upgrading.
--
Michael Peters
Developer
Plus Three, LP
in comparing 2 schemas and show what differences there might, or might not be.
It's all part of the sqlfairy.sourceforge.net project.
--
Michael Peters
Developer
Plus Three, LP
ues with incompatible versions or strange Apache configs. Never
having to worry about this application screwing with some other application. And
this benefit will only increase when we have relocatable Perls in a few months
:)
--
Michael Peters
Developer
Plus Three, LP
http://sourceforge.net/project/screenshots.php?group_id=161136
And other details here:
http://sourceforge.net/projects/smolder/
While completely functional, this is an alpha release so it is not feature
complete.
--
Michael Peters
Developer
Plus Three, LP
Yuval Kogman wrote:
> On Sat, Mar 04, 2006 at 09:09:00 -0500, Michael Peters wrote:
>> It's very similar in nature to the Pugs smoke test server, but is completely
>> project agnostic. It's also completely self contained (contains local copies
>> all
>> of
se Class::Trigger could not
> be found - it doesn't seem to be included in the package, although
> lib/Class/DBI.pm uses it. SO for the moment, I'm stuck.
Thanks for catching this. I've uploaded 0.02 to sourceforge. Could you give that
a try?
--
Michael Peters
Developer
Plus Three, LP
Matisse Enzer wrote:
>
> On Mar 5, 2006, at 3:15 PM, Michael Peters wrote:
>>
>> Matisse Enzer wrote:
>>> After some trouble, I managed to create a distribution tarball for my
>>> patched Redhat 8 system from smolder-0.01-src using
>>> bin/smolder_
t::More, etc) that print to STDOUT/STDERR, right?
And there sure are a lot of them. If I had any TAP producing test
modules on CPAN I'd volunteer to change them :)
--
Michael Peters
Developer
Plus Three, LP
ee that there is a difference between them. They are
both comments output by the tests. Just because one comes from the
testing routine used by the test and the other from the test itself
doesn't mean they aren't both just human readable comments on the test run.
--
Michael Peters
Developer
Plus Three, LP
but parse TAP. Then
it could be used in all kinds of test harness permutations.
--
Michael Peters
Developer
Plus Three, LP
Shlomi Fish wrote:
> On Monday 24 April 2006 01:46, Michael Peters wrote:
>> Shlomi Fish wrote:
>>> On Sunday 23 April 2006 22:35, chromatic wrote:
>>>> On Sunday 23 April 2006 12:05, Shlomi Fish wrote:
>>>>> This debate demonstrates why a plugin
s) and
then provides some basic reports. It can even email developers when a test fails
unexpectedly.
--
Michael Peters
Developer
Plus Three, LP
- awesome
> not ok 3
>
> I believe we also have an ACTION: tag.
another useful meta info for TAP:
# LINENO: (pugs links test messages to higlighted version of the test script)
--
Michael Peters
Developer
Plus Three, LP
est::Harness::* modules.
--
Michael Peters
Developer
Plus Three, LP
ort comments, so I'm
not sure why those modules do that.
This doesn't resolve the problem of non-test modules emitting things to STDERR
that could be useful when tracking down bugs, but what they might print is not
TAP, so can't really be associated with particular tests.
--
Michael Peters
Developer
Plus Three, LP
ber) then Test::Builder will automatically emit
these attributes. This could be controlled by the environment, so that under
normal testing the developers don't see too much clutter, but the harness itself
could ask the tests to emit as much data as it wants.
--
Michael Peters
Developer
Plus Three, LP
e.org – I don’t follow
> why this is a problem. And I certainly prefer talking SQL at a
> database than writing throwaway helper script code to manipulate
> serialised data structures.
And if you use DBD::SQLite you don't even need that. I has everything you'd
need.
--
Michael Peters
Developer
Plus Three, LP
Adriano Ferreira wrote:
> It would be something different, like
>
> use Test::More tests => 'defer';
>
> and then
>
> plan past_tests => $n;
What benefit would that give? plan() is nice because it provides protection
against you test scr
Tyler MacDonald wrote:
> Michael Peters <[EMAIL PROTECTED]> wrote:
>>> use Test::More tests => 'defer';
>>>
>>> and then
>>>
>>> plan past_tests => $n;
>> What benefit would that give? plan() is nice b
rove doesn't either. Can anyone make any
> recommendations on a good approach here? My ideas are terrible things like
> setting environment variables and other things I would simply prefer to avoid.
Test::Harness doesn't need to support this. I'm pretty sure that you can just
m
ke it will show how useful abstracting out the test output from the
harness can be. I'm planning on doing a 1.0 release of it pretty soon if
anyone's interested.
--
Michael Peters
Developer
Plus Three, LP
://sourceforge.net/projects/smolder
Thanks to Mark Stosberg and Sam Tregar for the help and for Plus Three, LP for
sponsoring my work on this, as well as providing guinea pigs.
--
Michael Peters
Developer
Plus Three, LP
est($request);
ok($mech->success);
It would be nice if there was a request_ok() method in T::W::M, but it's just an
extra line without it.
--
Michael Peters
Developer
Plus Three, LP
h is a part of svn. SWISH::HiLiter needs SWISH::API which is a
part of swish-e. There are lots of perl modules tied to various projects that
don't exist independently on CPAN.
--
Michael Peters
Developer
Plus Three, LP
ourceforge.net/). Maybe some ideas or code could
be borrowed.
--
Michael Peters
Developer
Plus Three, LP
Pittsburgh Perl Workshop -
http://plusthree.com/~mpeters/smolder_presentation.pdf
Enjoy
--
Michael Peters
Developer
Plus Three, LP
cs or
>> follow a pseudo-YAML convention).
>
> I find the YAML version easier to read, in part because it's more
> compact, but mainly because the indentation allows me to easily scan for
> and find the bit I need.
++, in addition, HEREDOC is so Perl :) It's not very l
the example script from analyze.pl
to something like 'perl_metrics' and include it as an installable script. Sorta
like Test::Harness does with 'prove'.
--
Michael Peters
Developer
Plus Three, LP
Jonathan Rockway wrote:
> That said, I am willing to clean it up when I have time, and hopefully
> be able to provide a libtap that makes TAP support easy for any language.
There is already a libtap -
http://www.onlamp.com/pub/a/onlamp/2006/01/19/libtap.html
--
Michael Peters
Develope
the right thing, so
> that instead of BAILing out at 3, it bails out of the current test, sets
> a flag, and then 4 and 5 can check for the flag and skip if it is not
> set.
I think you're expecting too much from the default test harness. If your project
has special needs then write y
arser (which wasn't available
when smolder was started) and drop the XML/YAML stuff I was doing before.
--
Michael Peters
Developer
Plus Three, LP
which is a command line utility
that makes it easy to use Test::Harness and is installed when Test::Harness is
installed. You have to mess with some environment vars to make it not use perl
on your executable though. Something like this:
HARNESS_PERL="" HARNESS_PERL_SWITCHES="&q
gle file output by the TAP::Parser so I
don't have to invent my own format and provide special tools for it.
Thoughts?
--
Michael Peters
Developer
Plus Three, LP
g 5497)');
If tracking a single test over time seems like something people would want, I
think it could be added to Smolder without too much difficulty.
--
Michael Peters
Developer
Plus Three, LP
) [Has never passed]
> * Fuzz testing: image with bogus manual thumbnail [Introduced
> between 08-Apr-2007 07:15:22, 1.10alpha (r21099) and 25-Apr-2007
> 07:15:46, 1.10alpha
By the way. Do you have a link to show this or is it just something that prints
to the console when you run t
Ovid wrote:
>- GUIs (anyone want to fix the GTK GUI?)
You could mention the Smolder (shameless plug). It's sort of a gui. Not to run
the tests but to share the results of a run. And the current 1.1 uses
TAP::Parser.
--
Michael Peters
Developer
Plus Three, LP
uff.
--
Michael Peters
Developer
Plus Three, LP
opriate sql file depending on the running
DBPlatform
* Updated various images
* Added bin/smolder_update_smoke_html so that you can regenerate the HTML of
the smoke reports after a template change
--
Michael Peters
Developer
Plus Three, LP
want to be able to package up a test runs results and send it somewhere else,
it would be nice if we could get the original run time of the tests. Duration
can be calculated from that.
--
Michael Peters
Developer
Plus Three, LP
durations of individual
tests, it's now the total "real" time of the test run.
> If we're going to calculate duration, it needs to be per-test.
I agree that this is also a useful metric to have. But Smolder can only use the
version of TAP that is currently being produced a
30]
> test_times:
> foo: [13:01.01, 13:01.08]
> bar: [13:01.01, 13:02.05]
> baz: [13:01.08, 13:02.30]
>
> (pretend I typed valid datetimes and YAML)
That looks good to me. And it's something that would be a little difficult for
TAP::Harness::Archive to do without getting in the way of other subclasses, so
either TAP::Harness::Aggregator needs to provide methods to get to that data or
TAP::Harness needs to generate the meta.yml file.
--
Michael Peters
Developer
Plus Three, LP
retty HTML report and an email about the run.
As soon as SmokeStack became beta (or just public) I was going to do an article
about the whole thing end-to-end.
--
Michael Peters
Developer
Plus Three, LP
David Cantrell wrote:
> It appears that I can't make any changes without registering for yet
> another bloody account. I already have too many.
It supports OpenID. Just get an OpenID account and use it in various places.
--
Michael Peters
Developer
Plus Three, LP
Matisse Enzer wrote:
> forgive me but what is the magic variable to get archname? for example,
> on my system archname is
>darwin-thread-multi-2level
use Config;
$Config{archname}
--
Michael Peters
Developer
Plus Three, LP
the plugin API
has settled (or if it even has) and whether or not it is currently possible to
have prove use 2 different plugins.
--
Michael Peters
Developer
Plus Three, LP
th
your troubles and that you're really saving any significant amount of time?
--
Michael Peters
Developer
Plus Three, LP
to emit a new
protocol is no easier than writing a script that picks the test files/subs to
run.
--
Michael Peters
Developer
Plus Three, LP
harness it becomes:
push('xt/frob', @tests)if frob_avail();
push('xt/slow) if all_the_time_in_the_world();
Test::Harness::runtests(@tests);
--
Michael Peters
Developer
Plus Three, LP
Andy Armstrong wrote:
> On 1 Nov 2007, at 14:32, Michael Peters wrote:
>> But if they just do their own harness it becomes:
>>
>> push('xt/frob', @tests)if frob_avail();
>> push('xt/slow) if all_the_time_in_the_world();
>> Test::Harness::runtests(@t
upposed to
ignore anything they didn't understand so none of these should really be a
problem.
--
Michael Peters
Developer
Plus Three, LP
rgument.
my $aggregator = TAP::Harness::Archive->aggregator_from_archive(
{
archive => 'my_tests.tar.gz',
parser_callbacks => {
plan=> sub { warn "Nice to see you plan ahead..." },
nadim khemir wrote:
> On Sunday 16 December 2007 23.29.55 Michael G Schwern wrote:
>> nadim khemir wrote:
>>> On Saturday 15 December 2007 20.53.30 Michael Peters wrote:
>>>> The uploaded file
>>>>
>>>> TAP-Harness-Archive-0.03.tar.gz
>
se this needs to be opt-in (META.yml?).
Then it would be pretty easy to setup an email account that is monitored by some
tool that would extract the archive and upload it to a Smolder install.
Or course I'm biased about Smolder, but having the full TAP available if the
author wants it would be nice.
--
Michael Peters
Developer
Plus Three, LP
e archive
>> feature is available and that gets back to the open TAP::Builder problem.
>
> I don't understand...
Me either.
--
Michael Peters
Developer
Plus Three, LP
Eric Wilhelm wrote:
> # from Michael Peters
> # on Thursday 20 December 2007 15:07:
>
>> What would be ideal (and it's something that RJBS has
>> poked me about before) would be to receive a TAP Archive (prove
>> --archive) that could get attached to the email. O
best way since that's the supported interface right
now for sending reports to smolder in an automated way.
--
Michael Peters
Developer
Plus Three, LP
Smylers wrote:
> Names normally descend in specificity from left to right, but
> Test::More::Even doesn't really make sense.
How about Test::More::Still, Test::More::Stuff, Test::More::Crap,
Test::More::More :)
--
Michael Peters
Developer
Plus Three, LP
as the failure happens so that I can start
working on it:
make test || echo -e "\a"
Would be nice if that would beep after the first failure instead of after 45
minutes and the whole thing is done.
--
Michael Peters
Developer
Plus Three, LP
Michael G Schwern wrote:
> Michael Peters wrote:
>>make test || echo -e "\a"
>
> I keep digging away at this because I'm looking for a problem other than "I
> want to see the first failure". And that's what I'm hearing from you and
I'd add a #3 which is Buildbot + Smolder. I'm going to start working on that
this weekend. I've got some enhancements and bug fixes to smolder that should
make that much easier and I just need to get off my duff and do it.
--
Michael Peters
Plus Three, LP
cts/smolder
BTW, as usual for open source stuff, those screen shots are out of date.
--
Michael Peters
Plus Three, LP
more than just my $work.
--
Michael Peters
Plus Three, LP
David Golden wrote:
> I'm arriving 9:35 AM (Continental from Newark)
That's actually the same flight that Ricardo and I are on.
--
Michael Peters
Plus Three, LP
but can be overridden by each
individual test if needed. For instance, the test is actually in some class
which you're using.
> Or, the filename *could* change between tests in a stream? Does it?
> Should it?
It definitely could.
--
Michael Peters
Plus Three, LP
Philippe Bruhat (BooK) wrote:
> Kobesearch's sources are available.
search.cpan.org uses swish-e which is very configurable. We (meaning $work) do
category/tagging searching with swish-e in lots of other places.
--
Michael Peters
Plus Three, LP
The default DB-Platform is now SQLite instead of MySQL
See the changle log for full details.
http://sourceforge.net/projects/smolder
--
Michael Peters
Plus Three, LP
f their
labels with 'HTTP-' or email headers doing a 'Email-' prefix either.
--
Michael Peters
Plus Three, LP
nd email alerts), so I do a test
to not do them if $ENV{HARNESS_ACTIVE}. But I do want to do them live. But since
I use TAP::Harness::Archive (which uses TAP::Harness) to get the results out of
an existing TAP archive Smolder constantly thinks it's being tested.
Thoughts?
--
Michael Peters
Plus Three, LP
st::Harness 3 tools.
> - Or would I write/subclass a complete own harness for my purposes?
For my larger projects that I care about I write a custom test harness that uses
the new Test::Harness 3 tools.
--
Michael Peters
Plus Three, LP
made_parser_callback.
+ Added extra_files argument to new()
+ Added extra_properties argument to new()
+ Fixed RT# 35649
--
Michael Peters
Plus Three, LP
didn't know ...
Smolder is a Smoke Test Aggregator used by developers and testers to upload
(automated or manually) and view smoke/regression tests using the Test
Anything Protocol. Details and trends are graphed and notifications provided
via email or Atom feeds.
--
Michael P
ose things for TAP anyway.
--
Michael Peters
Plus Three, LP
or that this module contains
them (because there aren't kwalitee points to say otherwise :) and another user on my machine does,
then ugly brown stuff can hit spinning blades designed to circulate air.
--
Michael Peters
Plus Three, LP
. I'm just saying
that since we know about this path, let's eliminate it, or at least make it public and known.
This is a CPANTS issue.
I agree.
--
Michael Peters
Plus Three, LP
L for something like this), etc.
--
Michael Peters
Plus Three, LP
Eric Wilhelm wrote:
1. TAP::Harness should store that when archiving?
It would be nice if TAP::Harness::Archive noted the exit code in the meta.yml file included in the
archive.
--
Michael Peters
Plus Three, LP
TAP stream and also a YAML
file containing the meta data for the run. I'm not entirely sure what a DWIM
would be in the case of a non-archive file. Just concat the TAP streams together
and drop the YAML? Something else?
--
Michael Peters
Plus Three, LP
Andy Armstrong wrote:
How about being able to nominate a directory to receive the TAP?
That sounds reasonable. But should anything be done for a plain file?
--
Michael Peters
Plus Three, LP
to the start of the discussion.
Not even considering the delimiter we already have non-TAP since multiple
streams will mean multiple plans, restarting test numbers, etc which will still
make it non-TAP.
--
Michael Peters
Plus Three, LP
st
use a real archive? I can add the feature to just dump the archived files into a
directory and not tar.gz it if that makes it easier. I know it's not a pipe
interface, but it's not radically different. Just use "&&" instead of "|"
prove -a some_dir && process_tap_archive some_dir
--
Michael Peters
Plus Three, LP
While I won't say it's perfect, and it wouldn't bother me if someone else had a
stricter system, but it hasn't caused us any problems so far.
--
Michael Peters
Plus Three, LP
Michael Peters wrote:
So, if you're already cutting it into individual TAP streams why don't
you just use a real archive? I can add the feature to just dump the
archived files into a directory and not tar.gz it if that makes it
easier. I know it's not a pipe interface, but it&
I just released a new module to CPAN to take care of a pretty common use case
when using Module::Build: run my tests but give me a TAP Archive.
perl Build.PL && ./Build test_archive
perl Build.PL && ./Build test_archive --archive_file mytests.tar.gz
--
Michael Peters
Plus Three, LP
stall here:
http://smolder.plusthree.com/app/developer_projects/smoke_reports/8
--
Michael Peters
Plus Three, LP
ng up to Smolder but have some related project to do the actual
code updates (integrating with various SCMs), smartly work with branches, etc.
My tuits are completely gone at this point, but it would be an interesting project.
--
Michael Peters
Plus Three, LP
y they are
never installed.
--
Michael Peters
Plus Three, LP
xpath_extract_ok($mech->content, //t...@id='thing']);
like($html, qr/stuff/);
I can see the benefit of having these as methods on the mech object itself since
you wouldn't have to keep calling $mech->content explicitly. So maybe a
WWW::Mechanize::XPath subclass?
--
Michael Peters
Plus Three, LP
it will be
supported going forward.
--
Michael Peters
Plus Three, LP
f the things that I think will make
Smolder really great. Nice tabled diagnostics, maybe a good visual diff viewer, etc.
--
Michael Peters
Plus Three, LP
t put back.
"local" should be good for doing that right?
--
Michael Peters
Plus Three, LP
chive overrides runtests() to add the archive creation after the
parent's runtests() have finished. What does TAP::Harness do when it encounters
a bailout? Is there some exception thrown that T::H::A should catch?
--
Michael Peters
Plus Three, LP
ed some APIs where it's not immediately
clear what's a function call and what's an argument to that function: is
reason() an argument to want() or have()?. It also seems more obvious for doing
things like data-driven tests where you just have a large data structure that
tests are run against.
--
Michael Peters
Plus Three, LP
1 - 100 of 113 matches
Mail list logo
