----- Original Message -----
From: "Julian Leviston" <[email protected]>
To: "Fundamentals of New Computing" <[email protected]>
Sent: Wednesday, October 13, 2010 9:13 PM
Subject: Re: [fonc] Spec-Driven & Self-Testing Code
On 11/10/2010, at 12:24 PM, BGB wrote:
<--
Does anyone know about a language (possibly something in smalltalk) that
involves spec-driven development?
A language that stipulates a spec (in code - ie a test) must be written
before any code can be written, and that can also do self-checks (sort of
like a checksum) before the code is executed?
Julian.
-->
possibly relevant questions:
what would be the merit of imposing such a restriction?
how would one enforce such a restriction?
how could this be done without compromising the ability to use the
language to get much useful done?
...
IMO, any such restrictions are better left under the control of "the
development process" rather than trying to enforce them at the language
level.
<--
The merit of imposing that users write specs first assumes that programmers
have a plan before they start writing code, and then encourages them to
become aware of their plan and requirements, explicitly laying them out in a
series of test of behaviour. (See http://behaviour-driven.org/ possibly for
a good explanation of where I'm coming from here).
-->
quick skim of some of it...
ok, there seems to be a conflict here...
writing specs/plans, and testing quick/early, are very different approaches
to software design.
in a typical spec or plan-based process, a person will try to work out
everything in advance, and delay coding until after the spec is relatively
complete. this is what is usually termed "waterfall method" or "BDUF".
waterfall: first spec; then code; then test and debug; then deploy.
in a test-driven setup, usually the tests and code are written early and
written alongside each other, and planning is usually very minimal (usually
just writing out maybe basic ideas for what the thing should do).
in this later form, usually specs are written *after* the code, usually
either to document the code, or to help identify which directions they want
to go, ...
this would seem to be more the style advocated by the posted link.
it seems like there is jumping between advocating several different
methodologies.
admittedly, I tend to combine both strategies, usually writing specs for
individual components or features, but using an incremental process and
regular testing for large scale features.
<--
I'm not advocating enforcement, rather encouragement. As most of us know,
you cannot happily enforce things on people, let alone programmers - for
example Java's nightmarish type system hinders more than helps (in my humble
personal opinion, only). One *can* encourage certain things through the FORM
of the language/IDE/workspace/etc., though (for example, smalltalk's class
browser encourages a programmer to think in terms of classes)...
unfortunately composition might be a better practice most of the time and it
*may* be that having a class browser as a default tool encouraged the
"over-use" of inheritance that we have seen as quite a dominant
practice/idea in the OOP world.
-->
enforcement also usually only works when there is something well defined to
enforce, or something can be enforced in terms of taking away some specific
feature.
<--
What I *am* advocating, though, is a checksum that is written by testing
suites at various levels that would mean that code wouldn't run unless it
had at least the most basic testing suite run against it. I'm advocating
that behaviour-tests should be a mandatory part of development (any
development process - be it writing a song, programming a program or
building a house). The best of us as developers in any realm do these kinds
of requirement-conformance "tests" when building whatever it is we're
building ANYWAY, so I see it as simply a formalisation of a hitherto
unexpressed best-practice.
-->
IMO, "checksum" is also a bad choice of a terms to use in reference to unit
tests or similar.
typically, what a checksum does and what a unit test do are very different,
and it would create unneeded confusion to use one term to describe the
other.
the problem is (with the proposed idea), until both code and tests can be
written, one is almost inevitably going to write a test which always passes,
so that they can run the code, effectively so that they can in turn test and
debug the code.
traditionally, code will be run regardless of whether it is known to work,
and the main point of unit testing then is to start throwing up red-flags
that something may need to be looked at or fixed, or to track new
functionality which "should" work, but doesn't as of yet (if wanting to add
a new feature may make the test fail, which may break the build, then people
are going to delay adding the test until after the feature is generally
known to work, partly defeating some of the purpose of unit testing).
<--
Code should simply be able to test itself, to perform minimal
self-diagnostics in the same way that advanced photocopiers or computers can
do this. If we bake this in at a micro-granular level, then the reliability
and stability of systems will skyrocket. Not only this, but code essentially
self-documents then. If we are as programmers, encouraged to document before
*and* after we write code, we will end up with a neat description of what we
were trying to do and not just the output.
-->
usually I just write tests as little special-purpose frontends which test
that a piece of code works as expected.
I have tried using these for my C compiler as well, but sadly I am then left
to realize that my compiler is partly broken (some tests fail for "reasons
not yet identified", seriously, in some cases I comb through the ASM output
and still can't find the cause of the error), and I have neither the time
nor the energy to really beat all the bugs out of it (my main codegen was
written with, sadly, very little testing, and is very internally complex, so
sadly it is left on faith that a lot of this stuff actually works...).
sadly, some pieces of code, particular compilers, are somewhat difficult to
test on a fine-grain basis, and so have to be tested on a more coarse scale:
if fragments of code compile and behave as expected. this leaves a lot of
need for manual intervention in testing and debugging, such as reviewing the
output of various internal processes, trying to track down specific errors
(in possibly a large mess of other data), making debugging a relatively
labor-intensive process.
this is, sadly, an ongoing battle in my cases.
<--
As a community, I'm trying to get us to shift from being the kid in maths
class who says "12" when the teacher says "What is 3 times 4"? The working
out is at least as important as the answer... as we all know in later
mathematics, more weight is attributed to the process than to the outcome...
if there are 12 steps in a process to build an answer, and only the 12th is
incorrect, then that is 11 out of 12 as a score, as averse to if there is
only an incorrect answer and no working out, that is 0 out of 12 as a
score... or 12 out of 12 if it is correct.
Do you see my point here?
-->
hmm... not all of use are good with maths...
<--
Of course, just as it's possible to write structured code inside Smalltalk,
or any Object-Oriented language, it would be possible to "get around" this
encouragement of behavioural-driven-testing built into a language. But
that's beside the point. I'm not trying to get to a state where things are
IMPOSSIBLE to break. I'm trying to get to a state where they're encouraged
to work.
If I didn't want to write tests, I could simply write tests that evaluated
in true. Hands dusted and done. Mind you, the stakeholders (maybe me) would
probably get frustrated as the project gets larger and more regression
errors creep in with each suite of changes imposed on the system.
-->
even OO is not a requirement:
even though OO is popular, not everything needs to be OO or in an OO
language, and OO or not-OO is really a side issue to the matter of unit
tests...
sadly, if the tests prevented the project from building or running if they
failed, this would essentially leave the tests always (or almost always)
evaluating to true as the only really reasonable outcome in most cases...
granted, testing could have separate "warning" and "error" conditions, where
a warning condition is a test which fails, but the code can still be run
(only something minor has broken, or correct behavior remains as a future
goal), and an error condition is where "something has just gone horribly
wrong here", with the assumption that the test itself can be used to help
debug the problem.
often, it is still needed to run even known broken code in order to help
better isolate the problem.
now, preventing doing an official release with broken code is an altogether
different issue...
or such...
_______________________________________________
fonc mailing list
[email protected]
http://vpri.org/mailman/listinfo/fonc
