On Tue, 11 Nov 2014, Thomas D. wrote:
Hi,
On 2014-11-11 03:13, David Lang wrote:
you still need those credentials for your sandbox, right?
If MySQL isn't installed on the system you will need to install the package
(requiring root + distro specific knowledge), if it is installed, you need to
configure users and permissions for the test user (requiring DB root equivalent
+ distro specific knowlege)
Yes, running tests against any third-party application would require
access to those applications while testing. Theses tests are called
"integration tests".
Because you should *not* depend on those third-party applications you
often cannot run your integration tests every time you run the normal
test suite.
Therefore you would split your tests. And this is already done for
rsyslog: The mysql tests for example are separated.
But this doesn't mean you cannot test the ommysql at all without mysql:
You can write a mock so you can test your code at least.
BTW: Travis will allow us to test against a running mysql each time...
Add in Oracle, Postgres, RabbitMQ, 0MQ, journald, ElasticSearch, etc.
The vast majority of the problems that we run into with rsyslog are in the
interfaces to these third party destinations.
But let me be clear. From this thread, it looks like these should be our
project priorities:
1. make the testbench fully automatic and self-contained
2. create unit tests
3. create more and better integration tests
4. document
5. fix bugs
6. develop new features
[...]
[...]
Yes, I am voting for changing the priority order like you said.
I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 lives is
subject to a lot of debate)
Wait, this order is not set in stone.
For example, to write tests, you have to know what a component should do
and how it should be used. So you need a documentation before you can
write tests.
<snip>
Our views are not so far apart.
To be clear: I don't want to establish a process above the product.
But the process should be part of the product. Change the way you
currently work:
When developing a new feature, start with the documentation.
If you know how your new feature should look like, should be used, start
writing tests so you can be sure your new feature is actual doing what
you are are expecting.
Now you can start the actual coding...
In the end you have a new feature, fully documented and tested.
If you don't follow this work flow, you will end up with the current
situation: You maybe have great features -- but nobody knows due to
missing/hard to use documentation.
The tests will help you to keep your hard work working. Imagine you
spend days for a new feature but another change will break it. Without
tests you will only notice if somebody will actual try to use your
feature and fill a bug. But we all know that this don't happen that
often. Also, if you start looking for a new product which maybe solve
your problem you are currently facing and experience a bug which is a
show stopper for you in this moment, you will skip this product.
you _are_ argueing for process over product. You are pushing for strict
test-driven-development, and while there are a lot of academics who really
believe in such development, there are almost no development teams in the real
world (commercial or opensource) that work that way for very long.
The problem with documentation isn't that no documents get written (with the
exception of some contributed modules), but rather that the documentation has
been written by someone so close to the problem that there is a huge amount of
implied context that isn't obvious to others, and that the organization of the
documentation is based on how the person writing it thinks about the problems,
not how the people who don't know about the software think about the problems.
There are also cases that the author (both software and documentation) didn't
think of when programming/wriring, and so those may not be documented.
So if Rainer should decide to switch the work flow like described, this
would only help new features. But we still have to deal with the past.
That's why I recommend to change priority for a moment. Like a
"sprint"... like a bug hunt day/week/month where everybody focus on
squashing bugs.
You keep missing that there isn't a lot of "everybody" in rsyslog development.
Rainer is the author of >90% of both the code and the documentation. He is very
willing to accept contributions, but we need to get other people involved. The
split of the documentation into a different repository was to help ease the
process of contributing documentation, but after a brief initial flurry, the
outside contributions have tapered off.
Please contribute tests, contribute documentation, contribute new features with
any process that you want.
And feel free to discuss development models, but recognize that there are a lot
of them out there and that there is no "one true way".
Writing good tests is a lot harder than you make it out to be, and frequently
tests get written that look like they test one thing, but actually end up
testing something subtly different.
Rainer is listening and adapting to suggestions, but he's also asking for help.
If there is an upcoming "national trade show" where you want to show
something new, yes... do that (but please follow the new work flow). But
I am not aware of something like that in the next 4 months and I am not
aware of a missing feature which must be implemented ASAP or rsyslog
will go away...
That was an example from another business of how strict adherence to "tests for
everything and all tests must pass" ended up pretty much dooming a company.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
