Hi everyone,
unit tests and integration tests are complementary and with test, the
more the better.
Implementing unit tests will be a great load of work but it is the same
with every kind of test. As Taher said, we will gain a lot from those
unit tests for avoiding regression while refactoring the framework. From
the start component scope, it has already be proved useful.
And what Ron said about encouraging the creation of "smaller class that
can be tested easily" is very important too. It can be a first step to
reach TDD in the OFBiz development. TDD can help to give confidence to
newcomer and veteran when updating some part of the code and as Hans
said, this lead to "continuous improvement and finally continuous
deployment".
I'm aware that this does not solve the technical implementation stuff,
but I think this is an improvement that need to be supported.
Have a nice day,
Florian Montalbano
Le 19/07/2016 09:04, Pierre Smits a écrit :
As with anything, the law of diminishing returns also applies to OFBiz and
tests. This is not true for unit tests and system integration tests, but
also for user acceptance and performance tests.
Nevertheless, the work done up to now is a good start and - I feel
confident - appreciated. And unit tests are certainly valuable in the
framework stack. How it will be for functions (regarding components in
application and special purpose stack) needs to be addressed when we reach
that bridge.
Best regards,
Pierre Smits
ORRTIZ.COM <http://www.orrtiz.com>
OFBiz based solutions & services
OFBiz Extensions Marketplace
http://oem.ofbizci.net/oci-2/
On Tue, Jul 19, 2016 at 8:22 AM, Taher Alkhateeb <[email protected]
wrote:
Hi Scott,
Well spoken thank you. Okay may I suggest that for any such work we will
discuss it here see its Merit and if it makes sense then we take it in. It
is a little early to discuss it right now because we did not yet go to the
higher-level components. Once we do I'll be sure to have a conversation
about this in here and would appreciate your input to it.
Regards,
Taher Alkhateeb
On Jul 19, 2016 9:00 AM, "Scott Gray" <[email protected]>
wrote:
Yeah I'm sure unit tests probably worked well for the start component
given
it is the lowest level component in the system and closest to a basic
java
app. I just think the value proposition might decrease the further up
the
stack we move with them. I'm not against unit tests when they prove
useful, but further up the stack I think we should prove the case for
them
before doing much work to support the mocking that will be required to
keep
them inline with 'best practices'.
In OFBiz "bad or unwanted" tends to come mostly in the way of increasing
complexity from features that require more effort to maintain than the
value they provide to the community. I think there's a chance attempting
to mock service tests could fall into that bucket. I could be wrong,
but I
think it's worth looking into before we declare that unit tests are the
best form of testing for OFBiz.
Regards
Scott
On 19 July 2016 at 17:37, Taher Alkhateeb <[email protected]>
wrote:
Hi Scott,
Thank you for the feedback. To be focused exactly on integration vs
unit, I
already mentioned above that at least for me the main objective is to
confidently and quickly run the tests in short bursts of red-green
refactor. This allows me to refactor code without waiting in front of
the
screen in between test cycles thus giving me immediate feedback on any
errors I made. Perhaps my intro was too long so this is the squeezed
out
version of it.
I already had one round of testing in the start component which was
much
faster that way and had an immediate impact. Oh and by the way, you
cannot
test the start component with integration tests for example unless you
do
it from an external component which cannot access package protected
items.
This style of coding is applicable I think to any software project
inclusive of OFBiz. Maybe in certain components more than others or
certain
areas more than others. But I can't see how it could be bad or
unwanted.
Taher Alkhateeb
On Jul 19, 2016 8:18 AM, "Scott Gray" <[email protected]>
wrote:
Thanks Ron and Taher for your responses, I appreciate them but I
don't
hear
much in the conversation that speaks to the value of unit tests over
integration tests. Most of the thoughts shared speaks more to the
value
of
tests rather than differences in the type of tests.
Speed: At an application level we have ~685 tests that run in 35
seconds
(excluding build and data load). Another point is that there isn't
much
reason why tests can't be run in parallel rather than sequentially.
TDD: Integration tests in OFBiz are as simple if not simpler to write
than
unit tests and just as useful for TDD.
I'm not sure if I missed any other points raised regarding
integration
vs.
unit tests?
I'm not looking to start a big long debate on the topic, I just
wanted
to
speak out that someone out there (me) doesn't think unit tests are
the
best
solution for testing OFBiz applications.
Regards
Scott
On 19 July 2016 at 16:52, Taher Alkhateeb <
[email protected]>
wrote:
Hi Scott,
Thank you for your input. Your ideas are thought provoking and
enriching
to
the conversation.
The way I look at it, the general rule is usually many unit tests,
less
integration tests, lesser functional tests. So we are not excluding
any
types of test, all of them are important in certain areas for
certain
purposes with certain quantity. Usually integration tests are less
because
as you said they just grab more. I like the picture below as a
general
guideline
http://i.stack.imgur.com/fjQvQ.png
As you already mentioned unit tests are useful for the framework. I
discovered errors in code that I wrote which I was very very
careful
with.
I immediately learned the lesson that humans are not designed to
code,
and
TDD gives you confidence as you build your code with those short
30-60
second red-green refactors. I feel much much safer and more
confident
writing code that way, the test also documents how to use the api,
refactoring and feature change becomes less terrifying. Also messy
code
is
usually not test friendly thus refactoring and unit tests go
hand-in-hand
for improving the code base.
Also, I am sorry to say that the framework code is rather messy and
brittle
in many places. I think you probably encountered this yourself. The
same
is
said for the applications. Now If we start refactoring without unit
tests
then we are back to scary business. So much can go wrong so fast
and
break
things you never expected to break. The framework with all
applications
require heavy refactoring of things like massive ugly methods,
sandwiched
logic, heavy shared mutable state, hidden dependencies, poor
interfaces, and much more. Every time I touch something I get
shocked
at
how bad it looks, spaghetti logic to no end. If you refactor with
integration tests instead of unit tests then you will come down to
a
screeching halt as you wait between these test cycles. My computer
actually
takes 10 minutes or more for a full clean load data and testing.
Now talking about mocking you raise some interesting points, what
can
you
mock vs utilizing integration tests? This is an excellent question
that
does not have a simple answer, and you mostly learn as you code,
difficult
to envision without coding. However simple things like Java
services
can
be
mocked with a standard mocking class that you use everywhere.
Transactions
are difficult to mock and better left to integration tests I think.
SECAs
might be mocked by simply passing output to input as chaining
methods.
Mind
you I am not 100% sure of all of this, coding is the ultimate guide
to
what
can or cannot be done.
So I am not suggesting unit tests as a best-practice (even though
it
is).
Instead I suggest it as something that I and others did and got a
huge
psychological relief and confidence and comfort from. Swimming in
code
without tests is a terrifying business, made more terrifying if the
code
is
bad. Short bursts of red/green/red/green makes you feel good as you
build
up your logic. And I don't have an exact vision of how to do it,
because
the details always win.
I look forward to hearing your thoughts and thank you for enriching
this
conversation.
Taher Alkhateeb
On Tuesday, 19 July 2016, Scott Gray <[email protected]
wrote:
I know I'm late to the party here, but I just want to say that I
think
integration tests have far greater value to OFBiz than unit
tests.
Mostly
because we tend to have quite a low number of tests and
integration
tests
give us much better coverage per line of test code and the tests
are
much
closer to the real world scenarios the application might
encounter.
I don't really see how unit tests could be applied to
non-framework
testing
in a useful manner, could you expand on your vision in that
regard?
I
mean
would we be testing something smaller than a service as the
'unit'?
What
would we mock? Would transaction management still be active? What
happens
when the service calls another service, I guess we mock the
response
from
that service (how)?
It just seems a very complicated method to achieve a less
thorough
but
albeit faster (maybe) test result.
A build, data load and full test run takes 4m 9s on my laptop.
Excluding
build, data load and framework tests: the application level tests
take
35s,
not very expensive IMO. The data load time can be reduced to
practically
nothing by copying a clean slate database into runtime for each
run.
I'm mostly just suggesting we be wary of adding complicated
testing
procedures in the hope of achieving some 'best practice' result
which
in
reality will provide minimal benefits.
Regards
Scott
On 18 July 2016 at 18:57, Taher Alkhateeb <
[email protected]
<javascript:;>>
wrote:
Hello Akash,
Fantastic, I have a few unit tests almost done to be included
in
the
start
component. I will create a new subtask under OFBIZ-1463 to
commit
the
tests
so you can use them as a reference if you like to.
I also recommend that you follow the same directory structure
between
the
test code and production code. So for example:
Production code:
framework/start/src/main/java/org/apache/ofbiz/base/start
Test code:
framework/start/src/test/java/org/apache/ofbiz/base/start
The benefit of this hierarchy is that you can access non-public
(package
protected) methods for testing. This is in fact exactly what I
needed
to
be
able to apply some of the tests.
Cheers,
Taher Alkhateeb
On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <
[email protected] <javascript:;>>
wrote:
Thanks Taher for nice initiative!
We are planning to written unit tests to all components under
OFBIZ-1463
Thanks and Regards
--
Akash Jain
On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb <
[email protected] <javascript:;>> wrote:
Hello Everyone,
In reference to this thread and the Jira OFBIZ-7254, I'm
very
happy
to
announce that OFBiz is now ready for applying unit tests
with
only
8
new
lines of code in the build script (r1753143) :)
I recommend we do the following moving forward:
1- Introduce unit tests as much as we can to all components
2- Migrate most of the integration tests we currently have
to
unit
tests
(since they are designed to do very little integration).
This is a great chance for us to practice real TDD in OFBiz
Cheers,
Taher Alkhateeb
On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey <
[email protected] <javascript:;>> wrote:
+1
Best regards,
Pranay Pandey
HotWax Systems
http://www.hotwaxsystems.com/
On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb <
[email protected] <javascript:;>
wrote:
Hello Everyone,
I was able to get a few tests running and this is very
doable.
But
I
faced
a big problem in designing the testing framework
because
of
ANT.
The problem
----------------
The way the build scripts are designed in OFBiz are
very
complex. A
master
file calls other files which call other files. And in
the
middle
you
have
external libraries (ant-contrib) and macros, and
variables,
and
class
path
declarations, and and and ....
I cannot declare the tests programmatically (with JUnit
test
suites)
because this means lower level components would depend
on
higher
level
components. So I have to do it in ANT, by navigating
this
maze
of
build
scripts, and it was a headache for me just to read
them,
let
alone
modify
them to create a testing framework.
Suggested Solution
------------------------
I suggest to implement the testing framework in Gradle,
and
simply
call
it
from within ant. This is a middle solution that
sustains
ant
for
now,
but
can allow us to switch out later.
This means I will just add one more file called
build.gradle
in
the
top
level directory, and figure out the business logic for
calling
the
test
suites from that file
I look forward to your feedback.
Regards,
Taher Alkhateeb
On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb <
[email protected] <javascript:;>>
wrote:
Hi Everyone,
Thank you all for your support, JIRA created in
https://issues.apache.org/jira/browse/OFBIZ-7254
I will start working on it and try to implement ASAP
to
get
my
focus
back
on refactoring.
Cheers!
Taher Alkhateeb
On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit <
[email protected] <javascript:;>>
wrote:
+1
Thanks & Regards
--
Deepak Dixit
www.hotwaxsystems.com
On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak <
[email protected] <javascript:;>>
wrote:
+1
Makes perfect sense.
--
Thanks & Regards,
Mridul Pathak
Senior Manager
HotWax Systems
http://www.hotwaxsystems.com
On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb <
[email protected] <javascript:;>>
wrote:
Hello Everyone,
After refactoring the start component and while
starting
on
the
base
component I realized that the testing framework
for
OFBiz
is
not
good.
You
cannot do real test driven development or
red-green-refactor
with
the
current setup, hence my proposal to change it. I
explain
below:
Problem with current design
----------------------------------------
- What we have right now is not unit tests, it's
really
integration
tests.
You have to start the framework, the database,
the
service
engine,
the
entity engine and pretty much everything.
- Testing is very slow, because it's an
integration
test
as
I
mentioned
above. 10 minutes on a good computer!
- There is zero mocking! We actually have to
--load-data
for
things
to
work. Again, these are integration tests.
- Too complex: Integration tests by their nature
are
grabbing
too
much.
Mind you, I am not objecting to integration
tests
(I
actually
like
them)
but I am objecting to not having real
unit-tests.
Unit
tests
should
all
run
in a few seconds.
Proposed solution
--------------------------
- We keep what is considered real integration
tests
the
way
they
are
right
now and keep using them
- We move what should be unit tests into simple
JUnit
classes,
and
we
do
not run them using java -jar ofbiz.jar --test,
but
instead
run
them
directly from the build.xml script, so these
files
are
not
identified
in
any XML document, but are simply called
immediately
from
the
build
scripts.
- We clearly mark the difference between
integration
tests
and
unit
tests
(inside the source files or in the suite
declarations).
- We change the run-tests target in build.xml to
run
both
unit
tests
and
integration tests.
I intend to heavily refactor the framework and I
would
feel
better
about
introducing this change while refactoring. What
do
you
guys
think?
Ideas?
Suggestions? Approvals and thumbs up?
Regards,
Taher Alkhateeb
