I know I'm late to the party here, but I just want to say that I think integration tests have far greater value to OFBiz than unit tests. Mostly because we tend to have quite a low number of tests and integration tests give us much better coverage per line of test code and the tests are much closer to the real world scenarios the application might encounter.
I don't really see how unit tests could be applied to non-framework testing in a useful manner, could you expand on your vision in that regard? I mean would we be testing something smaller than a service as the 'unit'? What would we mock? Would transaction management still be active? What happens when the service calls another service, I guess we mock the response from that service (how)? It just seems a very complicated method to achieve a less thorough but albeit faster (maybe) test result. A build, data load and full test run takes 4m 9s on my laptop. Excluding build, data load and framework tests: the application level tests take 35s, not very expensive IMO. The data load time can be reduced to practically nothing by copying a clean slate database into runtime for each run. I'm mostly just suggesting we be wary of adding complicated testing procedures in the hope of achieving some 'best practice' result which in reality will provide minimal benefits. Regards Scott On 18 July 2016 at 18:57, Taher Alkhateeb <[email protected]> wrote: > Hello Akash, > > Fantastic, I have a few unit tests almost done to be included in the start > component. I will create a new subtask under OFBIZ-1463 to commit the tests > so you can use them as a reference if you like to. > > I also recommend that you follow the same directory structure between the > test code and production code. So for example: > > Production code: framework/start/src/main/java/org/apache/ofbiz/base/start > Test code: framework/start/src/test/java/org/apache/ofbiz/base/start > > The benefit of this hierarchy is that you can access non-public (package > protected) methods for testing. This is in fact exactly what I needed to be > able to apply some of the tests. > > Cheers, > > Taher Alkhateeb > > On Mon, Jul 18, 2016 at 9:22 AM, Akash Jain <[email protected]> > wrote: > > > Thanks Taher for nice initiative! > > > > We are planning to written unit tests to all components under OFBIZ-1463 > > > > Thanks and Regards > > -- > > Akash Jain > > > > On Mon, Jul 18, 2016 at 10:36 AM, Taher Alkhateeb < > > [email protected]> wrote: > > > > > Hello Everyone, > > > > > > In reference to this thread and the Jira OFBIZ-7254, I'm very happy to > > > announce that OFBiz is now ready for applying unit tests with only 8 > new > > > lines of code in the build script (r1753143) :) > > > > > > I recommend we do the following moving forward: > > > > > > 1- Introduce unit tests as much as we can to all components > > > 2- Migrate most of the integration tests we currently have to unit > tests > > > (since they are designed to do very little integration). > > > > > > This is a great chance for us to practice real TDD in OFBiz > > > > > > Cheers, > > > > > > Taher Alkhateeb > > > > > > On Mon, Jun 13, 2016 at 7:43 AM, Pranay Pandey < > > > [email protected]> wrote: > > > > > > > +1 > > > > > > > > Best regards, > > > > > > > > Pranay Pandey > > > > HotWax Systems > > > > http://www.hotwaxsystems.com/ > > > > > > > > On Fri, Jun 10, 2016 at 7:46 PM, Taher Alkhateeb < > > > > [email protected] > > > > > wrote: > > > > > > > > > Hello Everyone, > > > > > > > > > > I was able to get a few tests running and this is very doable. But > I > > > > faced > > > > > a big problem in designing the testing framework because of ANT. > > > > > > > > > > The problem > > > > > ---------------- > > > > > The way the build scripts are designed in OFBiz are very complex. A > > > > master > > > > > file calls other files which call other files. And in the middle > you > > > have > > > > > external libraries (ant-contrib) and macros, and variables, and > class > > > > path > > > > > declarations, and and and .... > > > > > > > > > > I cannot declare the tests programmatically (with JUnit test > suites) > > > > > because this means lower level components would depend on higher > > level > > > > > components. So I have to do it in ANT, by navigating this maze of > > build > > > > > scripts, and it was a headache for me just to read them, let alone > > > modify > > > > > them to create a testing framework. > > > > > > > > > > Suggested Solution > > > > > ------------------------ > > > > > I suggest to implement the testing framework in Gradle, and simply > > call > > > > it > > > > > from within ant. This is a middle solution that sustains ant for > now, > > > but > > > > > can allow us to switch out later. > > > > > > > > > > This means I will just add one more file called build.gradle in the > > top > > > > > level directory, and figure out the business logic for calling the > > test > > > > > suites from that file > > > > > > > > > > I look forward to your feedback. > > > > > > > > > > Regards, > > > > > > > > > > Taher Alkhateeb > > > > > > > > > > On Wed, Jun 8, 2016 at 6:00 PM, Taher Alkhateeb < > > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi Everyone, > > > > > > > > > > > > Thank you all for your support, JIRA created in > > > > > > https://issues.apache.org/jira/browse/OFBIZ-7254 > > > > > > > > > > > > I will start working on it and try to implement ASAP to get my > > focus > > > > back > > > > > > on refactoring. > > > > > > > > > > > > Cheers! > > > > > > > > > > > > Taher Alkhateeb > > > > > > > > > > > > On Wed, Jun 8, 2016 at 4:58 PM, Deepak Dixit < > > > > > > [email protected]> wrote: > > > > > > > > > > > >> +1 > > > > > >> > > > > > >> Thanks & Regards > > > > > >> -- > > > > > >> Deepak Dixit > > > > > >> www.hotwaxsystems.com > > > > > >> > > > > > >> On Wed, Jun 8, 2016 at 7:12 PM, Mridul Pathak < > > > > > >> [email protected]> wrote: > > > > > >> > > > > > >> > +1 > > > > > >> > > > > > > >> > Makes perfect sense. > > > > > >> > > > > > > >> > -- > > > > > >> > Thanks & Regards, > > > > > >> > Mridul Pathak > > > > > >> > Senior Manager > > > > > >> > HotWax Systems > > > > > >> > http://www.hotwaxsystems.com > > > > > >> > > > > > > >> > > On Jun 8, 2016, at 2:41 PM, Taher Alkhateeb < > > > > > >> [email protected]> > > > > > >> > wrote: > > > > > >> > > > > > > > >> > > Hello Everyone, > > > > > >> > > > > > > > >> > > After refactoring the start component and while starting on > > the > > > > base > > > > > >> > > component I realized that the testing framework for OFBiz is > > not > > > > > good. > > > > > >> > You > > > > > >> > > cannot do real test driven development or red-green-refactor > > > with > > > > > the > > > > > >> > > current setup, hence my proposal to change it. I explain > > below: > > > > > >> > > > > > > > >> > > Problem with current design > > > > > >> > > ---------------------------------------- > > > > > >> > > - What we have right now is not unit tests, it's really > > > > integration > > > > > >> > tests. > > > > > >> > > You have to start the framework, the database, the service > > > engine, > > > > > the > > > > > >> > > entity engine and pretty much everything. > > > > > >> > > - Testing is very slow, because it's an integration test as > I > > > > > >> mentioned > > > > > >> > > above. 10 minutes on a good computer! > > > > > >> > > - There is zero mocking! We actually have to --load-data for > > > > things > > > > > to > > > > > >> > > work. Again, these are integration tests. > > > > > >> > > - Too complex: Integration tests by their nature are > grabbing > > > too > > > > > >> much. > > > > > >> > > Mind you, I am not objecting to integration tests (I > actually > > > like > > > > > >> them) > > > > > >> > > but I am objecting to not having real unit-tests. Unit tests > > > > should > > > > > >> all > > > > > >> > run > > > > > >> > > in a few seconds. > > > > > >> > > > > > > > >> > > Proposed solution > > > > > >> > > -------------------------- > > > > > >> > > - We keep what is considered real integration tests the way > > they > > > > are > > > > > >> > right > > > > > >> > > now and keep using them > > > > > >> > > - We move what should be unit tests into simple JUnit > classes, > > > and > > > > > we > > > > > >> do > > > > > >> > > not run them using java -jar ofbiz.jar --test, but instead > run > > > > them > > > > > >> > > directly from the build.xml script, so these files are not > > > > > identified > > > > > >> in > > > > > >> > > any XML document, but are simply called immediately from the > > > build > > > > > >> > scripts. > > > > > >> > > - We clearly mark the difference between integration tests > and > > > > unit > > > > > >> tests > > > > > >> > > (inside the source files or in the suite declarations). > > > > > >> > > - We change the run-tests target in build.xml to run both > unit > > > > tests > > > > > >> and > > > > > >> > > integration tests. > > > > > >> > > > > > > > >> > > I intend to heavily refactor the framework and I would feel > > > better > > > > > >> about > > > > > >> > > introducing this change while refactoring. What do you guys > > > think? > > > > > >> Ideas? > > > > > >> > > Suggestions? Approvals and thumbs up? > > > > > >> > > > > > > > >> > > Regards, > > > > > >> > > > > > > > >> > > Taher Alkhateeb > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > >
