Re: [darktable-dev] Unit Tests
Hi Andreas, Thank you very much for your feedback and the offer to review my code. I am willing to support you also on cmocka - should something pop into my eyes. To me, it makes sense to integrate also a code coverage analyzer. I don't have experience there but I will definitely look into this. Also thank you, Greg Hughes, for your input! Please find the pull request here: https://github.com/darktable-org/darktable/pull/4154 Looking forward to your feedback (and of course also from other devs). Please keep in mind that this is just a prototype :-) Do you know a better way to declare the mocked functions? Currently, I have to list every one in the src/tests/CMakeLists.txt Best regards, Martin On 2020-01-20 11:06, Andreas Schneider wrote: On Sunday, 19 January 2020 19:59:30 CET i...@burrima.ch wrote: Hi David, Hi Martin, Thank you for your reply. What a co-incidence, I was not aware of that. That's good news! I came to cmocka while analyzing different frameworks and I thought this might be the most suitable one. I'm the cmocka developer and maintainer. It looks like you already got the gist of it especially the mocking functionality. This is great. I also would appreciate patches to cmocka to improve the docs if you find anything. Only had a quick look at your integration, looks similar to mine. If you could create a pull request I would be happy to do a code review. There are code coverage tools around, though I don't know any of them. As a first start I thought it would be enough to go just straight (aka "human code coverage analyzer" ;-)) and then increase the coverage later. But I don't mind using such tools from beginning on. Do you know any such tools or shall I look around? As soon as you use ctest you have it available with cmake, see "make help", there is e.g. make ExperimentalCoverage if you set the path to gcov. However it runs it only over the tests so we are probably at 1% :-) I also thought it would be good to focus on the process() methods of the darktable modules first. Bigger units in terms of "unit testing", maybe not full coverage, but probably very effective as a first quality improvement (and also good for me to get into the code and into the mathematics of the modules). Thank you for the notice about IRC - I will look into this. I'm gladiac on IRC. Cheers, Andreas ___ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org ___ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
Re: [darktable-dev] Unit Tests
On Sunday, 19 January 2020 19:59:30 CET i...@burrima.ch wrote: > Hi David, Hi Martin, > Thank you for your reply. What a co-incidence, I was not aware of that. > That's good news! > > I came to cmocka while analyzing different frameworks and I thought this > might be the most suitable one. I'm the cmocka developer and maintainer. It looks like you already got the gist of it especially the mocking functionality. This is great. I also would appreciate patches to cmocka to improve the docs if you find anything. > Only had a quick look at your integration, looks similar to mine. If you could create a pull request I would be happy to do a code review. > There are code coverage tools around, though I don't know any of them. > As a first start I thought it would be enough to go just straight (aka > "human code coverage analyzer" ;-)) and then increase the coverage > later. But I don't mind using such tools from beginning on. Do you know > any such tools or shall I look around? As soon as you use ctest you have it available with cmake, see "make help", there is e.g. make ExperimentalCoverage if you set the path to gcov. However it runs it only over the tests so we are probably at 1% :-) > I also thought it would be good to focus on the process() methods of the > darktable modules first. Bigger units in terms of "unit testing", maybe > not full coverage, but probably very effective as a first quality > improvement (and also good for me to get into the code and into the > mathematics of the modules). > > Thank you for the notice about IRC - I will look into this. I'm gladiac on IRC. Cheers, Andreas ___ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
Re: [darktable-dev] Unit Tests
The usual suspect GCC toolchain coverage tool is gcov https://en.wikipedia.org/wiki/Gcov I don't have any experience with it. My Company uses Bullseye, however it is not free or open source. There are code coverage tools around, though I don't know any of them. As a first start I thought it would be enough to go just straight (aka "human code coverage analyzer" ;-)) and then increase the coverage later. But I don't mind using such tools from beginning on. Do you know any such tools or shall I look around? ___ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
Re: [darktable-dev] Unit Tests
Hi David, Thank you for your reply. What a co-incidence, I was not aware of that. That's good news! I came to cmocka while analyzing different frameworks and I thought this might be the most suitable one. Only had a quick look at your integration, looks similar to mine. There are code coverage tools around, though I don't know any of them. As a first start I thought it would be enough to go just straight (aka "human code coverage analyzer" ;-)) and then increase the coverage later. But I don't mind using such tools from beginning on. Do you know any such tools or shall I look around? I also thought it would be good to focus on the process() methods of the darktable modules first. Bigger units in terms of "unit testing", maybe not full coverage, but probably very effective as a first quality improvement (and also good for me to get into the code and into the mathematics of the modules). Thank you for the notice about IRC - I will look into this. Best regards, Martin PS: I have to manually add darktable-dev@lists.darktable.org if I like to reply also on the mailing list, right? On 2020-01-19 18:58, Coding Dave wrote: Hi Martin, Great timing. We have just discussed about unit tests 6 days ago and decided for cmocka as well. There is a test integration here: https://github.com/cryptomilk/darktable/tree/master-cmocka?files=1 We came to cmocka because I realized that I might break a lot of things when I try to fix the current bug I'm looking at and the cmocka maintainer is a dt dev too. I appreciate your enthusiasm and personally would be very happy to have unit tests (and later CI as well). I did not analyze your patch in detail but I wonder how you detect the units that need testing. Do you use sort of a C code coverage analyzer? Kinds regards David PS, if you want to reach some devs check out freenode #darktable. You could use matrix servers for accessing IRC, then it behaves more modern (as if you were having a bouncer). schrieb am So., 19. Jan. 2020, 18:08: Dear developers, First of all: thank you so much for your valuable work that you invest into darktable! I really enjoy using this software. I personally think that you could need some help and thus this mail. I am intending to add some unit tests in order to verify the code on a low level. I think you have some users/testers reporting issues that sound sometimes a bit fuzzy and are hard to find. Unit testing can test the code for corner cases and find flaws before the users find them. It is also extremely helpful when it comes to refactoring. The past few days I invested some time to prepare a small feasibility study based on cmocka (https://cmocka.org/). It is easy to use and has the possibility to mock functions, i.e. to change functions in their behavior for testing. I was able to test as an example the process() method of the new filmicrgb module by providing some minimal environment and just 1 pixel as input parameters. The code can be found under https://burrima.ch/dl/0001-Added-simple-unit-test-sample-for-demonstration.patch (note: I currently don't have a github account but I will create one later). In this first email, I won't go into technical details because I first want to know if you generally have interest in unit tests and in me participating in this project. I have many more ideas what could be done for darktable in order to increase the quality - e.g. stuff going towards modern development methods: continuous integration, automated testing, etc. etc. - but that goes way too far beyond the scope of this email :-) Some words about me: My name is Martin Burri and I live in Switzerland, close to Basel. You can find my website with more information about me under https://burrima.ch (bit old site but still valid). I work 90% for a big company as Senior Software Development Engineer. We are developing mission critical communication equipment for power utilities. Thus, I have a very high awareness of software quality. I always try to write simple and maintainable code. In my spare time, I already had some smaller personal coding projects, mainly for myself without releasing them to the public. But I am sick of that, I want to do something more reasonable where others can profit from. I can imagine to support darktable for a longer period of time, not just for a one-shot. Usually, I can invest about 5-10 hours per week into private projects, sometimes more, sometimes less. Et pour les dévelopeurs Français: Ma Femme est demi-Française et à cause de ça je peux comprendre pas mal - donc écrire est une histoire différente :-) Looking forward to your feedback! Best regards, Martin ___ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org ___ darktable developer mailing list to unsubscribe send
Re: [darktable-dev] Unit Tests
Hi Martin, Great timing. We have just discussed about unit tests 6 days ago and decided for cmocka as well. There is a test integration here: https://github.com/cryptomilk/darktable/tree/master-cmocka?files=1 We came to cmocka because I realized that I might break a lot of things when I try to fix the current bug I'm looking at and the cmocka maintainer is a dt dev too. I appreciate your enthusiasm and personally would be very happy to have unit tests (and later CI as well). I did not analyze your patch in detail but I wonder how you detect the units that need testing. Do you use sort of a C code coverage analyzer? Kinds regards David PS, if you want to reach some devs check out freenode #darktable. You could use matrix servers for accessing IRC, then it behaves more modern (as if you were having a bouncer). schrieb am So., 19. Jan. 2020, 18:08: > Dear developers, > > First of all: thank you so much for your valuable work that you invest > into darktable! I really enjoy using this software. > > I personally think that you could need some help and thus this mail. I > am intending to add some unit tests in order to verify the code on a low > level. I think you have some users/testers reporting issues that sound > sometimes a bit fuzzy and are hard to find. Unit testing can test the > code for corner cases and find flaws before the users find them. It is > also extremely helpful when it comes to refactoring. > > The past few days I invested some time to prepare a small feasibility > study based on cmocka (https://cmocka.org/). It is easy to use and has > the possibility to mock functions, i.e. to change functions in their > behavior for testing. I was able to test as an example the process() > method of the new filmicrgb module by providing some minimal environment > and just 1 pixel as input parameters. The code can be found under > > https://burrima.ch/dl/0001-Added-simple-unit-test-sample-for-demonstration.patch > (note: I currently don't have a github account but I will create one > later). > > In this first email, I won't go into technical details because I first > want to know if you generally have interest in unit tests and in me > participating in this project. > > I have many more ideas what could be done for darktable in order to > increase the quality - e.g. stuff going towards modern development > methods: continuous integration, automated testing, etc. etc. - but that > goes way too far beyond the scope of this email :-) > > Some words about me: > > My name is Martin Burri and I live in Switzerland, close to Basel. You > can find my website with more information about me under > https://burrima.ch (bit old site but still valid). I work 90% for a big > company as Senior Software Development Engineer. We are developing > mission critical communication equipment for power utilities. Thus, I > have a very high awareness of software quality. I always try to write > simple and maintainable code. > > In my spare time, I already had some smaller personal coding projects, > mainly for myself without releasing them to the public. But I am sick of > that, I want to do something more reasonable where others can profit > from. I can imagine to support darktable for a longer period of time, > not just for a one-shot. Usually, I can invest about 5-10 hours per week > into private projects, sometimes more, sometimes less. > > Et pour les dévelopeurs Français: Ma Femme est demi-Française et à cause > de ça je peux comprendre pas mal - donc écrire est une histoire > différente :-) > > Looking forward to your feedback! > > Best regards, > Martin > ___ > darktable developer mailing list > to unsubscribe send a mail to > darktable-dev+unsubscr...@lists.darktable.org > > ___ darktable developer mailing list to unsubscribe send a mail to darktable-dev+unsubscr...@lists.darktable.org
