Hi Chris, I agree with you and with the latest work you have done it gets more and more concise, I guess. So lets finish PLC4X-111 and then see where it brings us. And perhaps a solution could also be to simply bring the "proxy" thing out of the other structure as this would keep the build there (because modules are pretty tightly coupled), perhaps even without submodules for other languages... and only have those on Top Level.
Julian PS.: It really is an art to write short and concise emails... and it seems like we're only mediocre talented : / Am 29.04.19, 10:25 schrieb "Christofer Dutz" <[email protected]>: Hi Julian, just saw that I didn't respond to that email ... gotta add my thoughts ;-) I would totally put all +1s I have (which is 1) into arranging the project by language. The reason for this is the complexity of the build. If we put all Java stuff into plc4j and all C++ stuff into plc4cpp and all python stuff into plc4py, we can define all the settings in that particular root module and inherit them to all sub modules. This would prevent Java setting from leaking into c++ modules (Currently the build seems to be running JavaDoc on C++ code ;-) ) While maven would allow separating the directory structure from the pom structure, however I have used this several times before and it was always a nightmare in the end. We're currently not only using Maven ... maven is the coordinator. For the C++ modules we are using CMake which seems to be doing a great job for building on multiple platforms and Tool- and IDE-Support. I would try to stick to maven as close as possible though. Keep in mind, when doing releases we have to do quite a lot of stuff to comply to Apache's rules. With this "Maven as master build" we get a lot of bases covered: RAT checks, Source distribution building, signing, staging, ... If we start splitting up into separate builds. We have to ensure all of these for each build tool and especially we have to learn all of these in order to be able to maintain things. Right now regarding the tooling and as I already stated in multiple chats: I would like to make the build as simple as possible for new people to setup. I have worked in other Apache projects where it tool multiple attempts and hours of setting up before being able to start. This is the horror scenario, I would like to avoid. So I thought we should keep the parts that are very general outside the project, but the parts we need control over the version inside. So as an example, If we use Thrift for the proxies and the intermediate drivers, we need to have control over the version of the Thrift compiler used. If we don't we are guaranteed to get problems when updating to newer Thrift versions and we'll have to deal with those support complaints that arise from this. Also if in the C++ module, the base framework is Boost, this also has great impact on the code, having different versions of Boost could also produce hard to diagnose errors. That's the reason I updated the build to download and build exactly those versions we are using. So if for example we use a newer version of Thrift, we update a property in the master pom and run the build and it will build and use exactly that thrift version. There's nothing that scares new committers more than not being able to checkout and build the project in reasonable time. That’s why I'm investing so much time in the build, cause I know this is probably the most crucial part in getting new people on board. Chris PS: I really hope we won't continue these mega-email threads ... I really hate when emails I write and read exceed a screen size ;-) Am 27.04.19, 19:20 schrieb "Julian Feinauer" <[email protected]>: Hi Chris, first, thank you for all your work on the build. One feat that gets stronger and stronger for me is the complexity of our build, which I dislike. I know that this is a necessity of our current setup but I think we have to do something about it as the build himself currently has the complexity of a simple driver but "no" testcoverage and is very hard to debug (and very few are able to maintain or even understand it, myself included). So I think we have to think about how we can change this and this is, from my perspective the most important issue and should drive how we organize our modules. This includes questions like - should we organize by language or by "role" (which is basically your question) - should we only use maven as driver for the build or should we decouple all builds and leave it to Jenkins to execute all of these - should we split repos or should we keep one repo as it is currently - how do we handle interdependencies between different languages - how to we deal with general "tools" in the build pipeline (longer idempotent build vs complexer individual setup) I know that some of these suggestions would imply large changes but I think we should not drive these important decisions. Julian Am 27.04.19, 18:43 schrieb "Christofer Dutz" <[email protected]>: Hi all, currently we have a structure where Java modules are located in plc4j, c++ modules in plc4cpp and python modules in plc4py … however the integration modules and examples are at root level. Even if we don’t yet have any non java integration modules or examples, we will be getting some in the near future. What is observable, is that Java plugin configuration is leaking over to the cpp and python modules, which isn’t good. For the sake of a simpler build and cleaner configuration, I would propose to create integrations and examples modules inside each of the language directories and to move the existing modules there. What do you think? Chris
