Hi all, first, thanks Chris for bringing this up. I also wanted to start this discussion on the list but thank you very much for doing it and the nice writeup.
We now have 4 languages (Java, C++, C# and Python) and hopefully more to come (C would be really nice, I personally think and as some of you may know, I'm a C master now). So I think it is a good time now to "loosen" the coupling between the languages (at least until code-gen comes, then we have to reconsider). I also have the hope that we will grow our community and have "subgroups" that take care of these languages. So, I definitely favor the "native build system" approach which. On the other hand, I totally agree with Chris, that we have to keep an eye on keeping things release-able. There are two general options for that... separate repos and separate release cycles or, as Chris suggests, some kind of "meta-automation". I cannot judge (have not enough experience) to see which one of both approaches is worse, but I think I prefer the latter. I just checked how Apache Arrow handles this (they also have separate language bindings, see https://github.com/apache/arrow) and it looks like they have the latter approach and bind it together via CMake. To come to a conclusion I think our best bet is to keep it kind of as-is but do this a bit more explicit. That means, * every lang has a subfolder and uses a native build tool * we provide an opportunity to bind all that together to allow for "joint" builds and releases But, two other things which are wort notice. I agree with Björn that we should rethink the current way we handle thrift, as it feels a bit unnatural for me ( well, that’s well known by now). One option could be to provide a docker container for that (arrow does it that way, I think) to perform the code generation. Second, I suggest to consider splitting maven into to "separate" poms. This means we use maven as "multi-build-tool" on one hand to orchestrate the build for us but have a separate java maven build for the maven subfolder. I cannot say whether maven is the optimal solution for the "multi-build-tool", but I guess that’s always hard and our maven integration currently works pretty nice and flawless so it would be good to keep that. Perhaps Chris as maven expert can say something about if it is feasible to do what I suggest and makes any sense? Julian Am 27.05.19, 17:03 schrieb "Bjoern Hoeper" <hoe...@ltsoft.de>: Hey everyone, I would also opt for Maven being at least the triggering build system because it also integrates well with Jenkins and the mechanisms behind it are quite understandable. I think the main point we need to solve is to get some kind of documentation which build step in which profile triggers what, with which set of parameters to make it easier for people not working on the native parts to debug stuff. I think Chris Preflight check already adds a lot to this point. I think for .NET we found quite a nice solution because it integrates well. For C++ it is always difficult to get everything working with different architectures and stuff. But I think the current Maven / CMake Integration is as good as it gets. One question that remains open for me is if it is really necessary to build Thrift and all of Boost during the C++ compilation because both take a lot of time and are quite error prone. So on the bottom line +1 for the integrated Maven solution. But I think it is not a real yes/no question but more of a "yes...but" kind of thing. Best Regards Björn -----Ursprüngliche Nachricht----- Von: Christofer Dutz <christofer.d...@c-ware.de> Gesendet: Montag, 27. Mai 2019 14:42 An: dev@plc4x.apache.org Betreff: [BUILDS] What build system to use? Hi all, as the discussion had come up multiple times now … mainly on Slack, I would like to do a dedicated discussion here on the list about the topic. Up to now we had built the build to generally build every module with maven, utilizing plugins to enable triggering builds with things like CMake. The maven build then took care of packing the results and handling the dependencies (At least this was how I built things for the C++ module) With the plc4net however we’re using an execution of “dotnet” to build a solution. In this case the build is entirely handled by dotnet and all maven does is trigger the build and fail if the execution fails with any non 0 return code. We are currently doing something similar with python: where we’re just calling python to execute the setup.py script. What is your opinion on how we should do our builds? If we go down the path like I initially setup C++, we have the benefit of utilizing the maven ecosystem and we will not have any problems during releases. It however comes with some comfort disadvantages for the “native” guys. But things like code-generation will work nicely. If we go down a non-maven path we will have to deal with all of this ourselves … we will have to come up with a way to handle everything ourselves Or learn how to do it in python and CMake and dotnet, and … The other thing is that we can’t integrate the driver generation as easy as with maven. Either we manually execute the code generation multiple times with maven to generate multiple output directories before executing the individual build system Or we have to build custom generators for every build-system we are using. Even if I like the way I setup the C++ build, I doubt all people will like it as it requires them to do things differently than they are used to. But I would strongly object implementing multiple code-generators. One option would be to split up the plc4x git repo into multiple repos – each one for one language … so we’d have a plc4j, plc4cpp, plc4net and plc4python. Each of these would be dedicated to one particular language and have build tools that fit them. However I strongly object this option, as it would require the Release manager to understand and setup each of these environments. Another option which I think would be a valid compromise, would be to have all languages in one repo the way we currently have it and Maven as triggering build system. The code is generated by maven but the build itself is handled by the particular build system. This will require people working exclusively in VisualStudio or some Python IDE, to manually run a “mvn generate-sources” first, but I think that’s a reasonable restriction. The benefit is that releasing this should be possible with our current release process which has a limited setup cost for the Release Manager. I would opt for the hybrid option with Maven as initiating build system for releases and code generation and CI stuff but to have the native build system for every language to build the individual parts. Namely this would be: * Plc4cpp: CMake * Plc4net: dotnet * Plc4Python: python setup.py Not quite sure how we can get the test-results to be reported correctly. What are your thoughts on this? Chris
