AW: [BUILDS] What build system to use?
I'have push to Branch feature/s7-cpp Freundliche Grüße Markus Sommer Geschäftsführer isb innovative software businesses GmbH Otto-Lilienthal-Strasse 2 D - 88046 Friedrichshafen Tel.: +49 (0) 7541 3834-14 Mob: +49 (0) 171 537 8437 Fax: +49 (0) 7541 3834-20 E-Mail: som...@isb-fn.de Web: www.isb-fn.de Geschäftsführer: Markus Sommer, Thomas Zeler Sitz: Friedrichshafen Registergericht: Amtsgericht Ulm HRB-Nr. 631624 Important Note: This e-mail and any attachments are confidential, may contain trade secrets and may well also be legally privileged or otherwise protected from disclosure. If you have received it in error, you are on notice of its status. Please notify us immediately by reply e-mail and then delete his e-mail and any attachment from your system. If you are not the intended recipient please understand that you must not copy this e-mail or any attachments or disclose the contents to any other person. Thank you. -Ursprüngliche Nachricht- Von: Christofer Dutz Gesendet: Dienstag, 28. Mai 2019 12:34 An: dev@plc4x.apache.org Betreff: Re: [BUILDS] What build system to use? Hi Markus, that would be great :-) Chris Am 28.05.19, 11:37 schrieb "Markus Sommer" : Hello Chris, in Case of the CMake files I already did some things here. I would push the CMake files to compile the scources on the branch, so you only have to bind the Pom.xml. Best regards Markus Freundliche Grüße Markus Sommer Geschäftsführer isb innovative software businesses GmbH Otto-Lilienthal-Strasse 2 D - 88046 Friedrichshafen Tel.:+49 (0) 7541 3834-14 Mob: +49 (0) 171 537 8437 Fax: +49 (0) 7541 3834-20 E-Mail: som...@isb-fn.de Web: www.isb-fn.de Geschäftsführer: Markus Sommer, Thomas Zeler Sitz: Friedrichshafen Registergericht: Amtsgericht Ulm HRB-Nr. 631624 Important Note: This e-mail and any attachments are confidential, may contain trade secrets and may well also be legally privileged or otherwise protected from disclosure. If you have received it in error, you are on notice of its status. Please notify us immediately by reply e-mail and then delete his e-mail and any attachment from your system. If you are not the intended recipient please understand that you must not copy this e-mail or any attachments or disclose the contents to any other person. Thank you. -Ursprüngliche Nachricht- Von: Christofer Dutz Gesendet: Dienstag, 28. Mai 2019 10:22 An: dev@plc4x.apache.org Betreff: Re: [BUILDS] What build system to use? Hi all, I'll try to summarize responses to all ( So in general you are quite happy as it currently is, however the C++ module could use simplification. I would suggest that I get rid of the individual pom.xml files and all the packaging and unpacking and change the CMake files to use relative paths in the project structure? In general you are ok with using maven to trigger a native build tool (cmake, python setup.py or dotnet) and to have this build the entire language part of that particular module. I could make the maven plugin work in a way that it can generate code for multiple specs into different directories (think it already allows this anyway) and to run this at the language-root. So a build for any language would be: 1. Generate the code using the plc4x-maven-plugin 2. Execute the native build Beyond that we would need: - Clean by providing extra rules to the maven-clean-plugin to clean up the language dependent directories and resources. - If we want to ship artifacts ... find a way to do so in the package, deploy and install phases (For now this is mostly disabled anyway ... but we should come up with something) Things that are important here is a way to provide the version from maven to the native build tool as otherwise we would have problems releasing. I think I have implemented this at least for dotnet and python ... have to double-check with CMake. I am not focused on using Thrift at all ... it was something you folks came up with and I simply made it work reliably in our build. I do agree that I would prefer not having to build the thrift binary in advance. If you come up with an alternative, I'm happy to replace this. The reason for us building it, was that the Thrift project is not distributing thrift compiler binaries for anything except windows. I could contact them and donate our thrift config to the project and hope they start also distributing other binaries, but for now there was just no way to tun the thrift compiler without manually pre-installing it on every system. We could remove that thrift compilation and extend the pre-flight-check and README documentation with instructions on how to install thrift, but then again the list of things to install is continuously
AW: [BUILDS] What build system to use?
Hello, Chris, I pushed my changes to CMakeLists. Now we have to clarify the following points, where we generate the Boost-Libariers. I'd like to see if I can also map the topic in CMake. If not, we would have to go over maven. If the user creates the boost libraries without support, there may be problems with including and linking again. Best regards Markus Freundliche Grüße Markus Sommer Geschäftsführer isb innovative software businesses GmbH Otto-Lilienthal-Strasse 2 D - 88046 Friedrichshafen Tel.: +49 (0) 7541 3834-14 Mob: +49 (0) 171 537 8437 Fax: +49 (0) 7541 3834-20 E-Mail: som...@isb-fn.de Web: www.isb-fn.de Geschäftsführer: Markus Sommer, Thomas Zeler Sitz: Friedrichshafen Registergericht: Amtsgericht Ulm HRB-Nr. 631624 Important Note: This e-mail and any attachments are confidential, may contain trade secrets and may well also be legally privileged or otherwise protected from disclosure. If you have received it in error, you are on notice of its status. Please notify us immediately by reply e-mail and then delete his e-mail and any attachment from your system. If you are not the intended recipient please understand that you must not copy this e-mail or any attachments or disclose the contents to any other person. Thank you. -Ursprüngliche Nachricht- Von: Christofer Dutz Gesendet: Dienstag, 28. Mai 2019 12:34 An: dev@plc4x.apache.org Betreff: Re: [BUILDS] What build system to use? Hi Markus, that would be great :-) Chris Am 28.05.19, 11:37 schrieb "Markus Sommer" : Hello Chris, in Case of the CMake files I already did some things here. I would push the CMake files to compile the scources on the branch, so you only have to bind the Pom.xml. Best regards Markus Freundliche Grüße Markus Sommer Geschäftsführer isb innovative software businesses GmbH Otto-Lilienthal-Strasse 2 D - 88046 Friedrichshafen Tel.:+49 (0) 7541 3834-14 Mob: +49 (0) 171 537 8437 Fax: +49 (0) 7541 3834-20 E-Mail: som...@isb-fn.de Web: www.isb-fn.de Geschäftsführer: Markus Sommer, Thomas Zeler Sitz: Friedrichshafen Registergericht: Amtsgericht Ulm HRB-Nr. 631624 Important Note: This e-mail and any attachments are confidential, may contain trade secrets and may well also be legally privileged or otherwise protected from disclosure. If you have received it in error, you are on notice of its status. Please notify us immediately by reply e-mail and then delete his e-mail and any attachment from your system. If you are not the intended recipient please understand that you must not copy this e-mail or any attachments or disclose the contents to any other person. Thank you. -Ursprüngliche Nachricht- Von: Christofer Dutz Gesendet: Dienstag, 28. Mai 2019 10:22 An: dev@plc4x.apache.org Betreff: Re: [BUILDS] What build system to use? Hi all, I'll try to summarize responses to all ( So in general you are quite happy as it currently is, however the C++ module could use simplification. I would suggest that I get rid of the individual pom.xml files and all the packaging and unpacking and change the CMake files to use relative paths in the project structure? In general you are ok with using maven to trigger a native build tool (cmake, python setup.py or dotnet) and to have this build the entire language part of that particular module. I could make the maven plugin work in a way that it can generate code for multiple specs into different directories (think it already allows this anyway) and to run this at the language-root. So a build for any language would be: 1. Generate the code using the plc4x-maven-plugin 2. Execute the native build Beyond that we would need: - Clean by providing extra rules to the maven-clean-plugin to clean up the language dependent directories and resources. - If we want to ship artifacts ... find a way to do so in the package, deploy and install phases (For now this is mostly disabled anyway ... but we should come up with something) Things that are important here is a way to provide the version from maven to the native build tool as otherwise we would have problems releasing. I think I have implemented this at least for dotnet and python ... have to double-check with CMake. I am not focused on using Thrift at all ... it was something you folks came up with and I simply made it work reliably in our build. I do agree that I would prefer not having to build the thrift binary in advance. If you come up with an alternative, I'm happy to replace this. The reason for us building it, was that the Thrift project is not distributing thrift compiler binaries for anything except windows. I could contact them and donate our thrift config to the project and hope they start also distributing other
AW: [BUILDS] What build system to use?
Hey everyone, I would also opt for Maven being at least the triggering build system because it also integrates well with Jenkins and the mechanisms behind it are quite understandable. I think the main point we need to solve is to get some kind of documentation which build step in which profile triggers what, with which set of parameters to make it easier for people not working on the native parts to debug stuff. I think Chris Preflight check already adds a lot to this point. I think for .NET we found quite a nice solution because it integrates well. For C++ it is always difficult to get everything working with different architectures and stuff. But I think the current Maven / CMake Integration is as good as it gets. One question that remains open for me is if it is really necessary to build Thrift and all of Boost during the C++ compilation because both take a lot of time and are quite error prone. So on the bottom line +1 for the integrated Maven solution. But I think it is not a real yes/no question but more of a "yes...but" kind of thing. Best Regards Björn -Ursprüngliche Nachricht- Von: Christofer Dutz Gesendet: Montag, 27. Mai 2019 14:42 An: dev@plc4x.apache.org Betreff: [BUILDS] What build system to use? Hi all, as the discussion had come up multiple times now … mainly on Slack, I would like to do a dedicated discussion here on the list about the topic. Up to now we had built the build to generally build every module with maven, utilizing plugins to enable triggering builds with things like CMake. The maven build then took care of packing the results and handling the dependencies (At least this was how I built things for the C++ module) With the plc4net however we’re using an execution of “dotnet” to build a solution. In this case the build is entirely handled by dotnet and all maven does is trigger the build and fail if the execution fails with any non 0 return code. We are currently doing something similar with python: where we’re just calling python to execute the setup.py script. What is your opinion on how we should do our builds? If we go down the path like I initially setup C++, we have the benefit of utilizing the maven ecosystem and we will not have any problems during releases. It however comes with some comfort disadvantages for the “native” guys. But things like code-generation will work nicely. If we go down a non-maven path we will have to deal with all of this ourselves … we will have to come up with a way to handle everything ourselves Or learn how to do it in python and CMake and dotnet, and … The other thing is that we can’t integrate the driver generation as easy as with maven. Either we manually execute the code generation multiple times with maven to generate multiple output directories before executing the individual build system Or we have to build custom generators for every build-system we are using. Even if I like the way I setup the C++ build, I doubt all people will like it as it requires them to do things differently than they are used to. But I would strongly object implementing multiple code-generators. One option would be to split up the plc4x git repo into multiple repos – each one for one language … so we’d have a plc4j, plc4cpp, plc4net and plc4python. Each of these would be dedicated to one particular language and have build tools that fit them. However I strongly object this option, as it would require the Release manager to understand and setup each of these environments. Another option which I think would be a valid compromise, would be to have all languages in one repo the way we currently have it and Maven as triggering build system. The code is generated by maven but the build itself is handled by the particular build system. This will require people working exclusively in VisualStudio or some Python IDE, to manually run a “mvn generate-sources” first, but I think that’s a reasonable restriction. The benefit is that releasing this should be possible with our current release process which has a limited setup cost for the Release Manager. I would opt for the hybrid option with Maven as initiating build system for releases and code generation and CI stuff but to have the native build system for every language to build the individual parts. Namely this would be: * Plc4cpp: CMake * Plc4net: dotnet * Plc4Python: python setup.py Not quite sure how we can get the test-results to be reported correctly. What are your thoughts on this? Chris
AW: [BUILDS] What build system to use?
Hello everybody, it's always a problem to get different pholosphies under one hat. On the part of C++, CMake is not only available in Visual Studio, but also for other platforms. On the CMake side, different parameters can be specified for the build process. In the end, Maven could calculate the different parameters based on the information (system, target, etc.) and call CMake with different parameters. CMake would then generate the different solutions based on the parameters. What the interface looks like then we would just have to discuss, so that you can see it at .Net, Python. For me, the procedure is start and then wait had happened to coarse granular. Best Regards Markus Freundliche Grüße Markus Sommer Geschäftsführer isb innovative software businesses GmbH Otto-Lilienthal-Strasse 2 D - 88046 Friedrichshafen Tel.: +49 (0) 7541 3834-14 Mob: +49 (0) 171 537 8437 Fax: +49 (0) 7541 3834-20 E-Mail: som...@isb-fn.de Web: www.isb-fn.de Geschäftsführer: Markus Sommer, Thomas Zeler Sitz: Friedrichshafen Registergericht: Amtsgericht Ulm HRB-Nr. 631624 Important Note: This e-mail and any attachments are confidential, may contain trade secrets and may well also be legally privileged or otherwise protected from disclosure. If you have received it in error, you are on notice of its status. Please notify us immediately by reply e-mail and then delete his e-mail and any attachment from your system. If you are not the intended recipient please understand that you must not copy this e-mail or any attachments or disclose the contents to any other person. Thank you. -Ursprüngliche Nachricht- Von: Christofer Dutz Gesendet: Montag, 27. Mai 2019 14:42 An: dev@plc4x.apache.org Betreff: [BUILDS] What build system to use? Hi all, as the discussion had come up multiple times now … mainly on Slack, I would like to do a dedicated discussion here on the list about the topic. Up to now we had built the build to generally build every module with maven, utilizing plugins to enable triggering builds with things like CMake. The maven build then took care of packing the results and handling the dependencies (At least this was how I built things for the C++ module) With the plc4net however we’re using an execution of “dotnet” to build a solution. In this case the build is entirely handled by dotnet and all maven does is trigger the build and fail if the execution fails with any non 0 return code. We are currently doing something similar with python: where we’re just calling python to execute the setup.py script. What is your opinion on how we should do our builds? If we go down the path like I initially setup C++, we have the benefit of utilizing the maven ecosystem and we will not have any problems during releases. It however comes with some comfort disadvantages for the “native” guys. But things like code-generation will work nicely. If we go down a non-maven path we will have to deal with all of this ourselves … we will have to come up with a way to handle everything ourselves Or learn how to do it in python and CMake and dotnet, and … The other thing is that we can’t integrate the driver generation as easy as with maven. Either we manually execute the code generation multiple times with maven to generate multiple output directories before executing the individual build system Or we have to build custom generators for every build-system we are using. Even if I like the way I setup the C++ build, I doubt all people will like it as it requires them to do things differently than they are used to. But I would strongly object implementing multiple code-generators. One option would be to split up the plc4x git repo into multiple repos – each one for one language … so we’d have a plc4j, plc4cpp, plc4net and plc4python. Each of these would be dedicated to one particular language and have build tools that fit them. However I strongly object this option, as it would require the Release manager to understand and setup each of these environments. Another option which I think would be a valid compromise, would be to have all languages in one repo the way we currently have it and Maven as triggering build system. The code is generated by maven but the build itself is handled by the particular build system. This will require people working exclusively in VisualStudio or some Python IDE, to manually run a “mvn generate-sources” first, but I think that’s a reasonable restriction. The benefit is that releasing this should be possible with our current release process which has a limited setup cost for the Release Manager. I would opt for the hybrid option with Maven as initiating build system for releases and code generation and CI stuff but to have the native build system for every language to build the individual parts. Namely this would be: * Plc4cpp: CMake * Plc4net: dotnet * Plc4Python: python setup.py Not quite sure how we can get the test-results
