I'have push to Branch feature/s7-cpp
Freundliche Grüße
Markus Sommer
Geschäftsführer
isb innovative software businesses GmbH
Otto-Lilienthal-Strasse 2
D - 88046 Friedrichshafen
Tel.: +49 (0) 7541 3834-14
Mob: +49 (0) 171 537 8437
Fax: +49 (0) 7541 3834-20
E-Mail: som...@isb-fn.de
Web: www.isb-fn.de
Geschäftsführer: Markus Sommer, Thomas Zeler
Sitz: Friedrichshafen
Registergericht: Amtsgericht Ulm HRB-Nr. 631624
Important Note: This e-mail and any attachments are confidential, may contain
trade secrets and may well also be legally privileged or otherwise protected
from disclosure. If you have received it in error, you are on notice of its
status.
Please notify us immediately by reply e-mail and then delete his e-mail and any
attachment from your system. If you are not the intended recipient please
understand that you must not copy this e-mail or any attachments or disclose
the contents to any other person. Thank you.
-----Ursprüngliche Nachricht-----
Von: Christofer Dutz <christofer.d...@c-ware.de>
Gesendet: Dienstag, 28. Mai 2019 12:34
An: dev@plc4x.apache.org
Betreff: Re: [BUILDS] What build system to use?
Hi Markus,
that would be great :-)
Chris
Am 28.05.19, 11:37 schrieb "Markus Sommer" <som...@isb-fn.de>:
Hello Chris,
in Case of the CMake files I already did some things here.
I would push the CMake files to compile the scources on the branch, so you
only have to bind the Pom.xml.
Best regards
Markus
Freundliche Grüße
Markus Sommer
Geschäftsführer
isb innovative software businesses GmbH
Otto-Lilienthal-Strasse 2
D - 88046 Friedrichshafen
Tel.: +49 (0) 7541 3834-14
Mob: +49 (0) 171 537 8437
Fax: +49 (0) 7541 3834-20
E-Mail: som...@isb-fn.de
Web: www.isb-fn.de
Geschäftsführer: Markus Sommer, Thomas Zeler
Sitz: Friedrichshafen
Registergericht: Amtsgericht Ulm HRB-Nr. 631624
Important Note: This e-mail and any attachments are confidential, may
contain trade secrets and may well also be legally privileged or otherwise
protected from disclosure. If you have received it in error, you are on notice
of its status.
Please notify us immediately by reply e-mail and then delete his e-mail and
any attachment from your system. If you are not the intended recipient please
understand that you must not copy this e-mail or any attachments or disclose
the contents to any other person. Thank you.
-----Ursprüngliche Nachricht-----
Von: Christofer Dutz <christofer.d...@c-ware.de>
Gesendet: Dienstag, 28. Mai 2019 10:22
An: dev@plc4x.apache.org
Betreff: Re: [BUILDS] What build system to use?
Hi all,
I'll try to summarize responses to all (
So in general you are quite happy as it currently is, however the C++
module could use simplification. I would suggest that I get rid of the
individual pom.xml files and all the packaging and unpacking and change the
CMake files to use relative paths in the project structure?
In general you are ok with using maven to trigger a native build tool
(cmake, python setup.py or dotnet) and to have this build the entire language
part of that particular module. I could make the maven plugin work in a way
that it can generate code for multiple specs into different directories (think
it already allows this anyway) and to run this at the language-root. So a build
for any language would be:
1. Generate the code using the plc4x-maven-plugin 2. Execute the native
build
Beyond that we would need:
- Clean by providing extra rules to the maven-clean-plugin to clean up the
language dependent directories and resources.
- If we want to ship artifacts ... find a way to do so in the package,
deploy and install phases (For now this is mostly disabled anyway ... but we
should come up with something)
Things that are important here is a way to provide the version from maven
to the native build tool as otherwise we would have problems releasing. I think
I have implemented this at least for dotnet and python ... have to double-check
with CMake.
I am not focused on using Thrift at all ... it was something you folks came
up with and I simply made it work reliably in our build. I do agree that I
would prefer not having to build the thrift binary in advance. If you come up
with an alternative, I'm happy to replace this.
The reason for us building it, was that the Thrift project is not
distributing thrift compiler binaries for anything except windows. I could
contact them and donate our thrift config to the project and hope they start
also distributing other binaries, but for now there was just no way to tun the
thrift compiler without manually pre-installing it on every system. We could
remove that thrift compilation and extend the pre-flight-check and README
documentation with instructions on how to install thrift, but then again the
list of things to install is continuously growing.
I do have some strong Anti-Docker feelings. First of all ... it does
require jumping some extra Hoops on Mac and Windows (Here we need a VM to run
docker in ... this has been dealt with, but I think it could complicate
things). The far worse problem is that the current way of distributing Docker
images is a huge mine-field Apache distribution-policy wise. I wouldn't want to
audit any Docker images to not include category X stuff and I would like to
avoid even visiting this war-field ;-) It's currently sort of in a "we'll do it
and decide what to do policy wise sometime in the future" ...
Regarding the splitting into individual poms ... we have already done this
... if you change into the plc4j directory and run a "mvn install" it will only
build the java part and you won't need any special profiles to be enabled ...
so not quite sure what you need beyond that ... I usually only build the
modules I'm working on and do a full build quite often only before committing.
Regarding documentation ... this is a big to-do I have on my list ...
describing which steps in which order are executed for which reason. Just
didn't have the time to yet. But I'll happily prioritize this up, if it helps.
Ufff ... now I'm also starting to produce monster emails ;-)
Chris
Am 27.05.19, 17:22 schrieb "Dr. Julian Feinauer"
<j.feina...@pragmaticminds.de>:
Hi all,
first, thanks Chris for bringing this up.
I also wanted to start this discussion on the list but thank you very
much for doing it and the nice writeup.
We now have 4 languages (Java, C++, C# and Python) and hopefully more
to come (C would be really nice, I personally think and as some of you may
know, I'm a C master now).
So I think it is a good time now to "loosen" the coupling between the
languages (at least until code-gen comes, then we have to reconsider).
I also have the hope that we will grow our community and have
"subgroups" that take care of these languages.
So, I definitely favor the "native build system" approach which.
On the other hand, I totally agree with Chris, that we have to keep an
eye on keeping things release-able.
There are two general options for that... separate repos and separate
release cycles or, as Chris suggests, some kind of "meta-automation".
I cannot judge (have not enough experience) to see which one of both
approaches is worse, but I think I prefer the latter.
I just checked how Apache Arrow handles this (they also have separate
language bindings, see https://github.com/apache/arrow) and it looks like they
have the latter approach and bind it together via CMake.
To come to a conclusion I think our best bet is to keep it kind of
as-is but do this a bit more explicit. That means,
* every lang has a subfolder and uses a native build tool
* we provide an opportunity to bind all that together to allow for
"joint" builds and releases
But, two other things which are wort notice.
I agree with Björn that we should rethink the current way we handle
thrift, as it feels a bit unnatural for me ( well, that’s well known by now).
One option could be to provide a docker container for that (arrow does
it that way, I think) to perform the code generation.
Second, I suggest to consider splitting maven into to "separate" poms.
This means we use maven as "multi-build-tool" on one hand to
orchestrate the build for us but have a separate java maven build for the maven
subfolder.
I cannot say whether maven is the optimal solution for the
"multi-build-tool", but I guess that’s always hard and our maven integration
currently works pretty nice and flawless so it would be good to keep that.
Perhaps Chris as maven expert can say something about if it is feasible
to do what I suggest and makes any sense?
Julian
Am 27.05.19, 17:03 schrieb "Bjoern Hoeper" <hoe...@ltsoft.de>:
Hey everyone,
I would also opt for Maven being at least the triggering build
system because it also integrates well with Jenkins and the mechanisms behind
it are quite understandable.
I think the main point we need to solve is to get some kind of
documentation which build step in which profile triggers what, with which set
of parameters to make it easier for people not working on the native parts to
debug stuff. I think Chris Preflight check already adds a lot to this point.
I think for .NET we found quite a nice solution because it
integrates well. For C++ it is always difficult to get everything working with
different architectures and stuff. But I think the current Maven / CMake
Integration is as good as it gets.
One question that remains open for me is if it is really necessary
to build Thrift and all of Boost during the C++ compilation because both take a
lot of time and are quite error prone.
So on the bottom line +1 for the integrated Maven solution. But I
think it is not a real yes/no question but more of a "yes...but" kind of thing.
Best Regards
Björn
-----Ursprüngliche Nachricht-----
Von: Christofer Dutz <christofer.d...@c-ware.de>
Gesendet: Montag, 27. Mai 2019 14:42
An: dev@plc4x.apache.org
Betreff: [BUILDS] What build system to use?
Hi all,
as the discussion had come up multiple times now … mainly on Slack,
I would like to do a dedicated discussion here on the list about the topic.
Up to now we had built the build to generally build every module
with maven, utilizing plugins to enable triggering builds with things like
CMake.
The maven build then took care of packing the results and handling
the dependencies (At least this was how I built things for the C++ module) With
the plc4net however we’re using an execution of “dotnet” to build a solution.
In this case the build is entirely handled by dotnet and all maven
does is trigger the build and fail if the execution fails with any non 0 return
code.
We are currently doing something similar with python: where we’re
just calling python to execute the setup.py script.
What is your opinion on how we should do our builds?
If we go down the path like I initially setup C++, we have the
benefit of utilizing the maven ecosystem and we will not have any problems
during releases.
It however comes with some comfort disadvantages for the “native”
guys. But things like code-generation will work nicely.
If we go down a non-maven path we will have to deal with all of
this ourselves … we will have to come up with a way to handle everything
ourselves Or learn how to do it in python and CMake and dotnet, and … The other
thing is that we can’t integrate the driver generation as easy as with maven.
Either we manually execute the code generation multiple times with
maven to generate multiple output directories before executing the individual
build system Or we have to build custom generators for every build-system we
are using.
Even if I like the way I setup the C++ build, I doubt all people
will like it as it requires them to do things differently than they are used to.
But I would strongly object implementing multiple code-generators.
One option would be to split up the plc4x git repo into multiple
repos – each one for one language … so we’d have a plc4j, plc4cpp, plc4net and
plc4python.
Each of these would be dedicated to one particular language and
have build tools that fit them. However I strongly object this option, as it
would require the Release manager to understand and setup each of these
environments.
Another option which I think would be a valid compromise, would be
to have all languages in one repo the way we currently have it and Maven as
triggering build system.
The code is generated by maven but the build itself is handled by
the particular build system.
This will require people working exclusively in VisualStudio or
some Python IDE, to manually run a “mvn generate-sources” first, but I think
that’s a reasonable restriction.
The benefit is that releasing this should be possible with our
current release process which has a limited setup cost for the Release Manager.
I would opt for the hybrid option with Maven as initiating build
system for releases and code generation and CI stuff but to have the native
build system for every language to build the individual parts.
Namely this would be:
* Plc4cpp: CMake
* Plc4net: dotnet
* Plc4Python: python setup.py
Not quite sure how we can get the test-results to be reported
correctly.
What are your thoughts on this?
Chris
