Hey,
I am also glad to join the team and I am really happy that the discussion took
off this well.
I would +1 Julians Proposal for a personal meeting at due time for discussing
the code generation. Something that came to my mind was to join this meeting
with the Apache Edgent Meetup you mentioned at the conference (6th May if I
remember this correctly). Would this be an option?
Regarding the C# API I can work out a draft with just classes and interfaces.
I have a bit of experience using CMake with .NET so I could also work out a way
for integrating the C# parts into the build.
My ultimate goal would be to have the .NET Libraries compatible to .NET
Standard 2.0 so that it is usable with .NET Core and .NET Full. One implication
of this would be that we need to think about the exact driver integration
(Native code gets a little bit harder but is possible; but I would prefer a
full .NET based solution).
Best Regards
Björn
-----Ursprüngliche Nachricht-----
Von: Julian Feinauer <j.feina...@pragmaticminds.de>
Gesendet: Mittwoch, 10. April 2019 09:21
An: dev@plc4x.apache.org
Betreff: Re: Integration of other Languages / Daffodil
Hi,
I did not intend to hurry you in any way, chris : ) You are the one to set the
next steps... I just wanted to bring the topic back (after all that paperwork).
And after your statements here, I would even more like to have a meetup where
you present this a bit more detailed.
What do you think?
Perhaps in a month or so?
Julian
PS.: With the growing community, we should consider to do this perhaps more
regularly, or?
Am 10.04.19, 09:15 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:
Hi Julian,
I agree with bringing this topic more back to the core ... as you can
imagine it feels like I have not been working on this to
100% as about 90% of my time and energy was consumed with pre-graduation
work. I think this will be over about 08:45 CET tomorrow ;-)
I guess I still will need a few more days to get back in the saddle and
finish a first fully working draft of the code-generator.
Currently it generates empty java class files ... guess I'll still have to
do some tweaking and adjusting, but should not be too much work.
Till that I doubt working on templates makes too much sense as I think it
would take more time to do the transfer needed to
join in, than I would need to finish it. Of course would I document
everything, but I didn't up to now as I was building, tearing apart,
rebuilding, refactoring, ... didn't want to keep the documentation up to
date.
Also that will currently generated the model classes as well as the
serializers and de-serializers ... after that I need to get started on the
Driver logic (State-Machine) itself. But I guess having the class
generation for the messages would be a great first start.
But what we really need, is an API module (which is not generated) with a
beautiful C# API module. Then integrating that into
Our build will also be a manageable challenge (As I mentioned CMake does
seem to support C# and .NET builds) ... would be great
if someone could work on that.
I guess till I've finished a first working prototype of the model
generation, we could then start porting these templates to other languages.
In parallel me and some people interested, could work on the logic
generation (with java templates) and as soon as that's done,
Hopefully the language templates for the model are done, we could create
the logic templates for the other languages ...
You think that's a good idea and approach?
Chris
Am 10.04.19, 08:53 schrieb "Julian Feinauer" <j.feina...@pragmaticminds.de>:
Hi Björn,
I'm very happy to see you here on the list!
Currently, this "generated driver" topic is mostly on chris shoulders.
This is fine, as he really pushes things forward and has all the
experience and tools he needs to make this work.
BUT I think it’s the time now, to make this more of a "Top Level
Concern" to get others integrated in these efforts.
I think this is something absolutely NECESSSARY for the project, so we
should make it a central concern and join efforts on the next steps.
Especially having everything central in a repo (perhaps simly in a
different "develop" branch or so).
I guess, the next steps would be to
1. Add some documentation or Confluence on all that’s been done
previously and the workings of Chris approach
2. Play around to make templates for another language
3. Make thinks stable, add test suites, ...
So I could imagine (Chris, of course, feel free to hit the brakes) that
you could start by prototyping a S7 driver which is strongly oriented at the
(generated) Java one which can then be used to extract the freemarker templates
from it. This would be the optimal way towards C# integration, I think. All the
tooling and build automation would be a 2nd level concern for me.
Or perhaps you could join forces with Markus (Sommer) which already has
a lot of Cpp and a generated Cpp driver could also have a C# frontend as a
first step before we do all on the .NET Core bases.
What do you think of a dedicated Workshop or Hackday for this topic?
We learned a lot during our Hackday here in Nürtingen and for this
topic I would suggest to have a bit of a stronger Agenda.
But this could be a good kickoff to
1. get all (new) participants of the list on board
2. get a live introduction by Chris
3. Drink beer together
As we are no longer South Germany centered we could to this somewhere
more centrally located.
What are others thoughts on this?
Julian
Am 10.04.19, 04:23 schrieb "Christofer Dutz" <cd...@apache.org>:
Hi Björn and others,
glad you made it here ... hopefully email services will be back to
normal soon ...
Let me please explain in a little more detail, what I had in mind
with all this DSDL and generated drivers.
We are currently in the situation where all drivers are written in
Java and all drivers are manually written. As for example the ADS driver is
written by someone else than the S7 driver, you can see huge differences
between the way they are built. Also, being the main author of the S7 driver I
wasn't quite happy with how I did it and wanted to clean it up greatly as I
knew, porting this would not be easy.
On the other side we are currently working on C++ versions of the
drivers. Just thinking about the manual effort and what could go wrong by
manually porting the drivers and keeping them in sync, just gave me the
shivers. So an alternate approach had to come. I guess we all agree that we
have to automate most of the heavy lifting.
I am not a fan of any cross-compilation (Write drivers in Java and
have them automagically ported to C++, C#, Go ...) so I favor specifying the
protocols in a machine readable way and have all code generated from that
specification.
For this I evaluated about 7 different approaches and wasn't quite
happy with them, till I tried out DFDL which I first heard from at ApacheCon NA
2018 in Montreal. And it turned out to be the ideal format for this. However
with DFDL you can only specify the message format and nothing else, so I was
still missing the state-machine part ... DFDL being XML, it would be ideal if
this format too was in XML so I could embed DFDL into that ... so a simple
google query "state machine xml" directly guided me to SCXML and it turned out
that a SCXML engine is part of Apache Commons project.
So now I had to prove that it is possible to define an industrial
protocol completely with DFDL and SCXML. The result of this is the
"dynamic-driver". The performance is not acceptable (50-100ms for reading one
address), also is it not able to du multiple concurrent requests at the same
time. So we need generated code to bring the normal PLC4X performance to
dynamic drivers. DFDL and SCXML both don't have any code generation till now
and I bet this will remain that way for quite some time. So I started writing a
Maven plugin to use DFDL and Freemarker to generate source files. In the ideal
case we would require two artifacts as input: protocol-specification,
language-templates to generate a new driver for a given language.
Implementing the full DFDL spec is totally out of scope for now, so
I am currently working on a set of conventions we can use to write our
protocol-specs which the generator will be able to process ... this will be a
small sub-set of the entire DFDL spec, but it should be enough to achieve our
goal ... and it does make writing specs for new protocols easier as the author
has a reduced set of options to choose from.
The one part we'll not be able to generate is the API for a new
language. This will always be hand-written and that's a good thing. Cause an
API designed for Java doesn't look the same way an API written for C# or C++
will look. For example in Java we use CompletableFutures and Optionals ... in
other languages this might look different. So ideally every API module for a
language will adopt the ideal patterns for that language it is designed for. So
that's where you come in - Björn or others - I don't know how such an API would
look like in C#. So it would be super-great if you could whip up a small API
module for the Language of your choosing.
The next step then would be for us to somehow integrate that
modules build into the PLC4X build system.
This is usually quite challenging ... but I have read that CMake
does have options for this ... but we'll work that our as soon as it comes up.
The step after that would be to manually port a driver from one
existing language to C# (ideally the generated classes) and create templates
from that code ... or directly start writing templates, however I think that's
more challenging as you don't have content assist of your IDE in Freemarker (I
think).
I hope that explains things and I really hope this email goes
through (better save it first)
Chris
On 2019/04/08 13:44:35, Bjoern Hoeper <hoe...@ltsoft.de> wrote:
> Hi everyone,
> I got in touch with Chris and Julian during the buildingIoT
Conference in Cologne and I would like to participate in the further
development of PLC4X. Our main interest is to extend PLC4X to make it usable
with .NET / C#.
>
> So after looking at the sources and the discussions in the
mailing list I was wondering what would be the best move forward regarding the
integration of other languages in general and the decoupling of the protocol
definition from the actual API implementation in a particular language. As far
as I understand the current status Apache Daffodil is a candidate technology
for generating the adapter code. But at the current moment Daffodil does not
support templating of classes. Furthermore there are existing APIs for Java and
Scala but no implementation for languages not executed on the JVM so we would
need to have an implementation of the Daffodil APIs in every language we want
to support in the future.
> Are there any Ideas existing already about an Architecture to
support languages besides Java? If not I would like to encourage a discussion
about this topic because I think it will get quite fundamental as soon as the
adoption of further languages gains some traction.
> Best Regards
> Björn
>
> =================================
> Dipl.-Ing. B. Höper
> Geschäftsführer
>
> LTSoft - Agentur für Leittechnik-Software GmbH
> Veilchenweg 37a
> 51107 Köln
>
> Telefon: +49 (0) 221 - 79 00 35 31
> Telefax: +49 (0) 221 - 79 00 35 35
> Mobil: +49 (0) 173 - 28 36 904
> Email: hoe...@ltsoft.de<mailto:hoe...@ltsoft.de>
> =================================
>
>
