Hi all,
we already had an interesting debate (about version numbers for next release)
with Kai where he convinced me with his arguments, so I take his view extremely
serios.
I agree with both of you, kind of.
CRUNCH is, as I state it below, a sort of Scraper but a bit cleaner, nice API
and all that stuff.
So I assume that as soon as we launch it everybody builds its "solution" on top
of CRUNCH and the scraper could be deprecated / discarded.
The real question is indeed if we think that it has enough applications outside
plc4x to have it a separate module or not.
But, as our github records show... not yet, at least __
One of the reasons why I'd like to have it in the PLC4X PMC rather than a
separate (incubating) project is that we already have a community and people
who care and understand our problems and use cases. And it feels bad to try to
copy that to another project (and even have interdependencies).
The thing about the renaming is just my personal opinion (and I see that this
is perhaps pretty controversial). Thus, the matter we currently discuss is
really only "adopt CRUNCH, rename it and have it as a subproject of PLC4X".
Julian
Am 02.09.19, 16:03 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:
Hi all,
well I sort of think renaming it to something different would be a really
bad idea (Toddy also doesn't like the idea)
And I agree that PLC4X should be true to its project statement to provide
protocol adapters for the different industrial protocols in multiple languages.
I personally hate to have these projects that sort of want to address
everything and I don't see a reason for building the next industrial monolith.
I have no objections to add sub-projects which defines a processing layer,
as long as the core remains and the other modules are options to choose from
(or not choose any)
Chris
Am 02.09.19, 15:46 schrieb "Kai Wähner" <megachu...@gmail.com>:
*"So in the mid- or longterm I agree that it would be nice to rename
PLC4X
to something like "Industrial IoT and Industrie 4.0 we kick your ass" or
something with subprojects like - CRUNCH as processing pipeline - PLC4X
as
connection - hopefully some more to come"*
From an external perspective, I am not sure how much value it brings if
the
PLC4X project is "blown up" with several additional components.
I like the simplicity of it solving a specific problem: "*It is our
goal to
create a set of libraries, that allow unified access to any type of
PLC*".
PLC4X already integrates with many other "processing frameworks" like
Kafka, Nifi, Logstash. If you add "processing and hopefully some more"
components in the future, the project will grow, including its features,
but also complexity. Not sure if the "Hadoop approach" is the better way
here, too. You can pick and choose what you need (HDFS, Parquet, Avro,
Hive, Impala, many more), but they are independent of each other for
releases, bug fixes, dependencies. This has all its trade-offs, of
course.
Personally, I would like to see PLC4X improving in the things it was
build
for, including more protocol implementations (and increases maturity),
more
language support beyond Java, and other (small) enhancements. My point
of
view is not 100% objective as I focus on Kafka, of course.
But I am not sure if it is counterintuitive to provide features like
data
processing which are already solved by the frameworks PLC4X integrates
with
(Kafka, Nifi, et al). For example, with Kafka Connect and the PLC4X
connector, you can do data processing (filtering etc) out-of-the-box,
already (without the need for another engine / app; just use Connect
SMT,
https://docs.confluent.io/current/connect/transforms/index.html).
Just my two cents (from a Kafka enthusiast)...
On Mon, Sep 2, 2019 at 11:25 AM Julian Feinauer <
j.feina...@pragmaticminds.de> wrote:
> Hi Matthias,
>
> I agree with you view on things.
> In fact CRUNCH will be something like the "scraper on steroids" as it
has
> tons of other more useful features.
> So I guess lots of users (Kafka Adapter, ES Adapter, ...) will love
it.
>
> Regarding the Structure, I agree with you. It is not uncommon to have
> subprojects in a project and thus a PMC is not a project but a comitee
> managing one or more projects AFAIR.
> So in the mid- or longterm I agree that it would be nice to rename
PLC4X
> to something like "Industrial IoT and Industrie 4.0 we kick your ass"
or
> something with subprojects like
> - CRUNCH as processing pipeline
> - PLC4X as connection
> - hopefully some more to come
>
> But first step would be to start with one further subproject which
would
> be crunch.
>
> Julian
>
> Am 29.08.19, 14:47 schrieb "Strljic, Matthias Milan" <
> matthias.strl...@isw.uni-stuttgart.de>:
>
> Hi all,
>
> Is there a way to encapsulate all components in the project in
kind of
> sub projects?
> Because we will have at the end PLC4X which is a "lib"-attempt of
a
> jdbc-Driver for PLC in different languages.
> This uses a separated project which has a template-based code
> generation to built efficient the de-/serialization layer.
> On top of PLC4X we have different sub-projects like the scraper,
> Connectors, Bridge-Servers, platform independent Thrift-Server...
etc.
> But those are not really modules of PLC4X, they are more modules
> specific for PLC4J and will stay those modules, or?
>
> Because I would like to see CRUNCH as a more advanced Scraper
> extension. If the code and doc quality is ok. But now it is also for
me not
> that clear if I should vote for it or not. Then in my opinion is not
> directly a part of "PLC4X the JDBC driver API"-thing. As the Scraper,
> Thrift, code gen and the connectors it is more a nice tool but it is
> directly connected with our community and our PLC4X eco system.
> So is there a "Apache way" to manage/advertise/maintain some of
those
> elements as a kind of sub project?
>
> Sry if I sound there I bit silly. I have there not that much
> experience in managing OS projects.
>
> Greetings
> Matthias Strljic, M.Sc.
>
> Universität Stuttgart
> Institut für Steuerungstechnik der Werkzeugmaschinen und
> Fertigungseinrichtungen (ISW)
>
> Seidenstraße 36
> 70174 Stuttgart
> GERMANY
>
> Tel: +49 711 685-84530
> Fax: +49 711 685-74530
>
> E-Mail: matthias.strl...@isw.uni-stuttgart.de
> Web: http://www.isw.uni-stuttgart.de
>
> -----Ursprüngliche Nachricht-----
> Von: Julian Feinauer <j.feina...@pragmaticminds.de>
> Gesendet: Saturday, August 24, 2019 6:23 PM
> An: dev@plc4x.apache.org
> Betreff: AW: [DISCUSS] Integrate CRUNCH into PLC4X
>
> Hi Otto,
>
> Thanks for your comment.
> Indeed I agree with all of your points and I think this is also
one
> reason why I'd like to integrate it.
> It's better to develop software in such a community where you
> challenge each other.
>
> So if others see it also positive I agree to decide on a todo
list.
>
> Julian
>
> Von meinem Mobiltelefon gesendet
>
>
> -------- Ursprüngliche Nachricht --------
> Betreff: Re: [DISCUSS] Integrate CRUNCH into PLC4X
> Von: Otto Fowler
> An: dev@plc4x.apache.org
> Cc:
>
> From the peanut gallery this seems like a great idea.
>
> I would say though, that this is a nice chuck of code to bring in,
> without any documentation. Which means that it would be difficult to
have
> the
> *project* support it vs. Julian.
>
> I would suggest that there be some criteria before inclusion in
the
> project
> wrt:
>
> - design documentation
> - developer documentation
> - current outstanding issues ( either addressing or moving
them to
> jira )
> - possibly some review activity, that could generate issues
that
> either
> need to be addressed or would end up in jira
>
>
>
>
> On August 23, 2019 at 11:59:19, Julian Feinauer (
> j.feina...@pragmaticminds.de) wrote:
>
> Hi Chris,
>
> regarding separate repo and release cycle I agree, and I think ist
> reasonable to see it as subproject.
> And regarding Java Only... I guess so (it currently is).
>
> We also considered providing a JDBC interface, so that one can
query
> from every other language as well but I'm unsure about concrete use
cases
> in the PLC4X area.
>
> Julian
>
> Am 23.08.19, 17:53 schrieb "Christofer Dutz" <
> christofer.d...@c-ware.de>:
>
> Hi Julian,
>
> I like the idea. I know we discussed reviving Edgent and merging
the
> two projects. But for me the project is dead and I also wouldn't like
to
> restart building a new community, even if we could probably populate
it
> with plc4x pmcs.
>
> The donation world probably have to go through the incubator ip
> clearance, but I don't have any objections.
>
> Only id prefer it as a separate subproject in plc4x with a
separate
> repo and release cycle. I would assume this layer would be a Java only
> layer, correct?
>
> Chris
>
> Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
>
> ________________________________
> From: Julian Feinauer <j.feina...@pragmaticminds.de>
> Sent: Friday, August 23, 2019 5:30:11 PM
> To: dev@plc4x.apache.org <dev@plc4x.apache.org>
> Subject: [DISCUSS] Integrate CRUNCH into PLC4X
>
> Hi all,
>
> I wanted to start a discussion about a Topic that was already
> discussed several times between several people but never here on this
list.
> As some of you may know, we at pragmatic minds developed a
Framework
> for “Industrial Data” Evaluation we called CRUNCH.
> As we were unable to sell it back than (I think this stays true
till
> today) we decided to make it open source under Apache 2 License, here
[1].
>
> What we observed is that such a framework doesn’t have much use
> without data (clever, hm?).
> On the other hand, everybody who used PLC4X a bit comes sooner or
> later at the point where he needs to transform data in a certain way.
> Imagine simple situations like… watching a bit, because you want
to
> know every time it toggles.
> But more and more complex examples are imagineable like reading
> certain values only when a (probably complex) condition becomes true
and so
> on and so on.
> Exactly these issues are addressed by CRUNCH with a very fluent
Java
> API but we also planned (and still plan) to introduce some kind of
query
> language to enable to evaluate / start queries during runtime.
>
> For several reasons (some outlined above) I see that both projects
> could benefit a lot from each other.
> Another aspect is that our community is really great and I
dislike the
> idea of having several different communities around for (closely)
related
> project as this always leads to “brain split”.
>
> So we at pragmatic minds could imagine to donate the CRUNCH code
to
> the PLC4X project.
>
> But before taking any ‘official’ steps I like to discuss the topic
> with you and hear your thoughts on that.
>
> Julian
>
> PS.: Of course we would rename the framework. CRUNCH was and is a
bad
> name and we knew that from the start but never found a better one… :D
>
> [1] https://github.com/pragmaticminds/crunch
>
>
>
