Hi all,

we already had an interesting debate (about version numbers for next release) 
with Kai where he convinced me with his arguments, so I take his view extremely 
I agree with both of you, kind of.
CRUNCH is, as I state it below, a sort of Scraper but a bit cleaner, nice API 
and all that stuff.
So I assume that as soon as we launch it everybody builds its "solution" on top 
of CRUNCH and the scraper could be deprecated / discarded.

The real question is indeed if we think that it has enough applications outside 
plc4x to have it a separate module or not.
But, as our github records show... not yet, at least __

One of the reasons why I'd like to have it in the PLC4X PMC rather than a 
separate (incubating) project is that we already have a community and people 
who care and understand our problems and use cases. And it feels bad to try to 
copy that to another project (and even have interdependencies).

The thing about the renaming is just my personal opinion (and I see that this 
is perhaps pretty controversial). Thus, the matter we currently discuss is 
really only "adopt CRUNCH, rename it and have it as a subproject of PLC4X".


Am 02.09.19, 16:03 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:

    Hi all,
    well I sort of think renaming it to something different would be a really 
bad idea (Toddy also doesn't like the idea)
    And I agree that PLC4X should be true to its project statement to provide 
protocol adapters for the different industrial protocols in multiple languages.
    I personally hate to have these projects that sort of want to address 
everything and I don't see a reason for building the next industrial monolith.
    I have no objections to add sub-projects which defines a processing layer, 
as long as the core remains and the other modules are options to choose from 
(or not choose any)
    Am 02.09.19, 15:46 schrieb "Kai Wähner" <megachu...@gmail.com>:
        *"So in the mid- or longterm I agree that it would be nice to rename 
        to something like "Industrial IoT and Industrie 4.0 we kick your ass" or
        something with subprojects like - CRUNCH as processing pipeline - PLC4X 
        connection - hopefully some more to come"*
        From an external perspective, I am not sure how much value it brings if 
        PLC4X project is "blown up" with several additional components.
        I like the simplicity of it solving a specific problem: "*It is our 
goal to
        create a set of libraries, that allow unified access to any type of 
        PLC4X already integrates with many other "processing frameworks" like
        Kafka, Nifi, Logstash. If you add "processing and hopefully some more"
        components in the future, the project will grow, including its features,
        but also complexity. Not sure if the "Hadoop approach" is the better way
        here, too. You can pick and choose what you need (HDFS, Parquet, Avro,
        Hive, Impala, many more), but they are independent of each other for
        releases, bug fixes, dependencies. This has all its trade-offs, of 
        Personally, I would like to see PLC4X improving in the things it was 
        for, including more protocol implementations (and increases maturity), 
        language support beyond Java, and other (small) enhancements. My point 
        view is not 100% objective as I focus on Kafka, of course.
        But I am not sure if it is counterintuitive to provide features like 
        processing which are already solved by the frameworks PLC4X integrates 
        (Kafka, Nifi, et al). For example, with Kafka Connect and the PLC4X
        connector, you can do data processing (filtering etc) out-of-the-box,
        already (without the need for another engine / app; just use Connect 
        Just my two cents (from a Kafka enthusiast)...
        On Mon, Sep 2, 2019 at 11:25 AM Julian Feinauer <
        j.feina...@pragmaticminds.de> wrote:
        > Hi Matthias,
        > I agree with you view on things.
        > In fact CRUNCH will be something like the "scraper on steroids" as it 
        > tons of other more useful features.
        > So I guess lots of users (Kafka Adapter, ES Adapter, ...) will love 
        > Regarding the Structure, I agree with you. It is not uncommon to have
        > subprojects in a project and thus a PMC is not a project but a comitee
        > managing one or more projects AFAIR.
        > So in the mid- or longterm I agree that it would be nice to rename 
        > to something like "Industrial IoT and Industrie 4.0 we kick your ass" 
        > something with subprojects like
        > - CRUNCH as processing pipeline
        > - PLC4X as connection
        > - hopefully some more to come
        > But first step would be to start with one further subproject which 
        > be crunch.
        > Julian
        > Am 29.08.19, 14:47 schrieb "Strljic, Matthias Milan" <
        > matthias.strl...@isw.uni-stuttgart.de>:
        >     Hi all,
        >     Is there a way to encapsulate all components in the project in 
kind of
        > sub projects?
        >     Because we will have at the end PLC4X which is a "lib"-attempt of 
        > jdbc-Driver for PLC in different languages.
        >     This uses a separated project which has a template-based code
        > generation to built efficient the de-/serialization layer.
        >     On top of PLC4X we have different sub-projects like the scraper,
        > Connectors, Bridge-Servers,  platform independent Thrift-Server... 
        >     But those are not really modules of PLC4X, they are more modules
        > specific for PLC4J and will stay those modules, or?
        >     Because I would like to see CRUNCH as a more advanced Scraper
        > extension. If the code and doc quality is ok. But now it is also for 
me not
        > that clear if I should vote for it or not. Then in my opinion is not
        > directly a part of "PLC4X the JDBC driver API"-thing. As the Scraper,
        > Thrift, code gen and the connectors it is more a nice tool but it is
        > directly connected with our community and our PLC4X eco system.
        >     So is there a "Apache way" to manage/advertise/maintain some of 
        > elements as a kind of sub project?
        >     Sry if I sound there I bit silly. I have there not that much
        > experience in managing OS projects.
        >     Greetings
        >     Matthias Strljic, M.Sc.
        >     Universität Stuttgart
        >     Institut für Steuerungstechnik der Werkzeugmaschinen und
        > Fertigungseinrichtungen (ISW)
        >     Seidenstraße 36
        >     70174 Stuttgart
        >     GERMANY
        >     Tel: +49 711 685-84530
        >     Fax: +49 711 685-74530
        >     E-Mail: matthias.strl...@isw.uni-stuttgart.de
        >     Web: http://www.isw.uni-stuttgart.de
        >     -----Ursprüngliche Nachricht-----
        >     Von: Julian Feinauer <j.feina...@pragmaticminds.de>
        >     Gesendet: Saturday, August 24, 2019 6:23 PM
        >     An: dev@plc4x.apache.org
        >     Betreff: AW: [DISCUSS] Integrate CRUNCH into PLC4X
        >     Hi Otto,
        >     Thanks for your comment.
        >     Indeed I agree with all of your points and I think this is also 
        > reason why I'd like to integrate it.
        >     It's better to develop software in such a community where you
        > challenge each other.
        >     So if others see it also positive I agree to decide on a todo 
        >     Julian
        >     Von meinem Mobiltelefon gesendet
        >     -------- Ursprüngliche Nachricht --------
        >     Betreff: Re: [DISCUSS] Integrate CRUNCH into PLC4X
        >     Von: Otto Fowler
        >     An: dev@plc4x.apache.org
        >     Cc:
        >     From the peanut gallery this seems like a great idea.
        >     I would say though, that this is a nice chuck of code to bring in,
        > without any documentation. Which means that it would be difficult to 
        > the
        >     *project* support it vs. Julian.
        >     I would suggest that there be some criteria before inclusion in 
        > project
        >     wrt:
        >        - design documentation
        >        - developer documentation
        >        - current outstanding issues ( either addressing or moving 
them to
        > jira )
        >        - possibly some review activity, that could generate issues 
        > either
        >        need to be addressed or would end up in jira
        >     On August 23, 2019 at 11:59:19, Julian Feinauer (
        >     j.feina...@pragmaticminds.de) wrote:
        >     Hi Chris,
        >     regarding separate repo and release cycle I agree, and I think ist
        > reasonable to see it as subproject.
        >     And regarding Java Only... I guess so (it currently is).
        >     We also considered providing a JDBC interface, so that one can 
        > from every other language as well but I'm unsure about concrete use 
        > in the PLC4X area.
        >     Julian
        >     Am 23.08.19, 17:53 schrieb "Christofer Dutz" <
        > christofer.d...@c-ware.de>:
        >     Hi Julian,
        >     I like the idea. I know we discussed reviving Edgent and merging 
        > two projects. But for me the project is dead and I also wouldn't like 
        > restart building a new community, even if we could probably populate 
        > with plc4x pmcs.
        >     The donation world probably have to go through the incubator ip
        > clearance, but I don't have any objections.
        >     Only id prefer it as a separate subproject in plc4x with a 
        > repo and release cycle. I would assume this layer would be a Java only
        > layer, correct?
        >     Chris
        >     Holen Sie sich Outlook für Android<https://aka.ms/ghei36>
        >     ________________________________
        >     From: Julian Feinauer <j.feina...@pragmaticminds.de>
        >     Sent: Friday, August 23, 2019 5:30:11 PM
        >     To: dev@plc4x.apache.org <dev@plc4x.apache.org>
        >     Subject: [DISCUSS] Integrate CRUNCH into PLC4X
        >     Hi all,
        >     I wanted to start a discussion about a Topic that was already
        > discussed several times between several people but never here on this 
        >     As some of you may know, we at pragmatic minds developed a 
        > for “Industrial Data” Evaluation we called CRUNCH.
        >     As we were unable to sell it back than (I think this stays true 
        > today) we decided to make it open source under Apache 2 License, here 
        >     What we observed is that such a framework doesn’t have much use
        > without data (clever, hm?).
        >     On the other hand, everybody who used PLC4X a bit comes sooner or
        > later at the point where he needs to transform data in a certain way.
        >     Imagine simple situations like… watching a bit, because you want 
        > know every time it toggles.
        >     But more and more complex examples are imagineable like reading
        > certain values only when a (probably complex) condition becomes true 
and so
        > on and so on.
        >     Exactly these issues are addressed by CRUNCH with a very fluent 
        > API but we also planned (and still plan) to introduce some kind of 
        > language to enable to evaluate / start queries during runtime.
        >     For several reasons (some outlined above) I see that both projects
        > could benefit a lot from each other.
        >     Another aspect is that our community is really great and I 
dislike the
        > idea of having several different communities around for (closely) 
        > project as this always leads to “brain split”.
        >     So we at pragmatic minds could imagine to donate the CRUNCH code 
        > the PLC4X project.
        >     But before taking any ‘official’ steps I like to discuss the topic
        > with you and hear your thoughts on that.
        >     Julian
        >     PS.: Of course we would rename the framework. CRUNCH was and is a 
        > name and we knew that from the start but never found a better one… :D
        >     [1] https://github.com/pragmaticminds/crunch

Reply via email to