[GitHub] [incubator-streampipes-extensions] tenthe merged pull request #22: Create an adapter for Modbus protocol

[GitBox]

tenthe merged pull request #22:
URL: https://github.com/apache/incubator-streampipes-extensions/pull/22


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-streampipes-extensions] tenthe merged pull request #21: Fix in ValidateTable for MySQL Sink

[GitBox]

tenthe merged pull request #21:
URL: https://github.com/apache/incubator-streampipes-extensions/pull/21


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-streampipes-extensions] tenthe merged pull request #20: [STREAMPIPES-151] Add Cron window support to RateLimit PE

[GitBox]

tenthe merged pull request #20:
URL: https://github.com/apache/incubator-streampipes-extensions/pull/20


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Switching to the standard disclaimer?

[Dominik Riemer]

Hi again,

a question related to our next release:
For the last release, we used the Work-in-progress/WIP-Disclaimer [1] from
the Apache Incubator, which makes releasing a bit easier as releases do not
need to be fully compliant with ASF requirements.

In the release vote, we got some comments on things that do not fully comply
with the ASF release policy and most of them are already fixed (see Jira
issues in [2,3]).

We can switch to the standard disclaimer at any time, which potentially
increases the risk for getting -1 release votes, but helps to learn about
ASF policy requirements on releases. Before graduation, we need to show that
we're able to make fully-ASF-compliant releases anyways.

What do you think? Should we replace the WIP disclaimer [4] with the
standard disclaimer in our Github repos?

Dominik


[1]
https://incubator.apache.org/policy/incubation.html#work_in_progress_disclai
mer
[2] https://issues.apache.org/jira/browse/STREAMPIPES-137
[3] https://issues.apache.org/jira/browse/STREAMPIPES-138
[4] https://github.com/apache/incubator-streampipes/blob/dev/DISCLAIMER-WIP

Core-UI serialization

[Dominik Riemer]

Hi all,

 

during the last days, while starting to migrate the editor UI module to
Angular, I did some experiments with different serialization approaches.

 

Currently, we use two different serialization approaches in the UI:

1.  JSON for most things and the editor module also uses JSON to
represent static properties (which are used to render the UI components in
the pipeline element configuration)
2.  The (newer) Connect module uses JSON-LD (which is also used in the
core to represent pipeline elements) to represent static properties.

 

Both approaches have advantages and disadvantages:

For (1), we use GSON for serializing JSON, which is easy and fast for most
cases, but there is no direct mapping to TypeScript classes for complex
objects (like static property definitions with polymorphism), so that we
rely on the  type here in most cases (especially in the editor module)
-> all type information is lost and UI development is more error-prone.

For (2), we use a self-made JSON-LD serializer named Tson-LD. This works
great, but TypeScript-classes need to be manually modified every time
classes in streampipes-core change, which increases maintenance effort and
also is rather error-prone. On the other hand, Tson-LD deserializes JSON-LD
into TypeScript classes, which is quite convenient from a developer
perspective.

 

As we will soon be able to completely remove AngularJS from the UI, it also
makes sense to rethink and streamline our serialization approach (mainly to
have a single component that cares about generating UI components from
static properties, both in Connect and the Pipeline Editor).

 

So after doing some research and testing I'd propose the following approach:

*   Use only JSON as a serialization format for UI-core communication
and remove JSON-LD from the UI
*   Use Jackson over GSON as a serialization library, as this better
works with the next point:
*   Use typescript-generator [1] to auto-generate TypeScript classes
from Java models. There is not much configuration needed and there is a
Maven plugin which could automatically run the code generation at every
build.

 

This approach would have less overhead, guarantees that all models are
always up-to-date and reduces maintenance effort. Drawback is that quite a
few UI modules would need to be changed.

 

I have already a prototype running which shows that the approach works quite
well. But before I continue, I'm interested whether you think this is a good
or bad idea and if you have any other best practice approaches?

 

Dominik

 

[1]  
https://github.com/vojtechhabarta/typescript-generator

 

 

[jira] [Created] (STREAMPIPES-156) Processor: State Labeler

[Philipp Zehnder (Jira)]

Philipp Zehnder created STREAMPIPES-156:
---

 Summary: Processor: State Labeler
 Key: STREAMPIPES-156
 URL: https://issues.apache.org/jira/browse/STREAMPIPES-156
 Project: StreamPipes
  Issue Type: Improvement
Reporter: Philipp Zehnder
Assignee: Philipp Zehnder


Gets an array of numbers as an input, calculates some statistics on this array. 
Adds a label, according to a user defined rule.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

RE: Migrating the pipeline editor to Angular

[Dominik Riemer]

Hi Grainier,

that's totally correct - I knew I had this in mind when implementing 
Alternatives but obviously conditional fields are not supported right now.
I'll implement this as soon as the editor is migrated!

Dominik

-Original Message-
From: Grainier Perera  
Sent: Monday, June 1, 2020 3:07 PM
To: dev@streampipes.apache.org
Subject: Re: Migrating the pipeline editor to Angular

Hi Dominik,

Thank you for considering those improvements and creating an issue to track 
them. When it comes to conditional fields, I couldn't find a way to make 
following an AlternativeStaticProperty. For example, I need the user to have a 
toggle to select whether to enable group-by or not and only if he/she chose to 
have group-by, then show the field selection. I couldn't find it, but is there 
a way to achieve that?

.requiredStream(StreamRequirementsBuilder.create()
> 
> .requiredPropertyWithUnaryMapping(EpRequirements.anyProperty(),Labels.
> withId(GROUPING_FIELD),
> PropertyScope.NONE)
> .build())


Grainier Perera.


On Mon, 1 Jun 2020 at 02:53, Dominik Riemer  wrote:

> Ok, I've just seen that you already used this in the rate limit 
> component
> 
>
> So just ignore what I wrote in the second paragraph 
> Dominik
>
> -Original Message-
> From: Dominik Riemer 
> Sent: Sunday, May 31, 2020 11:19 PM
> To: dev@streampipes.apache.org
> Subject: RE: Migrating the pipeline editor to Angular
>
> Hi Grainier,
>
> yes, that would be nice to have and one of the things that will be 
> easier to implement after migration.
> I created a JIRA for tracking ideas for that kind of improvements, 
> feel free to add more if you have any ideas!
> https://issues.apache.org/jira/browse/STREAMPIPES-150
>
> There is currently a feature to create simple conditional fields 
> called
> AlternativeStaticProperty:
>
> https://github.com/apache/incubator-streampipes-examples/blob/dev/stre
> ampipes-pipeline-elements-examples-processors-jvm/src/main/java/org/ap
> ache/streampipes/pe/examples/jvm/staticproperty/StaticPropertyAlternat
> ivesController.java However, the API is currently quite verbose and 
> I'd also like to simplify this.
>
> Dominik
>
> -Original Message-
> From: Grainier Perera 
> Sent: Thursday, May 28, 2020 4:16 PM
> To: dev@streampipes.apache.org
> Subject: Re: Migrating the pipeline editor to Angular
>
> Hi Dominik,
>
> This is awesome. Also, I'd like to see some optional fields and 
> conditional fields (i.e different input fields depending on a toggle) 
> with user inputs to PEs. I don't know whether this is the correct 
> thread to ask, but I feel like it's nice to have.
>
> Grainier Perera.
>
>
> On Wed, 27 May 2020 at 22:46, Dominik Riemer  wrote:
>
> > Hi all,
> >
> >
> >
> > I’ve created an issue [1] to migrate the pipeline editor module in 
> > the UI from AngularJS to Angular.
> >
> >
> >
> > A little background:
> >
> > We are currently running AngularJS and Angular in parallel and have 
> > started to migrate old AngularJS modules component-by-component. The 
> > only modules left that are still implemented in AngularJS are the 
> > pipeline editor, the pipeline overview and the pipeline element 
> > installation modules. I’d personally prefer to abandon AngularJS 
> > sooner than later as it complicates the development of some UI-wide 
> > features (e.g., better user and access management) and hinders reuse 
> > of components. So I think I will start migrating the pipeline editor 
> > now and maybe we can get this finished before the next release.
> >
> >
> >
> > So as this implies a rather major refactoring anyways, I’m 
> > interested if you see any things in the UI of the pipeline editor 
> > that could be improved or things you are missing?
> >
> > Any thoughts and ideas are welcome 
> >
> >
> >
> > Dominik
> >
> >
> >
> > [1] https://issues.apache.org/jira/browse/STREAMPIPES-145
> >
> >
>
>
>

Re: Podling Streampipes Report Reminder - June 2020

[Patrick Wiener]

alright - then I add it to the Incubator Wiki.

Patrick

> Am 02.06.2020 um 19:36 schrieb Dominik Riemer :
> 
> Hi Patrick,
> 
> sounds good, I don't have anything to add!
> 
> Dominik
> 
> -Original Message-
> From: Patrick Wiener  
> Sent: Monday, June 1, 2020 11:06 PM
> To: dev@streampipes.apache.org
> Subject: Re: Podling Streampipes Report Reminder - June 2020
> 
> Hi all,
> 
> I prepared a first draft for the upcoming report (due on Wed, June 03).
> 
> It would be good if you could check the report and provide me with some 
> feedback:
> 
> ## StreamPipes
> 
>  StreamPipes is a self-service (Industrial) IoT toolbox to enable non-
>  technical users to connect, analyze and explore (Industrial) IoT data
>  streams.
> 
>  StreamPipes has been incubating since 2019-11-11.
> 
> ### Three most important unfinished issues to address before graduating:
> 
>  1. Multiple Apache Releases
>  2. Grow the developer community
>  3. Attract more users
> 
> ### Are there any issues that the IPMC or ASF Board need to be aware of?
> 
>  There are no issues right now.
> 
> ### How has the community developed since the last report?
> 
>  In the past three months, the community developed rather well: New users
>  appeared on the mailing lists, and we were able to elect the first
>  external committers. The mailing list activity has increased and many
>  discussions on development-related tasks were done on the developer
>  mailing list. Overall, we feel that the community is more and more
>  following the Apache way.
> 
>  Some specific community development activities include:
>  - We updated the wiki with developer information, getting started guides
>and step-by-step manuals for release management (see
>https://s.apache.org/ocz7t)
>  - Updated website to include better overview of existing pipeline elements
>  - Intensified work with manufacturing company to use StreamPipes for IIoT
>analytics
>  - We published a blog post on how to run StreamPipes on Kubernetes, we now
>plan to publish blog posts at regular intervals
>  - Added "communication channels & strategy" page in the wiki and gathered
>potential communication channels in close exchange with our mentors
>planning to intensify community growth
>  - Number of Twitter followers increased (last report: 103, currently: 128)
>  - Number of Github stars increased (last report: 109, currently: 143)
> 
> ### How has the project developed since the last report?
> 
>  - We successfully released our first official Apache release (version
>0.66.0) including several improvements (see release notes
>https://s.apache.org/2grsk), to name the most prominent changes:
>completely new dashboard and notification design in UI, more pipeline 
>elements and improved connectors
>  - Many new pipeline elements have been added by newly elected committers
>  - Both documentation and website were improved in order to increase the
>adoption of Apache StreamPipes and grow the user base
> 
>  In the next month, we plan to add better support for edge computing and
>  improve the UI.
> 
> ### How would you assess the podling's maturity?
> Please feel free to add your own commentary.
> 
>  - [ ] Initial setup
>  - [ ] Working towards first release
>  - [x] Community building
>  - [ ] Nearing graduation
>  - [ ] Other:
> 
> ### Date of last release:
> 
>  2020-05-19 (Version 0.66.0)
> 
> ### When were the last committers or PPMC members elected?
> 
>  Four new committers were elected and joined the team since the last report
>  aiding in contributing/maturing the project.
> 
>  - Florian Micklich (2020-03-26)
>  - Daniel Ebi (2020-05-12)
>  - Felix John (2020-05-12)
>  - Grainier Perera (2020-05-22)
> 
> ### Have your mentors been helpful and responsive?
> Are things falling through the cracks? If so, please list any open issues 
> that need to be addressed.
> 
>  Our mentors provided very good support in preparing and performing our
>  first Apache release. We would like to thank them for very helpful advice,
>  also for some licensing questions we had on the list.
> 
> ### Is the PPMC managing the podling's brand / trademarks?
> Are 3rd parties respecting and correctly using the podlings name and brand? 
> If not what actions has the PPMC taken to correct this? Has the VP, Brand 
> approved the project name?
> 
>  Yes, we started to prepare a wiki page to develop a communication
>  strategy, which also includes brand management. The next step is to
>  create a new logo that will be voted on by the community. We are
>  constantly monitoring the usage of the Apache StreamPipes name in
>  external websites and posts.
> 
> ### Signed-off-by:
> 
>  - [ ] (streampipes) Christofer Dutz
> Comments:
>  - [ ] (streampipes) Jean-Baptiste Onofré
> Comments:
>  - [ ] (streampipes) Julian Feinauer
> Comments:
>  - [ ] (streampipes) Justin Mclean
> Comments:
>  - [ ] (streampipes) Kenneth Knowles
> Comments:
> 
> ### IPMC/Shepherd notes:

RE: Podling Streampipes Report Reminder - June 2020

[Dominik Riemer]

Hi Patrick,

sounds good, I don't have anything to add!

Dominik

-Original Message-
From: Patrick Wiener  
Sent: Monday, June 1, 2020 11:06 PM
To: dev@streampipes.apache.org
Subject: Re: Podling Streampipes Report Reminder - June 2020

Hi all,

I prepared a first draft for the upcoming report (due on Wed, June 03).

It would be good if you could check the report and provide me with some 
feedback:

## StreamPipes

  StreamPipes is a self-service (Industrial) IoT toolbox to enable non-
  technical users to connect, analyze and explore (Industrial) IoT data
  streams.

  StreamPipes has been incubating since 2019-11-11.

### Three most important unfinished issues to address before graduating:

  1. Multiple Apache Releases
  2. Grow the developer community
  3. Attract more users

### Are there any issues that the IPMC or ASF Board need to be aware of?

  There are no issues right now.

### How has the community developed since the last report?

  In the past three months, the community developed rather well: New users
  appeared on the mailing lists, and we were able to elect the first
  external committers. The mailing list activity has increased and many
  discussions on development-related tasks were done on the developer
  mailing list. Overall, we feel that the community is more and more
  following the Apache way.

  Some specific community development activities include:
  - We updated the wiki with developer information, getting started guides
and step-by-step manuals for release management (see
https://s.apache.org/ocz7t)
  - Updated website to include better overview of existing pipeline elements
  - Intensified work with manufacturing company to use StreamPipes for IIoT
analytics
  - We published a blog post on how to run StreamPipes on Kubernetes, we now
plan to publish blog posts at regular intervals
  - Added "communication channels & strategy" page in the wiki and gathered
potential communication channels in close exchange with our mentors
planning to intensify community growth
  - Number of Twitter followers increased (last report: 103, currently: 128)
  - Number of Github stars increased (last report: 109, currently: 143)

### How has the project developed since the last report?

  - We successfully released our first official Apache release (version
0.66.0) including several improvements (see release notes
https://s.apache.org/2grsk), to name the most prominent changes:
completely new dashboard and notification design in UI, more pipeline 
elements and improved connectors
  - Many new pipeline elements have been added by newly elected committers
  - Both documentation and website were improved in order to increase the
adoption of Apache StreamPipes and grow the user base

  In the next month, we plan to add better support for edge computing and
  improve the UI.

### How would you assess the podling's maturity?
Please feel free to add your own commentary.

  - [ ] Initial setup
  - [ ] Working towards first release
  - [x] Community building
  - [ ] Nearing graduation
  - [ ] Other:

### Date of last release:

  2020-05-19 (Version 0.66.0)

### When were the last committers or PPMC members elected?

  Four new committers were elected and joined the team since the last report
  aiding in contributing/maturing the project.

  - Florian Micklich (2020-03-26)
  - Daniel Ebi (2020-05-12)
  - Felix John (2020-05-12)
  - Grainier Perera (2020-05-22)

### Have your mentors been helpful and responsive?
Are things falling through the cracks? If so, please list any open issues that 
need to be addressed.

  Our mentors provided very good support in preparing and performing our
  first Apache release. We would like to thank them for very helpful advice,
  also for some licensing questions we had on the list.

### Is the PPMC managing the podling's brand / trademarks?
Are 3rd parties respecting and correctly using the podlings name and brand? If 
not what actions has the PPMC taken to correct this? Has the VP, Brand approved 
the project name?

  Yes, we started to prepare a wiki page to develop a communication
  strategy, which also includes brand management. The next step is to
  create a new logo that will be voted on by the community. We are
  constantly monitoring the usage of the Apache StreamPipes name in
  external websites and posts.

### Signed-off-by:

  - [ ] (streampipes) Christofer Dutz
 Comments:
  - [ ] (streampipes) Jean-Baptiste Onofré
 Comments:
  - [ ] (streampipes) Julian Feinauer
 Comments:
  - [ ] (streampipes) Justin Mclean
 Comments:
  - [ ] (streampipes) Kenneth Knowles
 Comments:

### IPMC/Shepherd notes:


> Am 28.05.2020 um 00:30 schrieb jmcl...@apache.org:
> 
> Dear podling,
> 
> This email was sent by an automated system on behalf of the Apache 
> Incubator PMC. It is an initial reminder to give you plenty of time to 
> prepare your quarterly board report.
> 
> The board meeting is scheduled for 

[GitHub] [incubator-streampipes-extensions] bossenti opened a new pull request #22: Create an adapter for Modbus protocol

[GitBox]

bossenti opened a new pull request #22:
URL: https://github.com/apache/incubator-streampipes-extensions/pull/22


   @tenthe as discussed on Thursday



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-streampipes-extensions] Madabaru opened a new pull request #21: Fix in ValidateTable for MySQL Sink

[GitBox]

Madabaru opened a new pull request #21:
URL: https://github.com/apache/incubator-streampipes-extensions/pull/21


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Documentation for third party services

[Philipp Zehnder]

Hi all,

after we received some feedback from the community, we are currently improving 
the SQL integration.
During the development I realized that it always takes a lot of time to set up 
all third party services (e.g. databases, message brokers, PLCs, …).
We already have services for most of them in the CLI, but I think it makes 
sense to provide some documentation how to set them up. 

The documentation should help users and developers to quickly try it out, not 
for a production ready deployment.
I was wondering where to put it, to the wiki or to the documentation on the 
website?

What do you think?

Philipp

Re: Podling Streampipes Report Reminder - June 2020

[Philipp Zehnder]

Hi Patrick,

thanks a lot, the report looks good.
We can also mention the webinar which is now on YouTube.

Philipp


> On 1. Jun 2020, at 23:06, Patrick Wiener  wrote:
> 
> Hi all,
> 
> I prepared a first draft for the upcoming report (due on Wed, June 03).
> 
> It would be good if you could check the report and provide me with some 
> feedback:
> 
> ## StreamPipes
> 
>  StreamPipes is a self-service (Industrial) IoT toolbox to enable non-
>  technical users to connect, analyze and explore (Industrial) IoT data
>  streams.
> 
>  StreamPipes has been incubating since 2019-11-11.
> 
> ### Three most important unfinished issues to address before graduating:
> 
>  1. Multiple Apache Releases
>  2. Grow the developer community
>  3. Attract more users
> 
> ### Are there any issues that the IPMC or ASF Board need to be aware of?
> 
>  There are no issues right now.
> 
> ### How has the community developed since the last report?
> 
>  In the past three months, the community developed rather well: New users
>  appeared on the mailing lists, and we were able to elect the first
>  external committers. The mailing list activity has increased and many 
>  discussions on development-related tasks were done on the developer 
>  mailing list. Overall, we feel that the community is more and more 
>  following the Apache way.
> 
>  Some specific community development activities include:
>  - We updated the wiki with developer information, getting started guides
>and step-by-step manuals for release management (see
>https://s.apache.org/ocz7t)
>  - Updated website to include better overview of existing pipeline elements
>  - Intensified work with manufacturing company to use StreamPipes for IIoT
>analytics
>  - We published a blog post on how to run StreamPipes on Kubernetes, we now
>plan to publish blog posts at regular intervals
>  - Added "communication channels & strategy" page in the wiki and gathered
>potential communication channels in close exchange with our mentors
>planning to intensify community growth
>  - Number of Twitter followers increased (last report: 103, currently: 128)
>  - Number of Github stars increased (last report: 109, currently: 143)
> 
> ### How has the project developed since the last report?
> 
>  - We successfully released our first official Apache release (version
>0.66.0) including several improvements (see release notes
>https://s.apache.org/2grsk), to name the most prominent changes:
>completely new dashboard and notification design in UI, more pipeline 
>elements and improved connectors  
>  - Many new pipeline elements have been added by newly elected committers
>  - Both documentation and website were improved in order to increase the
>adoption of Apache StreamPipes and grow the user base
> 
>  In the next month, we plan to add better support for edge computing and
>  improve the UI.
> 
> ### How would you assess the podling's maturity?
> Please feel free to add your own commentary.
> 
>  - [ ] Initial setup
>  - [ ] Working towards first release
>  - [x] Community building
>  - [ ] Nearing graduation
>  - [ ] Other:
> 
> ### Date of last release:
> 
>  2020-05-19 (Version 0.66.0)
> 
> ### When were the last committers or PPMC members elected?
> 
>  Four new committers were elected and joined the team since the last report
>  aiding in contributing/maturing the project.
> 
>  - Florian Micklich (2020-03-26)
>  - Daniel Ebi (2020-05-12)
>  - Felix John (2020-05-12)
>  - Grainier Perera (2020-05-22)
> 
> ### Have your mentors been helpful and responsive?
> Are things falling through the cracks? If so, please list any
> open issues that need to be addressed.
> 
>  Our mentors provided very good support in preparing and performing our
>  first Apache release. We would like to thank them for very helpful advice,
>  also for some licensing questions we had on the list.
> 
> ### Is the PPMC managing the podling's brand / trademarks?
> Are 3rd parties respecting and correctly using the podlings
> name and brand? If not what actions has the PPMC taken to
> correct this? Has the VP, Brand approved the project name?
> 
>  Yes, we started to prepare a wiki page to develop a communication
>  strategy, which also includes brand management. The next step is to 
>  create a new logo that will be voted on by the community. We are 
>  constantly monitoring the usage of the Apache StreamPipes name in 
>  external websites and posts.
> 
> ### Signed-off-by:
> 
>  - [ ] (streampipes) Christofer Dutz
> Comments:
>  - [ ] (streampipes) Jean-Baptiste Onofré
> Comments:
>  - [ ] (streampipes) Julian Feinauer
> Comments:
>  - [ ] (streampipes) Justin Mclean
> Comments:
>  - [ ] (streampipes) Kenneth Knowles
> Comments:
> 
> ### IPMC/Shepherd notes:
> 
> 
>> Am 28.05.2020 um 00:30 schrieb jmcl...@apache.org:
>> 
>> Dear podling,
>> 
>> This email was sent by an automated system on behalf of the Apache
>> Incubator PMC. It is an initial reminder to