Re: [VOTE] Confluence wiki for StreamPipes

2019-12-03 Thread Patrick Wiener
Hi Leesf, thanks for your vote and welcome to the StreamPipes community. Cheers, Patrick > Am 03.12.2019 um 06:07 schrieb leesf : > > +1 > > Best, > Leesf > > Dominik Riemer 于2019年12月3日周二 上午5:58写道: > >> Hi Justin, >> I guess Chris' proposal to cast a vote was just to let us learn how votes

Re: December Podling Report - Draft

2019-12-01 Thread Patrick Wiener
Cheers Dominik - with the remarks of Philipp the report looks fine to me. Patrick > Am 01.12.2019 um 14:57 schrieb Dominik Riemer : > > Thanks Philipp! > > I added to new items under "How has the project developed since the last > report" mentioning the submission of ICLAs, software grant

Exclude timestamp from mapping properties

2019-12-02 Thread Patrick Wiener
Hi, I was just working on updating the aggregation pipeline element in flink to support nary-mappings, when I stumbled upon this. Currently, we show the timestamp fields as potential mapping properties in the user dialog as suitable candidates, e.g. to perform aggregation such as

Re: Exclude timestamp from mapping properties

2019-12-03 Thread Patrick Wiener
you configure a pipeline > element, there is already a switch called "Show recommended settings only" - > this could be on by default and filter event properties for properties that > match the required scope. > > Dominik > > -Original Message- > From

Multi-arch docker images for StreamPipes

2020-02-24 Thread Patrick Wiener
Hi, as a result of a PoC described in a blog post (https://streampipes.apache.org/docs/blog/2020/01/23/streampipes_on_rpi/ ) I was working on bringing this multi-arch feature officially into the StreamPipes community

Re: Camera Adapter in StreamPipes Connect

2020-03-06 Thread Patrick Wiener
ve > multiple drivers (e.g. GStreamer). > I tried to run an example locally, but it did not work on my computer yet. > But maybe we should test it directly on a Pi. > > > [1] https://github.com/sarxos/webcam-capture > <https://github.com/sarxos/webcam-capture> &g

Re: Multi-arch docker images for StreamPipes

2020-02-25 Thread Patrick Wiener
eamPipes on a Pi or Nano. > > Dominik > > -Original Message- > From: Patrick Wiener > Sent: Monday, February 24, 2020 6:38 PM > To: dev@streampipes.apache.org > Subject: Multi-arch docker images for StreamPipes > > Hi, > > as a result of a PoC described in

Re: [DISCUSS] Processor to synchronize two data streams

2020-01-30 Thread Patrick Wiener
eams, but the image is not displayed in StreamPipes. > > I think it would be useful to include images in the documentation to explain > the components. So does anyone know how we can include images? Is that > feature currently missing or did I just use the wrong URL in Markdown? > > P

Re: Setup dev project in Intelij

2020-02-18 Thread Patrick Wiener
Hi, @Florian: cheers for your contrib and really cool processors :) I would not recommend putting this in a single component description. IMHO the documentation of a processor should be as accurate as necessary, not as accurate as possible such that a user can quickly decide for what kinds of

Re: [DISCUSS] Processor to synchronize two data streams

2020-01-22 Thread Patrick Wiener
> order, since we will use dimension properties for partitioning events in > Kafka. This means the events should be in order. > Do you think we need an additional mechanism in the component to ensure the > ordering of events or is the Kafka guarantee sufficient for the moment? > >

Re: ApacheCon NA 2020

2020-01-22 Thread Patrick Wiener
Hi, I also like the idea of one joint talk with other projects from the Apache IoT world to showcase the interoperability of our projects. Speaking about ideas for the other talk: we broadly showcased what StreamPipes is and how one can leverage the processor toolbox to quickly built

Re: Hey there!

2020-04-27 Thread Patrick Wiener
g MOA __ >> On the long run, I'd also like to add my own research, but we will see __ >> >> Best >> Marco >> >> Am 23.04.20, 11:21 schrieb "Patrick Wiener" : >> >> Hi Marco, >> >> welcome on board :) Sounds interesting

Re: Hey there!

2020-04-23 Thread Patrick Wiener
Hi Marco, welcome on board :) Sounds interesting and def gonna be a great add on to the toolbox. Ha! Philipp was quicker than me - I was about to ask the same. Do you have any concrete use cases in mind? Patrick > Am 23.04.2020 um 11:20 schrieb Philipp Zehnder : > > > Hi Marco, > >

Re: Preparing release

2020-04-27 Thread Patrick Wiener
Hi, I also tested the process and had no problems. I only added „Docker Compose“ to the required build/test tools in the confluence docs. Patrick > Am 26.04.2020 um 21:25 schrieb Dominik Riemer : > > Hi Philipp, > > great, thanks for improving the guide! > > I've just tested Ditto sink,

Re: DataSink for Redis

2020-05-13 Thread Patrick Wiener
Hi, for Kubernetes deployment, you can have a look at how we did it for IoTDB [1] You would need to declare three files: - deployment - service - persistent volume claim Please make sure to add the following template as this will be parsed by helm, so that we have optional external services

Re: [VOTE] Apache StreamPipes (incubating) 0.66.0 RC1 release

2020-05-06 Thread Patrick Wiener
Hi all, +1 (binding) I checked: - Specified url to download staged release artifacts (core, extensions, installer) are valid - Signature is correct for all staged release artifacts - Signature reference contain valid Apache email address - Checksum is correct - LICENSE, NOTICE, README,

[DISCUSS] StreamPipes in Slack

2020-05-11 Thread Patrick Wiener
Hi, I wanted to start a discussion on how we should proceed with StreamPipes in Slack in the future. Currently, we have run two options both in use: 1) Single StreamPipes channel in the ASF Slack 2) Own StreamPipes-Community Slack (currently references on the streampipes.apache.org

Re: Data processor to evaluate JavaScript

2020-05-14 Thread Patrick Wiener
Hi, sounds reasonable. I share the thoughts of Philipp regarding the output schema. But as you said Grainier, we can also improve :) btw: I really like the idea to also allow the more technical user to easily integrate their function snippets without having to care about the actual execution.

Re: New Contributor

2020-03-20 Thread Patrick Wiener
Hi Felix, welcome to the StreamPipes family :) @Julian: thx for pointing it out Cheers, Patrick > Am 20.03.2020 um 13:38 schrieb Julian Feinauer : > > Hi Felix, > > welcome and very nice to have you here : ) > One rather technical comment, could you subscribe to this mailing list (by >

Re: Promoting our first release

2020-05-07 Thread Patrick Wiener
Hi, I was thinking about that as well hence the blog post on running StreamPipes on K8s via our helm chart. In general, we should discuss various „channels“ to promote StreamPipes and to grow our community. Blog posts is definitely a good approach - however there are more ways. Apache Beam has

Re: implementing a new service postgis

2020-05-23 Thread Patrick Wiener
Hi Florian, What kind of scenario do you have in mind for what you referred to as „internal use“? Do you want to query some information from the database? Could you elaborate on that? In addition, I’m wondering what does this init sql script do? Create certain databases, schemas, tables etc?

Re: Release 0.67.0 preparation and open issues

2020-09-05 Thread Patrick Wiener
Hi, I updated and tested our maven archetypes for * pe-source-jvm * pe-processor-jvm * pe-processor-flink * pe-sink-jvm * pe-sink-flink and cherry picked them to be part of this release. Patrick > Am 03.09.2020 um 15:49 schrieb Patrick Wiener : > > Hi, > > I’ll fix the docke

Re: Missing Run Configurations

2020-09-11 Thread Patrick Wiener
gt; extensions twice and developers just developing the backend have some > additional run configs in IntelliJ. > Since these configurations don't change often and it is rather an exception > to work only in the backend, I don't think this are big problems. > > Philipp > > >> On 11. S

Re: Missing Run Configurations

2020-09-11 Thread Patrick Wiener
Hi, I’m with you. We kinda „experimented“ a little bit to sort out what would be a good way to get started in a convenient fashion. Every option has its pros but also some drawbacks. What you missed is that the whole discussion about runConfigurations is only tailored & relevant for developers

Re: Missing Run Configurations

2020-09-11 Thread Patrick Wiener
me window (which is eg required for refactoring), and I'd personally > consider this more important. > > What do you think? > > Dominik > > -Original Message- > From: Patrick Wiener > Sent: Friday, September 11, 2020 11:42 AM > To: dev@streampipes.apache.o

Re: [DISCUSS] Sending all Jira notifications to the issues list

2020-09-09 Thread Patrick Wiener
Sounds good to me :) Patrick > Am 09.09.2020 um 20:41 schrieb Dominik Riemer : > > Hi, > > I like Chris' proposal - moving the close-mails to issues@ and keeping the > created-mails here. > > Is that ok for everyone? Then I'd ask Infra to change the Jira settings. > > Dominik > > On

Re: NullPointer in Image Enrichment Processor

2020-09-14 Thread Patrick Wiener
berry PI to make sure the minimum hardware > requirements are still met. > > Is there another way to import the fonts or to change the code so that the > image enricher works without access to local font libraries? > > Dominik > > On 2020/09/13 17:06:10, Patrick Wiener

Re: Release 0.67.0 preparation and open issues

2020-09-14 Thread Patrick Wiener
> > -----Original Message- > From: Patrick Wiener > Sent: Saturday, September 5, 2020 9:59 AM > To: dev@streampipes.apache.org > Subject: Re: Release 0.67.0 preparation and open issues > > Hi, > > I updated and tested our maven archetypes for > > * pe-source-

Re: NullPointer in Image Enrichment Processor

2020-09-14 Thread Patrick Wiener
Hi, I fixed the issue. It happened that the below changes worked. I’ll commit the changes accordingly. Patrick > Am 14.09.2020 um 11:38 schrieb Patrick Wiener : > > That was my thought - and it would make it „harder“ to automate things again. > Now that we got a common base i

EventProcessor interface methods

2020-09-04 Thread Patrick Wiener
Hi, while going through our archetypes one thing I found was that we currently have the following interface for data processors, that is implemented [1]: public interface EventProcessor extends PipelineElement { void onInvocation(B parameters, SpOutputCollector spOutputCollector,

Re: [DISCUSS] Sending all Jira notifications to the issues list

2020-09-01 Thread Patrick Wiener
Hi, that is definitely the case - I myself had the same problems not really seeing the discussion mails on the dev list. So def a +1 for changing the settings. Patrick > Am 01.09.2020 um 15:59 schrieb Dominik Riemer : > > Hi all, > > currently, Jira notifications about created and closed

Re: NullPointer in Image Enrichment Processor

2020-09-13 Thread Patrick Wiener
It would be really helpful if you could post a snippet of the concrete failure log. Patrick > Am 13.09.2020 um 18:43 schrieb Philipp Zehnder : > > Hi, > > we currently have a bug in our image enrichment processor. > Everything works fine when we running in development, but in docker some >

Re: StreamPipes visualization

2020-10-15 Thread Patrick Wiener
Hi all, @Branislav, I added some preliminary work (image) and some experience with the ngx-charts (visualization framework we use in the dashboard) to the wiki. Feel free to comment and further upload your pics :) Already really liked you suggestions Patrick > Am 14.10.2020 um 19:43 schrieb

Re: Apache StreamPipes logo

2020-08-17 Thread Patrick Wiener
Hi, thanks for putting it on Confluent. Even though I would like to see the ASF feather and/or color scheme as part of our Logo I guess this cannot be easily done by combining it with our existing style - greenish, blueish. This would really involve a drastical change, such that we would

Re: [Discuss] New installer structure, and CLI

2020-08-17 Thread Patrick Wiener
duction usage? Should we take the standard docker-compose files for that? > The advantage is that all ports are closed by default there, which is > probably a rather important security consideration. > > Dominik > > On 2020/08/17 10:20:49, Patrick Wiener <mailto:w

Re: Proposal for new improved docs

2020-08-18 Thread Patrick Wiener
33, Dominik Riemer wrote: >>> >>> Hi Patrick, >>> >>> great! I'm fully +1 for improving the documentation as proposed. We could >>> work on the new structure while the next release is under vote and release >>> the new docs version togethe

Re: Release 0.67.0 preparation and open issues

2020-08-27 Thread Patrick Wiener
there anyone who whants to volunteer as a release manager this time? ;-) >>> I'd be happy to support, and there is also a documentation that describes >>> all release steps [3]. >>> >>> Dominik >>> >>> [1] https://issues.apache.org/jira/br

Release 0.67.0 preparation and open issues

2020-08-20 Thread Patrick Wiener
Hi, I guess we should start preparing our next StreamPipes release 0.67.0. One big milestone was already made by Dominik porting the UI to Angular 10. There are still some open issue, that need to resolved tho [1]. If you’re eager, there are still some issues left to be fixed - the sooner the

Re: [Discuss] New installer structure, and CLI

2020-08-24 Thread Patrick Wiener
is environment variable? >> Is there anything I need to be aware of during an installation? >> >> Philipp >> >>> On 17. Aug 2020, at 21:40, Patrick Wiener wrote: >>> >>> Good point - I guess both work. So it’s the user’s/developer’s choice.

[Discuss] New installer structure, and CLI

2020-08-17 Thread Patrick Wiener
Hi, I was working on completely refactoring the installer on the weekend [1] The problem was always having some sort of too many areas where we needed to touch and that we did not have a clear focus on what kinda user role should use what kinda way to set up his/her environment. Thus, I

Re: [VOTE] Apache StreamPipes 0.67.0 (incubating) RC1 release

2020-09-26 Thread Patrick Wiener
my environment: - os: macOS 10.15.6, x86_64 - java: v1.8.0_242-b08 (adoptopenjdk-8-openj9.jdk) - mvn: v3.6.3 - npm: v6.14.4 - node: v13.13.0 - docker: v19.03.12, build fec3683 - docker-compose: v1.27.2, build 18f557f9 I checked: [OK] Download all staged artifacts under the url specified in the

Re: [DISCUSS] [VOTE] Apache StreamPipes 0.67.0 (incubating) RC1 release

2020-09-26 Thread Patrick Wiener
+1 for a new RC Patrick > Am 26.09.2020 um 12:29 schrieb Grainier Perera : > > I'm also +1 to go with a new RC given that the fix is relatively small and > won't affect other features of SP. Also, since the fix is at core-ui level, > It'll be needed for a user who wants to write a PE extension

Re: [VOTE] Apache StreamPipes 0.67.0 (incubating) RC2 release

2020-09-27 Thread Patrick Wiener
+1 (non-binding) my environment: - os: macOS 10.15.6, x86_64 - java: v1.8.0_242-b08 (adoptopenjdk-8-openj9.jdk) - mvn: v3.6.3 - npm: v6.14.4 - node: v13.13.0 - docker: v19.03.12, build fec3683 - docker-compose: v1.27.2, build 18f557f9 I checked: [OK] Download all staged artifacts under the url

Re: Telegram publisher to send event notifications

2020-05-26 Thread Patrick Wiener
Hi Grainier, this is really nice. We were at some point actually talking about such a sink so super cool that you implemented it. I used telegram once in another project where one was informed about alerts/warnings and quite liked it. Thanks for the contribution and valuable work. Patrick >

Re: Telegram publisher to send event notifications

2020-05-27 Thread Patrick Wiener
ficationStream; > > > Further, I'll do some research on how similar products have done this, and > update the thread with how we can implement the same with StreamPipes. > What do you think? > > [1] https://siddhi.io/en/v5.1/docs/examples/if-then-else/ > [2] https://siddhi.io/e

Re: implementing a new service postgis

2020-05-25 Thread Patrick Wiener
settings and controls will be > set and checked in the depending onInvocation method. > > Greetings > Florian > > > > Am Samstag, den 23.05.2020, 09:54 +0200 schrieb Patrick Wiener: >> Hi Florian, >> >> What kind of scenario do you have in mind for

Re: Podling Streampipes Report Reminder - June 2020

2020-06-01 Thread Patrick Wiener
Hi all, I prepared a first draft for the upcoming report (due on Wed, June 03). It would be good if you could check the report and provide me with some feedback: ## StreamPipes StreamPipes is a self-service (Industrial) IoT toolbox to enable non- technical users to connect, analyze and

Re: Podling Streampipes Report Reminder - June 2020

2020-06-02 Thread Patrick Wiener
alright - then I add it to the Incubator Wiki. Patrick > Am 02.06.2020 um 19:36 schrieb Dominik Riemer : > > Hi Patrick, > > sounds good, I don't have anything to add! > > Dominik > > -Original Message- > From: Patrick Wiener > Sent: Monday

Re: Release version of installer

2020-09-21 Thread Patrick Wiener
ipp Zehnder : > > How will the StreamPipes version be set? > Currently we start by default the latest release. > Does the user then set the version manually or can we use something like > LATEST to alway get the newest version? > > Philipp > >> On 18. Sep 2020

Re: Adding window semantics to the sdk/core

2020-07-16 Thread Patrick Wiener
I also tend towards option 2. It makes it more intuative to develop windowed operations by having a dedicated windowed event processor. We can collect the design choices etc in the wiki and discussion points in the wiki. Patrick > Am 16.07.2020 um 22:55 schrieb Philipp Zehnder : > > Hi, > >

Re: Adding StreamPipes Python wrapper

2020-07-17 Thread Patrick Wiener
interfaces of the Java wrapper, right? So we do not >> have any python specific endpoints. >> I think this would ease the usage for people in the community and already >> try an early version of the wrapper. >> Alternatively, we can put it into the core in streampipes

Re: Adding StreamPipes Python wrapper

2020-07-19 Thread Patrick Wiener
7.2020 um 15:41 schrieb Patrick Wiener : > > To give you a sneak peak - it currently looks like this on the python side. > However, note that the main magic (registration, model declaration etc) > still happens on the java side. >

Re: Adding StreamPipes Python wrapper

2020-08-03 Thread Patrick Wiener
uence/display/STREAMPIPES/SIP-02+Python+wrapper > <https://cwiki.apache.org/confluence/display/STREAMPIPES/SIP-02+Python+wrapper> > > <https://cwiki.apache.org/confluence/display/STREAMPIPES/SIP-02+Python+wrapper > > <https://cwiki.apache.org/confluence/display/STREAMPIPES/SIP-0

Re: Timestamp in event

2020-08-04 Thread Patrick Wiener
Hi Philipp, I think that is definitely a valuable feature to check for timestamp existence before creating the adapter since we have a various processors or sinks that rely on a timestamp. One possible solution could be to notify users immediately in case a timestamp field is missing, e.g.

Proposal for new improved docs

2020-08-05 Thread Patrick Wiener
Hi, since we often got the feedback that our current docs do not provide enough technical details and „feel“ to tutorial-like, I’d like to start a proposal in the confluent wiki for a new docs structure that combines developer & user guide and provides more in-depths chapters, in order for

Re: Timestamp in event

2020-08-10 Thread Patrick Wiener
; >>> Hi, >>> >>> is there an advantage of requiring a timestamp in every event? Maybe we >>> could also only display a warning or actively ask users in Connect in case >>> a timestamp is missing and force the addition of timestamps in one of the >>>

Re: Added RunConfigurations as Project Files for IntelliJ

2020-07-01 Thread Patrick Wiener
Hi Felix, cool - that def helps the onboarding process for those using IntelliJ. Do these runConfigurations also need to have the env-File plugin pre-installed to work? Patrick > Am 01.07.2020 um 18:34 schrieb Felix John : > > Hi, > > I have added Run Configurations as project files for

Re: Data processor to evaluate JavaScript

2020-06-17 Thread Patrick Wiener
On Tue, 19 May 2020 at 03:05, Dominik Riemer >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Grainier, >>>>>>>>>> >>>>>>>>>> I just pushed an initial version of t

Re: PE to rate-limit events

2020-06-03 Thread Patrick Wiener
Hi Grainier, indeed a very nice extension to the toolbox. What I was thinking of is the following. Since you basically implemented window-semantic in plain java this is something we could need for various pipeline elements, e.g. a lightweight aggregation component in Java other than in Flink.

Re: Switching to the standard disclaimer?

2020-06-05 Thread Patrick Wiener
also +1 from me. we’ll have to it eventually so the earlier the better - no doubts. Patrick > Am 05.06.2020 um 10:31 schrieb Philipp Zehnder : > > Hi, > > from my site also a +1. > > The sooner we solve the issues the better. > > Philipp > > On 2020/06/03 06:42:31, Christofer Dutz wrote:

Adding StreamPipes Python wrapper

2020-07-16 Thread Patrick Wiener
Hi guys, this mail is to inform you and discuss the addition of a new wrapper for StreamPipes: StreamPipes Python Wrapper Current wrappers such as standalone (JVM) or distributed (Flink) already allow us to develop new processors in the given runtime environment. I suppose to add the Python

Re: Added RunConfigurations as Project Files for IntelliJ

2020-07-20 Thread Patrick Wiener
lipp > >> On 2. Jul 2020, at 11:17, Felix John wrote: >> >> Hi Patrick, >> >> good question. I just tested one configuration whilst disabling the >> "EnvFile" plugin. I worked out nicely. >> >> >> Greetings, >> Felix >

Re: Added RunConfigurations as Project Files for IntelliJ

2020-07-23 Thread Patrick Wiener
-streampipes/pull/25 > <https://github.com/apache/incubator-streampipes/pull/25> > >> On 20. Jul 2020, at 10:46, Patrick Wiener wrote: >> >> Hi all, >> >> While it works on Mac/Windows hosts, a problem that still arise and we >> should be aware o

Re: Adding StreamPipes Python wrapper

2020-07-18 Thread Patrick Wiener
the community and already try > an early version of the wrapper. > Alternatively, we can put it into the core in streampipes-wrapper-python as > you suggested, but then a user has to checkout the backend and the extensions > project to develop a new processor. > Whats your opinion on

Re: Promotional Material

2020-11-29 Thread Patrick Wiener
Hi, that’s fine for me - we can have a default set of slides there and some sample data for a demo. Patrick > Am 29.11.2020 um 22:52 schrieb Dominik Riemer : > > Hi, > > +1 for creating a wiki page with a presentation template and an how-to guide > on showing some simple demos. > > For

Re: [DISCUSS] Naming of internal sinks

2020-11-29 Thread Patrick Wiener
Hi all, Im with the others - I would rather go for option 1 with the suggestions of Grainier using a small SP Icon (maybe top left part). Otherwise the name is very long as well. Patrick > Am 30.11.2020 um 02:13 schrieb Grainier Perera : > > Hi all, > > I think we shouldn't use `Internal`

Re: [ANNOUNCE] New PPMC member: Marco Heyden

2020-12-09 Thread Patrick Wiener
Hi Marco, welcome aboard :) Patrick > Am 09.12.2020 um 15:47 schrieb Marco Heyden : > > Thank you! I really look forward to contributing! > > > Marco > > Am 08.12.20, 23:16 schrieb "Christofer Dutz" : > >YAY!!! ... welcome to the club ;-) > >Chris > > >-Ursprüngliche

Re: Podling Streampipes Report Reminder - December 2020

2020-11-30 Thread Patrick Wiener
Hi, report looks good to me - thanks Dominik for preparing it. I’ll go through it more thoroughly tomorrow. Patrick > Am 30.11.2020 um 21:47 schrieb Philipp Zehnder : > > Hi, > > thank you Dominik for the report, it looks great. > > I added the talk we gave last week [1], organized for

Re: [DISCUSS] Single StreamPipes repository

2020-11-10 Thread Patrick Wiener
into a single one and integrate the extension modules in the >> top-level folder of the core repo? >> >> Dominik >> >> On 2020/11/09 15:38:52, Patrick Wiener wrote: >>> For the extensions CI pipeline, we first checkout the latest dev commit and >>>

Re: CI docker builds & minor repo structure refactoring

2020-11-10 Thread Patrick Wiener
ould also think of > renaming it to something like "JS", as we will probably not only have the UI, > but also some JS core modules and an SDK/wrapper for JS in the near future as > well. > > Dominik > > On 2020/11/09 15:46:14, Patrick Wiener wrote: >> Well,

Correction value preprocessing rule in connect adapters

2020-11-11 Thread Patrick Wiener
Hi, i just added a new preprocessing rule for connect adapters. As you might know, connect adapters come with a variety of different preprocessing rules, e.g. timestamp/unit transformation, add static event fields etc. These preprocessing rules allow to perform simple functions directly

Re: [DISCUSS] Single StreamPipes repository

2020-11-09 Thread Patrick Wiener
oject, > because the backend has to be built first, right? > > Philipp > >> On 6. Nov 2020, at 16:18, Patrick Wiener wrote: >> >> Hi all, >> >> we have thrown up the discussion on merging our repositories into a single >> StreamPipes repo for core

Re: CI docker builds & minor repo structure refactoring

2020-11-09 Thread Patrick Wiener
en in the build and I didn’t know that. > > > Philipp > >> On 6. Nov 2020, at 16:03, Patrick Wiener wrote: >> >> Ok - I’ll start another discussion in a separate thread. >> >> Regarding the Dockerfile: yepp - this works! you simply have to adjust the >> build

CI docker builds & minor repo structure refactoring

2020-11-06 Thread Patrick Wiener
Hi guys, i noticed the current CI workflow file using Github Actions is pretty confusing when looking at it. - I’d suggest to have a more clean structure and using a separate shell script that performs the actual build, e.g., tools/ci/docker_build_push.sh - We could move the Dockerfiles

Re: CI docker builds & minor repo structure refactoring

2020-11-06 Thread Patrick Wiener
e docker files are in a different > location? > > Philipp > > >> On 6. Nov 2020, at 15:16, Patrick Wiener wrote: >> >> This would definitely have a major effect on it. I like the idea of merging >> extensions + core together into one holistic streampipes projec

Re: CI docker builds & minor repo structure refactoring

2020-11-06 Thread Patrick Wiener
on if we should merge the > extensions project with the backend project in GitHub to simplify our project > structure. > I was wondering if we should do this first because it will also have a big > effect on the CI job, right? > > Whats your opinion on that? > > Philipp

[DISCUSS] Single StreamPipes repository

2020-11-06 Thread Patrick Wiener
Hi all, we have thrown up the discussion on merging our repositories into a single StreamPipes repo for core and extensions on several occasions. Now, we should discuss the pros and cons about this to see if it’s worth doing this step. Context: Merge core [1] and extensions [2] into a single

Re: [VOTE] Apache StreamPipes logo

2021-01-25 Thread Patrick Wiener
Hi, by far my favorite: #2 Patrick > Am 25.01.2021 um 21:37 schrieb Dominik Riemer : > > Hi all, > > this is a vote to (finally) select a new logo for Apache StreamPipes! > > We have five candidates which can be found in our wiki: >

Re: Google Summer of Code 2021

2021-01-24 Thread Patrick Wiener
Hi all, I like the idea of also following the structured issues as shown in the APISIX project. Find infos for mentors: http://community.apache.org/guide-to-being-a-mentor.html Here the official timeline:

[ANNOUNCE] New PPMC member: Grainier Perera

2021-01-29 Thread Patrick Wiener
Hi all, on behalf of the StreamPipes PPMC, it is my pleasure to inform you that Grainier Perera has accepted the offer to become PPMC member of the StreamPipes project. Over the last 8 month as a committer, Grainier has worked on several valuable features for StreamPipes and has been actively

Re: What about a new release?

2021-06-16 Thread Patrick Wiener
;> >> Dominik >> >> On 2021/06/08 14:26:02, udeho wrote: >>> Hi, >>> >>> just merged the open PR for the changes in the SQL sinks, so no more to-dos >>> from my side either. >>> I am also in favor of a new release. >>> >&

Re: Performance Issues

2021-06-22 Thread Patrick Wiener
Hi Muhammad, Well, I don’t think the version diffs of Flink have such an impact. Do you have more fine-grained evaluation figures, e.g. whats the throughput of the SP Connect Kafka adapter? The Connect Kafka adapter is the first entry point into SP and thus determines the max throughput after

Re: What about a new release?

2021-06-07 Thread Patrick Wiener
Hi, looks good to me - there are already several new features but also some important fixes. As mentioned, we can push the other minor open issues to 0.69.0 to get this one on the road especially with ApacheCon Asia ahead of us in August it would be great to have the 0.68.0 version out :)

Re: [ANNOUNCE] New committer Mohanadarshan Vivekanandalingam

2021-05-28 Thread Patrick Wiener
Welcome aboard Mohan :) Patrick > Am 28.05.2021 um 09:40 schrieb Marco Heyden : > > Congratulations and welcome! > > Marco > > Am 28.05.21, 08:50 schrieb "Grainier Perera" : > >Congratulations Mohan! :) > >Grainier Perera. > > >On Fri, 28 May 2021 at 11:19, Philipp Zehnder

Re: [VOTE] Apache StreamPipes 0.68.0 (incubating) RC1 release

2021-07-07 Thread Patrick Wiener
Hi, +1 from my side. I checked: [ok] Download all staged artifacts under the url specified in the release vote email [ok] Verify the signature is correct [ok] Check if the signature references an Apache email address [ok] Verify the SHA512 checksum [ok] Unzip the archive [ok] Verify the

Re: No submissions for any of the two ApacheCons?

2021-04-26 Thread Patrick Wiener
Hi, just as a reminder: has anyone submitted a talk yet? And are there any other suggestions for topics? Patrick > Am 19.04.2021 um 17:51 schrieb Patrick Wiener : > > Hi, > > I agree - there should be at least 2 talks about new features. Indeed, > StreamPipes client

Re: Documentation update

2021-05-04 Thread Patrick Wiener
Hi, looks awesome - great work. Maybe just a minor that we can fix soon: the landing page (middle part) looks a bit wild/cluttered with many text-based links in the middle that seem to reflect the same content as the navigation on the left. Besides this it looks nice. Patrick > Am

Re: Accessing Log files to find error and solve issues running StreamPipes

2021-01-24 Thread Patrick Wiener
Hi, Indeed we should get to a point where it’s easier to get to know what is happening. This is not bound to only logging but should also include monitoring as well. However, monitoring and logging in distributed systems especially in a event-driven architecture with lots of moving parts is

Re: Support for TimescaleDB

2021-03-24 Thread Patrick Wiener
Hi, I have good experience with Postgres in general and every now and then stumbled on some comparing benchmarks between each of these technologies., including TimescaleDB extension. We should shortlist the relevant options and discuss our requirements beforehand, then evaluate the options.

Re: Streampipes für Automotive

2021-03-23 Thread Patrick Wiener
> www.denso.com > > > > -Original Message- > From: Muhammad Faizan > Sent: Tuesday, March 23, 2021 11:15 AM > To: dev@streampipes.apache.org; Patrick Wiener > Cc: Christian Prehofer > Subject: RE: Streampipes für Automotive > > Hi Patrick, >

Re: Exposing certain ports during runtime

2021-03-11 Thread Patrick Wiener
Hi Marco, AFAIK there is no such thing as dynamic port mapping with Docker as you map these ports on container startup. So there is no way for that container to once started to update this port configuration. I don’t see a solution as of now. In a single node setup this is kind of okay'ish,

Re: GSoC - StreamPipes

2021-03-16 Thread Patrick Wiener
Hi Ilias, welcome to the StreamPipes project :) I’ll port this over to the regular dev mailing list. There’s already another thread on the mailing list with a GSoC candidate (Anush) being interested in contributing and helping to develop the Python wrapper for StreamPipes. Please read the

Re: StreamPipes - Help needed

2021-03-15 Thread Patrick Wiener
; or starter-level issues I can start working on? > I feel it will help me writing up my proposal and come up with some good > ideas. > Is there a Slack or a Discord channel I can join to interact and learn more > about the project from the team? > > On Mon, 22 Feb 2021 at 13:

Re: Improve Service Discovery

2021-03-17 Thread Patrick Wiener
Hi, Do you plan on running consul in cluster mode for clustered StreamPipes deployments? You refer to a concept that should be independent of the respective service discovery (SD) tool, but you use the concept of „tags“ to identify services. Is this something that is generic throughout all

Re: Service authentication

2021-03-10 Thread Patrick Wiener
Hi, good points and a relevant topic. Do you still consider having an unauthenticated way in case getting these API keys would end up involving several additional steps to the installation process? I see both options as feasible - either create/manage keys via the UI or have a mechanism

Re: Streampipes für Automotive

2021-03-22 Thread Patrick Wiener
> > > Mit freundlichen Grüßen / Best regards / Yoroshiku Onegai Shimasu > > i.A. Muhammad Faizan > Student / Corporate R / DENSO AUTOMOTIVE Deutschland GmbH > Phone: +49 8165 944 201 / NiceNet: 5033-201 / m.fai...@eu.denso.com > <mailto:m.fai...@eu.denso.com> / w

Re: Streampipes für Automotive

2021-03-22 Thread Patrick Wiener
o.com > <http://www.denso.com/><http://www.denso.com <http://www.denso.com/>> > > > > From: Patrick Wiener mailto:wie...@apache.org>> > Sent: Monday, March 22, 2021 3:50 PM > To: Muhammad Faizan mailto:m.fai...@eu.denso.com>>; > dev@str

Re: Question

2021-03-19 Thread Patrick Wiener
Hi, StreamPipes is a platform for IoT/IIoT analytics providing a rich toolbox of components that can basically be installed either on-premise or in the cloud. For installation, as described on the website you’ll find several option among the most popular is a simple Docker-based installation

Re: Question

2021-03-19 Thread Patrick Wiener
Yes but you have to do manually, i.e., setting up an EC2 instance in case of AWS or any other cloud provider of your choice. Configure the VM and install required Docker + Docker-Compose. Patrick > Am 19.03.2021 um 10:50 schrieb Oswa Haddar : > > Good afternoon, > > I have two questions: >

Re: StreamPipes - Help needed

2021-03-15 Thread Patrick Wiener
er level issues on the Wrapper task so that I can use the time to > understand the project better and get some work done > > On Mon, 15 Mar 2021 at 15:18, Patrick Wiener <mailto:wie...@apache.org>> wrote: > Hi Anush, > > great to hear from you :) > > For c

  1   2   3   >