Hi all, some time ago I implemented the EventStatisticLogger [1], which is currently not used. The PE wrappers call this logger for each incoming event. The logs were delivered to elasticsearch/kibana via filebeat.
Regards Johannes [1] https://github.com/apache/incubator-streampipes/blob/dev/streampipes-logging/src/main/java/org/apache/streampipes/logging/impl/EventStatisticLogger.java On 2021/10/20 17:42:04, Dominik Riemer <[email protected]> wrote: > Hi Philipp, > > I think we can either catch exceptions somewhere in the upper parts of our > processing chain and send them to a broker topic, where we collect exceptions > similar to the monitoring feature. > We also had the concept of a StreamPipes-specific logging system, which we > can also bring to work so that user-defined logging is available to the core > and user UI. I'm not fully aware of the current status of this logger. > > Dominik > > On 2021/10/20 09:48:01, Philipp Zehnder <[email protected]> wrote: > > Hi Domink, > > > > yes, I think that would also be a good option to transmit the events. > > I was wondering, how can we intercept the exceptions that occur within a > > service? > > > > Philipp > > > > > On 19. Oct 2021, at 22:32, Dominik Riemer <[email protected]> wrote: > > > > > > Hi Philipp, > > > > > > looks good! I wonder if we should also think about monitoring errors in > > > the adapter and pipeline elements and how to submit the stack trace of > > > exceptions (e.g., an adapter or processing element which throws a runtime > > > error). > > > > > > Does it make sense to include an array containing exceptions into the > > > message payload? > > > > > > Dominik > > > > > > On 2021/10/19 14:24:11, Philipp Zehnder <[email protected]> wrote: > > >> Hi all, > > >> > > >> I’d like to discuss a feature to monitore our pipeline elements, like > > >> adapters, processing elements and sinks. > > >> I created a wiki page with a description of the idea [1]. > > >> The main idea is to detect if an element is failing (e.g. an adapter > > >> stops sending events) and notify the admin of the system. > > >> > > >> I’ll create a prototype according to the description in the wiki. If you > > >> have any ideas or feature request please write them so that we can > > >> discuss them. > > >> I am looking forward to your feedback on this feature. > > >> > > >> Philipp > > >> > > >> > > >> [1] > > >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191335025 > > >> > > >> <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191335025> > > >> > > >> > > > > >
