Hi Philipp, good point - I've directly added this and the adapter overview now shows the time of the last event as well as the number of events sent.
Dominik On 2022/10/21 15:59:14 Philipp Zehnder wrote: > Hi Dominik, > > I really like this feature. For me, the view is now much clearer. > > I have a suggestion for a further improvement. Can we integrate the date of > the last published event into the “My adapters” view for each adapter. > This would really help to directly see if all adapters are still running as > expected. > Maybe we can also directly include a rule into the UI that color codes when > the last event exceeded a certain threshold. > > What do you think? > > Philipp > > Von: Dominik Riemer <[email protected]> > Datum: Mittwoch, 19. Oktober 2022 um 14:37 > An: [email protected] <[email protected]> > Betreff: Pipeline Monitoring / STREAMPIPES-598 > Hi, > > I've worked on a new feature to collect monitoring infos for pipelines and > adapters. > In the current release, only very basic monitoring info was displayed for > running pipelines, which only worked when using Kafka as message broker and > often displayed strange values as we were receiving the info directly from > the Kafka admin client. The new implementation, independent from the > underlying broker, collects metrics for adapters and pipeline elements > (messages received, messages sent and last timestamp of received messages) > and makes runtime exceptions available in the UI. Exceptions are shown in the > "Logs" section within the details view of pipelines and adapters. On the SDK > side, there is also a new method in the RuntimeContext of processors and > sinks to forward custom exceptions to the core. > > I hope this greatly improves transparency for users as runtime exceptions > were previously only available in the log files. > > There are two things left: > > * I know that some community members have asked for a central REST > endpoint which provides monitoring info in a single document, e.g., to read > this from external monitoring software. This shouldn't be hard to implement > now and I'm happy to assist. > * Exceptions for adapters are currently very basic, as there is now easy > way to globally catch exceptions from adapters. I'll try to improve the > StreamPipes Connect API and codebase first, which should then also ease the > definition of integration tests Zike has started to work on. > > I'll merge the changes into dev soon and I'm happy for any feedback! > > Cheers > Dominik > > >
