GitHub user tenthe added a comment to the discussion: Monitoring and Analytics Built into Streampipes
Hi @heisenbergs-uncertainty, Thanks for bringing this up — I also see a lot of value in improving monitoring and observability in StreamPipes. Like you mentioned, I also think we should first align on what kind of monitoring functionality should be built directly into StreamPipes, and what should be delegated to external tools like Grafana, before jumping into implementation. There are already some built-in monitoring features that might serve as a good starting point. For example, when you go to *Connect* or *Pipelines* and click on `Show details`, you'll see: * A **metrics tab** * A **logs tab** showing output from the extension service Here are some existing classes that could serve as a starting point: * `AbstractMonitoringResource` * `AdapterMonitoringResource` * `PipelineMonitoring` * `ExtensionsLogProvider` We could consider extending these components to support some of your ideas. I particularly like the idea of **notifying when an adapter stops emitting data** — for example via email — as an early step. That could be quite helpful operationally. Let’s maybe collect and prioritize use cases (e.g., adapter throughput, Kafka topic sizes, processor-level metrics), and then see how far we can get with extending the existing functionality before introducing new components. What do you think? GitHub link: https://github.com/apache/streampipes/discussions/3662#discussioncomment-13445719 ---- This is an automatically sent email for dev@streampipes.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@streampipes.apache.org
