Hi guys,
Thank you for the welcome and the swift feedback. StreamPipes looks like
an excellent tool for making dataops much more manageable, so I am
looking forward to getting properly to grips with it.
It's working again with docker-compose up -d. Thanks.
So I am following the example in Linux Magazine
(https://www.linux-magazine.com/Issues/2021/244/Apache-StreamPipes/(offset)/3),
and have gotten so far as putting together the pipeline. When I tried to
start the pipeline, the following happened:
Pipeline Status
warning
Could not start pipeline ISS Distance and Speed.
Reverse Geocoder at URL
http://pipeline-elements-all-jvm:8090/sepa/org.apache.streampipes.processor.geo.jvm.reversegeocodingdone
Speed Calculator at URL
http://pipeline-elements-all-jvm:8090/sepa/org.apache.streampipes.processors.geo.jvm.processor.speedwarning
status code: 500, reason phrase: Internal Server Error
Static Distance Calculator at URL
http://pipeline-elements-all-jvm:8090/sepa/org.apache.streampipes.processors.geo.jvm.processor.staticdistancecalculatorwarning
status code: 500, reason phrase: Internal Server Error
Numerical Filter at URL
http://pipeline-elements-all-jvm:8090/sepa/org.apache.streampipes.processors.filters.jvm.numericalfilterwarning
status code: 500, reason phrase: Internal Server Error
Dashboard Sink at URL
http://pipeline-elements-all-jvm:8090/sec/org.apache.streampipes.sinks.internal.jvm.dashboardwarning
status code: 500, reason phrase: Internal Server Error
Dashboard Sink at URL
http://pipeline-elements-all-jvm:8090/sec/org.apache.streampipes.sinks.internal.jvm.dashboardwarning
status code: 500, reason phrase: Internal Server Error
Notification at URL
http://pipeline-elements-all-jvm:8090/sec/org.apache.streampipes.sinks.internal.jvm.notificationwarning
status code: 500, reason phrase: Internal Server Error
Only the Reverse Geocoder and the Static Distance Calculator worked. So
I uninstalled and reinstalled all the pipeline elements (all 76). Now,
the Reverse Geocoder and the Dashboard Sinks work, but the rest doesn't.
Help much appreciated.
Cheers,
Eric
On 15/7/21 8:25, Philipp Zehnder wrote:
Hi Eric,
welcome and thanks for writing.
As Patrick mentioned, there is no restart policy defined in the current version
(0.67.0).
We have fixed this for the new version (0.68.0) which will be out soon.
Then the containers will be restarted automatically when the system is rebooted, unless
you stop StreamPipes manually with "docker-compose down".
Philipp
On 14. Jul 2021, at 22:11, Patrick Wiener <[email protected]> wrote:
Hi Eric,
welcome to the mailing list.
Currently, the docker-compose configuration does not handle system reboots, i.e.
upon reboot the containers are not restarted.
You will have to run docker-compose up -d from within the /compose directory to
start
the services. Simply starting only the backend will not work, as StreamPipes
relies on
multiple services (specified within the docker-compose.yml). Hence, the error
that the
backend cannot find consul (see UnknownHostException) as the consul container
is not running.
Hope this’ll help fix your problem.
Best
Patrick
Am 14.07.2021 um 20:21 schrieb Eric Eikrem <[email protected]>:
Hi,
I am new to Streampipes and wanted to try it out. First time around it worked
great. I followed the instructions on https://streampipes.apache.org/download
and had no problems. After rebooting, and revisiting localhost:80, I get stuck
on the welcome page. Note: I am no Docker-expert either...
System: Ubuntu 21.04
Docker version 20.10.7, build f0df350
List of docker images:
REPOSITORY TAG IMAGE ID
CREATED SIZE
rocker/verse latest 9f3753ef6df6 5 weeks
ago 3.67GB
rocker/tidyverse 4.0.0 7f945ed0e6a9 2 months
ago 2.55GB
hello-world latest d1165f221234 4
months ago 13.3kB
fogsyio/zookeeper 3.4.13 bf2c688c6d60 5
months ago 205MB
fogsyio/kafka 2.2.0 be0129895757 5 months
ago 526MB
fogsyio/couchdb 2.3.1 12c3d987f40a 5 months
ago 285MB
fogsyio/consul 1.7.1 bf610ce2b6f0 5 months
ago 116MB
fogsyio/activemq 5.15.9 e38a7c3c6b4c 5
months ago 210MB
fogsyio/influxdb 1.7 1389e2a18e91 5 months ago
311MB
apachestreampipes/sources-watertank-simulator 0.67.0 4a35a41ce2ef 8
months ago 353MB
apachestreampipes/pipeline-elements-all-jvm 0.67.0 01d1a1941e7c 8
months ago 408MB
apachestreampipes/connect-worker 0.67.0 50e2849e85c5 8
months ago 416MB
apachestreampipes/ui 0.67.0 fde8163af024 8
months ago 178MB
apachestreampipes/streampipes-connect-container 0.67.0 159b912b0d39 8
months ago 315MB
apachestreampipes/backend 0.67.0 45d000e041db 8
months ago 342MB
List of active containers: None
Tried to start the apachestreampipes/backend image with the following result:
$ sudo docker run 45d000e041db
16:40:22,960 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
NOT find resource [logback-test.xml]
16:40:22,961 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
NOT find resource [logback.groovy]
16:40:22,961 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found
resource [logback.xml] at
[jar:file:/streampipes-backend.jar!/BOOT-INF/lib/streampipes-commons-0.67.0.jar!/logback.xml]
16:40:23,001 |-INFO in
ch.qos.logback.core.joran.spi.ConfigurationWatchList@84aac766 - URL
[jar:file:/streampipes-backend.jar!/BOOT-INF/lib/streampipes-commons-0.67.0.jar!/logback.xml]
is not of type file
16:40:23,065 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction
- debug attribute not set
16:40:23,067 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About
to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
16:40:23,081 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming
appender as [STDOUT]
16:40:23,150 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This
appender no longer admits a layout as a sub-component, set an encoder instead.
16:40:23,150 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure
compatibility, wrapping your layout in LayoutWrappingEncoder.
16:40:23,150 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also
http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
16:40:23,152 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction -
Setting level of logger [org.apache.streampipes] to INFO
16:40:23,152 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction -
Setting level of logger [org.lightcouch] to ERROR
16:40:23,152 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction -
Setting level of ROOT logger to INFO
16:40:23,152 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction -
Attaching appender named [STDOUT] to Logger[ROOT]
16:40:23,153 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction
- End of configuration.
16:40:23,154 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@71d700fe
- Registering current configuration as safe fallback point
_______ __ ______ __
| __| |_.----.-----.---.-.--------.| __ \__|.-----.-----.-----.
|__ | _| _| -__| _ | || __/ || _ | -__|__ --|
|_______|____|__| |_____|___._|__|__|__||___| |__|| __|_____|_____|
|__|
** StreamPipes Pipeline Management **
16:40:23.967 SP [background-preinit] INFO o.h.validator.internal.util.Version -
HV000001: Hibernate Validator 6.1.5.Final
16:40:24.061 SP [main] INFO o.a.s.b.StreamPipesBackendApplication - Starting
StreamPipesBackendApplication v0.67.0 on 311523f99a52 with PID 1
(/streampipes-backend.jar started by root in /)
16:40:24.061 SP [main] INFO o.a.s.b.StreamPipesBackendApplication - No active
profile set, falling back to default profiles: default
16:40:25.963 SP [main] INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat
initialized with port(s): 8030 (http)
16:40:25.981 SP [main] INFO o.a.coyote.http11.Http11NioProtocol - Initializing
ProtocolHandler ["http-nio-8030"]
16:40:25.981 SP [main] INFO o.a.catalina.core.StandardService - Starting
service [Tomcat]
16:40:25.981 SP [main] INFO o.a.catalina.core.StandardEngine - Starting
Servlet engine: [Apache Tomcat/9.0.37]
16:40:26.090 SP [main] INFO o.a.c.c.C.[.[.[/streampipes-backend] -
Initializing Spring embedded WebApplicationContext
16:40:26.090 SP [main] INFO o.s.b.w.s.c.ServletWebServerApplicationContext -
Root WebApplicationContext: initialization completed in 1965 ms
16:40:26.376 SP [main] INFO o.a.c.c.C.[.[.[/streampipes-backend] -
Initializing Shiro environment
16:40:26.377 SP [main] INFO o.a.shiro.web.env.EnvironmentLoader - Starting
Shiro environment initialization.
16:40:26.551 SP [main] INFO o.a.shiro.web.env.EnvironmentLoader - Shiro
environment initialized in 173 ms.
16:40:26.957 SP [main] ERROR o.a.c.c.C.[.[.[/streampipes-backend] - Exception
sending context initialized event to listener instance of class
[org.apache.streampipes.rest.notifications.NotificationListener]
java.lang.ExceptionInInitializerError: null
at java.lang.J9VMInternals.ensureError(J9VMInternals.java:147)
at
java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:136)
at
org.apache.streampipes.rest.notifications.NotificationListener.contextInitialized(NotificationListener.java:37)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4676)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5139)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at
org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
at
org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:841)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at
org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
at
org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at
org.apache.catalina.core.StandardService.startInternal(StandardService.java:421)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at
org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Tomcat.start(Tomcat.java:486)
at
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:123)
at
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:104)
at
org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:437)
at
org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:191)
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:178)
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:158)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545)
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
at
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758)
at
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750)
at
org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at
org.apache.streampipes.backend.StreamPipesBackendApplication.main(StreamPipesBackendApplication.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:109)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: com.orbitz.consul.ConsulException: Error connecting to Consul
at com.orbitz.consul.AgentClient.ping(AgentClient.java:69)
at com.orbitz.consul.Consul$Builder.build(Consul.java:731)
at
org.apache.streampipes.config.consul.ConsulSpConfig.<init>(ConsulSpConfig.java:60)
at org.apache.streampipes.config.SpConfig.getSpConfig(SpConfig.java:38)
at
org.apache.streampipes.config.backend.BackendConfig.<init>(BackendConfig.java:35)
at
org.apache.streampipes.config.backend.BackendConfig.<clinit>(BackendConfig.java:29)
... 48 common frames omitted
Caused by: java.net.UnknownHostException: consul: System error
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
at java.net.InetAddress.getAllByName(InetAddress.java:1193)
at java.net.InetAddress.getAllByName(InetAddress.java:1127)
at okhttp3.Dns$1.lookup(Dns.java:40)
at
okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)
at
okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149)
at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84)
at
okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:214)
at
okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
at
okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
at
okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at
okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at
okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at
com.orbitz.consul.cache.TimeoutInterceptor.intercept(TimeoutInterceptor.java:53)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at
okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:250)
at okhttp3.RealCall.execute(RealCall.java:93)
at retrofit2.OkHttpCall.execute(OkHttpCall.java:186)
at com.orbitz.consul.AgentClient.ping(AgentClient.java:62)
... 53 common frames omitted
16:40:26.957 SP [main] ERROR o.a.catalina.core.StandardContext - One or more
listeners failed to start. Full details will be found in the appropriate
container log file
16:40:26.957 SP [main] ERROR o.a.catalina.core.StandardContext - Context
[/streampipes-backend] startup failed due to previous errors
16:40:26.958 SP [main] INFO o.a.c.c.C.[.[.[/streampipes-backend] - Cleaning up
Shiro Environment
16:40:26.964 SP [main] INFO o.a.coyote.http11.Http11NioProtocol - Starting
ProtocolHandler ["http-nio-8030"]
16:40:26.982 SP [main] INFO o.a.coyote.http11.Http11NioProtocol - Pausing
ProtocolHandler ["http-nio-8030"]
16:40:26.988 SP [main] INFO o.a.catalina.core.StandardService - Stopping
service [Tomcat]
16:40:26.989 SP [main] INFO o.a.coyote.http11.Http11NioProtocol - Stopping
ProtocolHandler ["http-nio-8030"]
16:40:26.990 SP [main] INFO o.a.coyote.http11.Http11NioProtocol - Destroying
ProtocolHandler ["http-nio-8030"]
16:40:26.990 SP [main] WARN
o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext - Exception
encountered during context initialization - cancelling refresh attempt:
org.springframework.context.ApplicationContextException: Unable to start web
server; nested exception is
org.springframework.boot.web.server.WebServerException: Unable to start
embedded Tomcat
16:40:26.991 SP [main] INFO o.a.s.b.StreamPipesBackendApplication - Shutting
down StreamPipes...
16:40:27.018 SP [main] WARN o.s.c.a.CommonAnnotationBeanPostProcessor - Destroy
method on bean with name 'streamPipesBackendApplication' threw an exception:
java.lang.ExceptionInInitializerError
16:40:27.053 SP [main] INFO o.s.b.a.l.ConditionEvaluationReportLoggingListener -
Error starting ApplicationContext. To display the conditions report re-run your
application with 'debug' enabled.
16:40:27.056 SP [main] ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.context.ApplicationContextException: Unable to start web
server; nested exception is
org.springframework.boot.web.server.WebServerException: Unable to start
embedded Tomcat
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:161)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545)
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
at
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758)
at
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750)
at
org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at
org.apache.streampipes.backend.StreamPipesBackendApplication.main(StreamPipesBackendApplication.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:109)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: org.springframework.boot.web.server.WebServerException: Unable to
start embedded Tomcat
at
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:142)
at
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:104)
at
org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:437)
at
org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:191)
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:178)
at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:158)
... 17 common frames omitted
Caused by: java.lang.IllegalStateException:
StandardEngine[Tomcat].StandardHost[localhost].TomcatEmbeddedContext[/streampipes-backend]
failed to start
at
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.rethrowDeferredStartupExceptions(TomcatWebServer.java:187)
at
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:126)
... 22 common frames omitted
Any help much appreciated.
Cheers,
Eric
--
Programmer and project manager
R, Python and more. Data science.
Site: https://ericeikrem.com
R-blog: https://r-blog.ericeikrem.com
LinkedIn: https://www.linkedin.com/in/ericeikrem/
Twitter: https://twitter.com/ericeikrem
+34 698 160 494 (Spain)
+47 950 80 850 (Norway)
[email protected]
Skype: eric.eikrem
Discord: ElToro1966#5579
Signal: ericeikrem66
Alcalde Suarez Ferrín, 12
ES-15011 - A Coruña
Spain
