Hi Robert, good to hear that we got it to register.
The second problem seems to be due to the fact that the service health check on the processor is failing given that you set the url to localhost:8092 for SP_HOST. Thats also why you see the red dot in the overview. The reason for this basically is due to the hybrid dev environment with parts of your setup running in the IDE (the processor) and the other parts inside Docker (here Consul for service discovery and service health check). While connections from your Docker Host (your Debian machine) work on Consul over localhost:8500 (outside -> inside Docker) the issue occurs when services like Consul try to communicate to other services running outside Docker, here on your Debian host, especially via „localhost“. The reason for this is that Docker containers run isolated in their own namespace and are not aware of each other. While docker to docker connections work due to some convenient help from Docker using service DNS names (for containers within the same network), it fails for development purposes (with some exceptions). Hence, when using „localhost“ Consul thinks the service is also running within the same container as Consul is, which is false, as it is running on your machine in your IDE without Docker. So given your setup on a linux machine, what I would recommend for development is to not use „localhost“ but the actual IP of your linux machine (see ifconfig). Docker containers can communicate to the NIC's (e.g. eth0) IP address. However, this only works on linux as Docker runs natively. Yet it is worthy to note that if you regularly change your network environments which may lead to different IP addresses assigned to this NIC, the service health check would start failing again, as the service was originally registered with another IP. We currently work on improving the whole service discovery/service management part to hopefully resolve all these issues within the current dev branch which is part of the next release (0.69.0). As of now, see if it helps using the machine’s IP address (e.g., 192.168.x.x) instead of localhost. Best Patrick > Am 29.08.2021 um 17:26 schrieb Robert Edward Herding > <[email protected]>: > > Hello Patrick, > > Thank you very much, your explanation helped a lot. > I was running the compose docker file, not the cli enviroment. > So I created a pipeline element template with cli, and it worked perfectly, > pretty straightforward. > Now I have a registered service at Consul. > > After that I tried to import the processor in Install Pipeline Elements, with > the url given at http://localhost:8092/ <http://localhost:8092/> (I had to > change the port). > The Install Pipeline Elements shows no elements (it keeps searching) but I > can see "geofence tutorial" in Pipeline Element Configuration with a red dot, > for sure it is not working but it can see the container. > Can you please give me a north on this? > > Yours sincerely, > > Robert > > > > > > > 28.08.2021, 10:37, "Patrick Wiener" <[email protected]>: > Hi Robert, > > first of all I’d like to welcome you to the list. > > I assume you already started the developer environment such that Consul > Docker container is running? > You can use the StreamPipes CLI to easily configure a development > environment. You can find relevant parts from here [1]. > > A little bit of background: we leverage Consul for service discovery such > that all extending services, e.g. processors etc > first register at Consul from where the StreamPipes core picks up service > endpoints of respective extensions. > Hence, currently also for development purposes it is necessary to set of a > minimal development environment using the CLI and > a corresponding environment template. > > Can you also provide an excerpt of „docker ps -a“? > > Cheers > Patrick > > [1] https://streampipes.apache.org/docs/docs/extend-setup.html > <https://streampipes.apache.org/docs/docs/extend-setup.html> > > Am 27.08.2021 um 23:19 schrieb Robert Edward Herding > <[email protected] <mailto:[email protected]>>: > > Hello folks, > > I am a developer located in South Africa. > I have been testing StreamPipes for a few weeks now and I have decided to > build, a processor entity for testing and developing for a future industrial > use. > Said that I start following the tutorial > https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html > <https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html>. > The project is building perfectly, but when it gets to the point of > registering items in Consul it just loops with the message. > > 22:41:53.553 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > 22:41:53.554 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying in > 1 second > 22:41:54.554 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > 22:41:54.555 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying in > 1 second > 20:41:55.555 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > 20:41:55.556 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying in > 1 second > > I am really not used to work with Consul, so have been really hard to > understand why it can't find it. > I tried to change the SP_HOST in the run configuration. > > localhost > 0.0.0.0 > 127.0.0.1 > 192.168.1.1 > 192.168.1.12 > host.docker.internal > > Below the docker IP and docker network. > > udp 0 0 192.168.1.12:bootpc host.docker.inte:bootps > ESTABLISHED > unix 3 [ ] STREAM CONNECTED 701942 > /run/snap.docker/containerd/containerd.sock > unix 3 [ ] STREAM CONNECTED 692953 > /run/snap.docker/containerd/containerd.sock > unix 3 [ ] STREAM CONNECTED 703571 > /run/snap.docker/containerd/containerd.sock > > > NETWORK ID NAME DRIVER SCOPE > 305ea4177bdb bridge bridge local > 8c6df02eeef9 host host local > ead9e6d74aa1 none null local > > As you can see I am running linux Debian. > Any help would be greatly appreciated. > > Yours sincerely, > > Robert >
