Hi Robert, did you also reset everything to a clean slate using the CLI? There might be some misconfigurations due to your initial settings using „localhost“ that might not get overwritten. Hopefully, if you perform the following steps we get you sorted:
Remove StreamPipes and perform cleanup (don’t forget the -v flag as this is required to delete the docker volumes): ./streampipes down -v Restart StreamPipes: ./streampipes up -d Finish setup process in the SP UI Start processor from IDE with your 192.168.x.x IP Check the SP UI at the „Configuration“ > „Pipeline Element Configuration“ tab and validate the green dot for your new processor service Go to „Install Pipeline Elements“ and check if you can install the new processor Best Patrick > Am 31.08.2021 um 11:18 schrieb Robert Edward Herding > <[email protected]>: > > Hello Patrick, > > Thanks once more for your answer, very interesting. > I have set my IP as you said. > Now I can access the RDF description, only if I set the IP address in the URL > without namespaces, like > "http://192.168.1.12:8092/sepa/org.streampipes.tutorial.geofencing"; > But still doesn't import the processor > Funny thing I can access the RDF page also with the docker IP > > Reading the logs I also found an warning, only when accessing the RDF page: > > AM org.glassfish.jersey.server.wadl.WadlFeature configure > WARNING: JAXBContext implementation could not be found. WADL feature is > disabled. > > As you said "manage endpoints" is not connecting to Consul(inside docker), > while RDF description page is(outside docker). > One question; It is possible to create an external Consul service(in the > cloud) and import from there? That would be the easier/faster way ? > > Best regards, > Robert > > > > 30.08.2021, 03:27, "Patrick Wiener" <[email protected]>: > Hi Robert, > > good to hear that we got it to register. > > The second problem seems to be due to the fact that the service health check > on the processor is failing given that you set the url to localhost:8092 for > SP_HOST. > Thats also why you see the red dot in the overview. > > The reason for this basically is due to the hybrid dev environment with parts > of your setup running in the IDE (the processor) and the other parts inside > Docker (here Consul for > service discovery and service health check). While connections from your > Docker Host (your Debian machine) work on Consul over localhost:8500 (outside > -> inside Docker) the > issue occurs when services like Consul try to communicate to other services > running outside Docker, here on your Debian host, especially via „localhost“. > The reason for this is > that Docker containers run isolated in their own namespace and are not aware > of each other. While docker to docker connections work due to some convenient > help from Docker > using service DNS names (for containers within the same network), it fails > for development purposes (with some exceptions). Hence, when using > „localhost“ Consul thinks the service > is also running within the same container as Consul is, which is false, as it > is running on your machine in your IDE without Docker. > > So given your setup on a linux machine, what I would recommend for > development is to not use „localhost“ but the actual IP of your linux machine > (see ifconfig). Docker containers > can communicate to the NIC's (e.g. eth0) IP address. However, this only works > on linux as Docker runs natively. Yet it is worthy to note that if you > regularly change your network environments > which may lead to different IP addresses assigned to this NIC, the service > health check would start failing again, as the service was originally > registered with another IP. > We currently work on improving the whole service discovery/service management > part to hopefully resolve all these issues within the current dev branch > which is part of the next release > (0.69.0). > > As of now, see if it helps using the machine’s IP address (e.g., 192.168.x.x) > instead of localhost. > > Best > Patrick > > Am 29.08.2021 um 17:26 schrieb Robert Edward Herding > <[email protected] <mailto:[email protected]>>: > > Hello Patrick, > > Thank you very much, your explanation helped a lot. > I was running the compose docker file, not the cli enviroment. > So I created a pipeline element template with cli, and it worked perfectly, > pretty straightforward. > Now I have a registered service at Consul. > > After that I tried to import the processor in Install Pipeline Elements, with > the url given at http://localhost:8092/ <http://localhost:8092/> (I had to > change the port). > The Install Pipeline Elements shows no elements (it keeps searching) but I > can see "geofence tutorial" in Pipeline Element Configuration with a red dot, > for sure it is not working but it can see the container. > Can you please give me a north on this? > > Yours sincerely, > > Robert > > > > > > > 28.08.2021, 10:37, "Patrick Wiener" <[email protected] > <mailto:[email protected]>>: > Hi Robert, > > first of all I’d like to welcome you to the list. > > I assume you already started the developer environment such that Consul > Docker container is running? > You can use the StreamPipes CLI to easily configure a development > environment. You can find relevant parts from here [1]. > > A little bit of background: we leverage Consul for service discovery such > that all extending services, e.g. processors etc > first register at Consul from where the StreamPipes core picks up service > endpoints of respective extensions. > Hence, currently also for development purposes it is necessary to set of a > minimal development environment using the CLI and > a corresponding environment template. > > Can you also provide an excerpt of „docker ps -a“? > > Cheers > Patrick > > [1] https://streampipes.apache.org/docs/docs/extend-setup.html > <https://streampipes.apache.org/docs/docs/extend-setup.html> > > Am 27.08.2021 um 23:19 schrieb Robert Edward Herding > <[email protected] <mailto:[email protected]>>: > > Hello folks, > > I am a developer located in South Africa. > I have been testing StreamPipes for a few weeks now and I have decided to > build, a processor entity for testing and developing for a future industrial > use. > Said that I start following the tutorial > https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html > <https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html>. > The project is building perfectly, but when it gets to the point of > registering items in Consul it just loops with the message. > > 22:41:53.553 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > 22:41:53.554 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying in > 1 second > 22:41:54.554 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > 22:41:54.555 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying in > 1 second > 20:41:55.555 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > 20:41:55.556 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying in > 1 second > > I am really not used to work with Consul, so have been really hard to > understand why it can't find it. > I tried to change the SP_HOST in the run configuration. > > localhost > 0.0.0.0 > 127.0.0.1 > 192.168.1.1 > 192.168.1.12 > host.docker.internal > > Below the docker IP and docker network. > > udp 0 0 192.168.1.12:bootpc host.docker.inte:bootps > ESTABLISHED > unix 3 [ ] STREAM CONNECTED 701942 > /run/snap.docker/containerd/containerd.sock > unix 3 [ ] STREAM CONNECTED 692953 > /run/snap.docker/containerd/containerd.sock > unix 3 [ ] STREAM CONNECTED 703571 > /run/snap.docker/containerd/containerd.sock > > > NETWORK ID NAME DRIVER SCOPE > 305ea4177bdb bridge bridge local > 8c6df02eeef9 host host local > ead9e6d74aa1 none null local > > As you can see I am running linux Debian. > Any help would be greatly appreciated. > > Yours sincerely, > > Robert >
