Hi Robert, could you please go and check the backend service logs? Maybe it provides more details on why the installation fails.
Patrick > Am 31.08.2021 um 14:49 schrieb Robert Edward Herding > <[email protected]>: > > Hi Patrick and Dominik, > > Thanks for all the patience. I really appreciate the help. > > Yes I am always deleting the volume to have a fresh start. But I still get > the red dot. > Then I tried what Dominik said and now I have a beautiful DataProcessor > Geofencing component :) . > But when I tried to install it: > > Installing 1 of 1 (Geofencing)...error > > More Details: > geofence-tutorial: System error > > Also again I have got this warning (not sure if this is the problem, but it > seems related as it only shows when its time to import the component) > > PM org.glassfish.jersey.server.wadl.WadlFeature configure > WARNING: JAXBContext implementation could not be found. WADL feature is > disabled. > > I will fix this warning and review the program for some mistype (just to be > sure it is perfectly ok) and I hope coming back with good news. > If you guys have any more ideas please let me know. > > > Best regard, > Robert Edward > > 31.08.2021, 07:13, "Dominik Riemer" <[email protected]>: > Hi Robert, > > on some development setups the locally started extensions service is not > detected as a healthy service by Consul and thus not added to the list of > pipeline elements in the UI. > You can also try to click on "Manage Endpoints" and add the URL of the > extensions service manually (just add the URL such as > http://192.168.1.12:8092 <http://192.168.1.12:8092/>). Maybe that's also > worth a try ;-) > > Dominik > > On 2021/08/31 10:25:06, Patrick Wiener <[email protected] > <mailto:[email protected]>> wrote: > > Hi Robert, > > did you also reset everything to a clean slate using the CLI? There might be > some misconfigurations due to your initial settings using „localhost“ that > might not get overwritten. > Hopefully, if you perform the following steps we get you sorted: > > Remove StreamPipes and perform cleanup (don’t forget the -v flag as this is > required to delete the docker volumes): ./streampipes down -v > Restart StreamPipes: ./streampipes up -d > Finish setup process in the SP UI > Start processor from IDE with your 192.168.x.x IP > Check the SP UI at the „Configuration“ > „Pipeline Element Configuration“ > tab and validate the green dot for your new processor service > Go to „Install Pipeline Elements“ and check if you can install the new > processor > > Best > Patrick > > > Am 31.08.2021 um 11:18 schrieb Robert Edward Herding > <[email protected] <mailto:[email protected]>>: > > > > Hello Patrick, > > > > Thanks once more for your answer, very interesting. > > I have set my IP as you said. > > Now I can access the RDF description, only if I set the IP address in the > URL without namespaces, like > "http://192.168.1.12:8092/sepa/org.streampipes.tutorial.geofencing > <http://192.168.1.12:8092/sepa/org.streampipes.tutorial.geofencing>" > > But still doesn't import the processor > > Funny thing I can access the RDF page also with the docker IP > > > > Reading the logs I also found an warning, only when accessing the RDF page: > > > > AM org.glassfish.jersey.server.wadl.WadlFeature configure > > WARNING: JAXBContext implementation could not be found. WADL feature is > disabled. > > > > As you said "manage endpoints" is not connecting to Consul(inside docker), > while RDF description page is(outside docker). > > One question; It is possible to create an external Consul service(in the > cloud) and import from there? That would be the easier/faster way ? > > > > Best regards, > > Robert > > > > > > > > 30.08.2021, 03:27, "Patrick Wiener" <[email protected] > <mailto:[email protected]>>: > > Hi Robert, > > > > good to hear that we got it to register. > > > > The second problem seems to be due to the fact that the service health > check on the processor is failing given that you set the url to > localhost:8092 for SP_HOST. > > Thats also why you see the red dot in the overview. > > > > The reason for this basically is due to the hybrid dev environment with > parts of your setup running in the IDE (the processor) and the other parts > inside Docker (here Consul for > > service discovery and service health check). While connections from your > Docker Host (your Debian machine) work on Consul over localhost:8500 (outside > -> inside Docker) the > > issue occurs when services like Consul try to communicate to other > services running outside Docker, here on your Debian host, especially via > „localhost“. The reason for this is > > that Docker containers run isolated in their own namespace and are not > aware of each other. While docker to docker connections work due to some > convenient help from Docker > > using service DNS names (for containers within the same network), it fails > for development purposes (with some exceptions). Hence, when using > „localhost“ Consul thinks the service > > is also running within the same container as Consul is, which is false, as > it is running on your machine in your IDE without Docker. > > > > So given your setup on a linux machine, what I would recommend for > development is to not use „localhost“ but the actual IP of your linux machine > (see ifconfig). Docker containers > > can communicate to the NIC's (e.g. eth0) IP address. However, this only > works on linux as Docker runs natively. Yet it is worthy to note that if you > regularly change your network environments > > which may lead to different IP addresses assigned to this NIC, the service > health check would start failing again, as the service was originally > registered with another IP. > > We currently work on improving the whole service discovery/service > management part to hopefully resolve all these issues within the current dev > branch which is part of the next release > > (0.69.0). > > > > As of now, see if it helps using the machine’s IP address (e.g., > 192.168.x.x) instead of localhost. > > > > Best > > Patrick > > > > Am 29.08.2021 um 17:26 schrieb Robert Edward Herding > <[email protected] <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>>: > > > > Hello Patrick, > > > > Thank you very much, your explanation helped a lot. > > I was running the compose docker file, not the cli enviroment. > > So I created a pipeline element template with cli, and it worked > perfectly, pretty straightforward. > > Now I have a registered service at Consul. > > > > After that I tried to import the processor in Install Pipeline Elements, > with the url given at http://localhost:8092/ <http://localhost:8092/> > <http://localhost:8092/ <http://localhost:8092/>> (I had to change the port). > > The Install Pipeline Elements shows no elements (it keeps searching) but I > can see "geofence tutorial" in Pipeline Element Configuration with a red dot, > for sure it is not working but it can see the container. > > Can you please give me a north on this? > > > > Yours sincerely, > > > > Robert > > > > > > > > > > > > > > 28.08.2021, 10:37, "Patrick Wiener" <[email protected] > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>>>: > > Hi Robert, > > > > first of all I’d like to welcome you to the list. > > > > I assume you already started the developer environment such that Consul > Docker container is running? > > You can use the StreamPipes CLI to easily configure a development > environment. You can find relevant parts from here [1]. > > > > A little bit of background: we leverage Consul for service discovery such > that all extending services, e.g. processors etc > > first register at Consul from where the StreamPipes core picks up service > endpoints of respective extensions. > > Hence, currently also for development purposes it is necessary to set of a > minimal development environment using the CLI and > > a corresponding environment template. > > > > Can you also provide an excerpt of „docker ps -a“? > > > > Cheers > > Patrick > > > > [1] https://streampipes.apache.org/docs/docs/extend-setup.html > <https://streampipes.apache.org/docs/docs/extend-setup.html> > <https://streampipes.apache.org/docs/docs/extend-setup.html > <https://streampipes.apache.org/docs/docs/extend-setup.html>> > > > > Am 27.08.2021 um 23:19 schrieb Robert Edward Herding > <[email protected] <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>>: > > > > Hello folks, > > > > I am a developer located in South Africa. > > I have been testing StreamPipes for a few weeks now and I have decided to > build, a processor entity for testing and developing for a future industrial > use. > > Said that I start following the tutorial > https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html > <https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html> > > <https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html > > <https://streampipes.apache.org/docs/docs/extend-tutorial-data-processors.html>>. > > The project is building perfectly, but when it gets to the point of > registering items in Consul it just loops with the message. > > > > 22:41:53.553 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > > 22:41:53.554 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying > in 1 second > > 22:41:54.554 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > > 22:41:54.555 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying > in 1 second > > 20:41:55.555 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Trying to > connect to Consul to register config items > > 20:41:55.556 SP [main] INFO o.a.s.config.consul.ConsulSpConfig - Retrying > in 1 second > > > > I am really not used to work with Consul, so have been really hard to > understand why it can't find it. > > I tried to change the SP_HOST in the run configuration. > > > > localhost > > 0.0.0.0 > > 127.0.0.1 > > 192.168.1.1 > > 192.168.1.12 > > host.docker.internal > > > > Below the docker IP and docker network. > > > > udp 0 0 192.168.1.12:bootpc host.docker.inte:bootps ESTABLISHED > > unix 3 [ ] STREAM CONNECTED 701942 > /run/snap.docker/containerd/containerd.sock > > unix 3 [ ] STREAM CONNECTED 692953 > /run/snap.docker/containerd/containerd.sock > > unix 3 [ ] STREAM CONNECTED 703571 > /run/snap.docker/containerd/containerd.sock > > > > > > NETWORK ID NAME DRIVER SCOPE > > 305ea4177bdb bridge bridge local > > 8c6df02eeef9 host host local > > ead9e6d74aa1 none null local > > > > As you can see I am running linux Debian. > > Any help would be greatly appreciated. > > > > Yours sincerely, > > > > Robert > > > >
