Re: [ClusterLabs] how to connect to the cluster from a docker container
Hi, On Tue, Aug 06, 2019 at 01:36:49PM +0200, Jan Pokorný wrote: > Hello Dejan, > > nice to see you around, > > On 06/08/19 10:37 +0200, Dejan Muhamedagic wrote: > > Hawk runs in a docker container on one of the cluster nodes (the > > nodes run Debian and apparently it's rather difficult to install > > hawk on a non-SUSE distribution, hence docker). Now, how to > > connect to the cluster? Hawk uses the pacemaker command line > > tools such as cibadmin. I have a vague recollection that there is > > a way to connect over tcp/ip, but, if that is so, I cannot find > > any documentation about it. > [...] > 2. use modern enough libqb (v1.0.2+) and use > > touch /etc/libqb/force-filesystem-sockets > >on both host and within the container (assuming those two locations >are fully disjoint, i.e., not an overlay-based reuse), you should >then be able to share the respective reified sockets simply by >sharing the pertaining directory (normally /var/run it seems) > >- if indeed a directory as generic as /var/run is involved, > it may also lead to unexpected interferences, so the more > minimalistic the container is, the better I think > (or you can recompile libqb and play with path mapping > in container configuration to achieve smoother plug-in) > > Then, pacemaker utilities would hopefully work across the container > boundaries just as if they were fully native, hence hawk shall as > well. > > Let us know how far you'll get and where we can colletively join you > in your attempts, I don't think we had such experience disseminated > here. I know for sure I haven't ever tried this in practice, some > one else here could have. Also, there may be a lot of fun with various > Linux Security Modules like SELinux. pacemakerd is not happy with the filesystem sockets: Aug 07 14:12:26 alpaca1-pc pacemakerd [7606] (crm_ipc_connect) debug: Could not establish pacemakerd connection: No such file or directory (2) Aug 07 14:12:26 alpaca1-pc pacemakerd [7606] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect() Aug 07 14:12:26 alpaca1-pc pacemakerd [7606] (mcp_read_config) info: cmap connection setup failed: CS_ERR_NOT_EXIST . Retrying in 1s ... Aug 7 14:12:41 alpaca1-pc pacemakerd[7606]: Could not connect to Cluster Config uration Database API, error 12 Apparently, it fails to connect to corosync. Any ideas? Cheers, Dejan ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] how to connect to the cluster from a docker container
Hi Ken, On Tue, Aug 06, 2019 at 08:58:20AM -0500, Ken Gaillot wrote: > On Tue, 2019-08-06 at 14:03 +0200, Jan Pokorný wrote: > > On 06/08/19 13:36 +0200, Jan Pokorný wrote: > > > On 06/08/19 10:37 +0200, Dejan Muhamedagic wrote: > > > > Hawk runs in a docker container on one of the cluster nodes (the > > > > nodes run Debian and apparently it's rather difficult to install > > > > hawk on a non-SUSE distribution, hence docker). Now, how to > > > > connect to the cluster? Hawk uses the pacemaker command line > > > > tools such as cibadmin. I have a vague recollection that there is > > > > a way to connect over tcp/ip, but, if that is so, I cannot find > > > > any documentation about it. > > I think one of the solutions Jan suggested would be best, but what > you're likely remembering is remote-tls-port: > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Administration/#s-remote-connection > > However that only works for the CIB, so anything that needed to contact > other daemons wouldn't work. Right, that's what I couldn't recall! I'm not sure if hawk uses anything other than the connection to the cib. Cheers, Dejan > > > > > > I think that what you are after is one of: > > > > > > 1. have docker runtime for the particular container have the > > > abstract > > >Unix sockets shared from the host (--network=host? don't > > > remember > > >exactly) > > > > > >- apparently, this weak style of compartmentalization comes with > > > many drawbacks, so you may be facing hefty work of cutting any > > > other interferences stemming from pre-chrooting assumptions of > > > what is a singleton on the system, incl. sockets etc. > > > > > > 2. use modern enough libqb (v1.0.2+) and use > > > > > > touch /etc/libqb/force-filesystem-sockets > > > > > >on both host and within the container (assuming those two > > > locations > > >are fully disjoint, i.e., not an overlay-based reuse), you > > > should > > >then be able to share the respective reified sockets simply by > > >sharing the pertaining directory (normally /var/run it seems) > > > > > >- if indeed a directory as generic as /var/run is involved, > > > it may also lead to unexpected interferences, so the more > > > minimalistic the container is, the better I think > > > (or you can recompile libqb and play with path mapping > > > in container configuration to achieve smoother plug-in) > > > > Oh, and there's additional prerequisite for both to at least > > theoretically work -- 1:1 sharing of /dev/shm (which may also > > be problematic in a sense). > > > > > Then, pacemaker utilities would hopefully work across the container > > > boundaries just as if they were fully native, hence hawk shall as > > > well. > > > > > > Let us know how far you'll get and where we can colletively join > > > you > > > in your attempts, I don't think we had such experience disseminated > > > here. I know for sure I haven't ever tried this in practice, some > > > one else here could have. Also, there may be a lot of fun with > > > various > > > Linux Security Modules like SELinux. > > > > ___ > > Manage your subscription: > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > ClusterLabs home: https://www.clusterlabs.org/ > -- > Ken Gaillot > > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] how to connect to the cluster from a docker container
Hi Jan, On Tue, Aug 06, 2019 at 01:36:49PM +0200, Jan Pokorný wrote: > Hello Dejan, > > nice to see you around, Nice to see you too. > On 06/08/19 10:37 +0200, Dejan Muhamedagic wrote: > > Hawk runs in a docker container on one of the cluster nodes (the > > nodes run Debian and apparently it's rather difficult to install > > hawk on a non-SUSE distribution, hence docker). Now, how to > > connect to the cluster? Hawk uses the pacemaker command line > > tools such as cibadmin. I have a vague recollection that there is > > a way to connect over tcp/ip, but, if that is so, I cannot find > > any documentation about it. > > I think that what you are after is one of: > > 1. have docker runtime for the particular container have the abstract >Unix sockets shared from the host (--network=host? don't remember >exactly) > >- apparently, this weak style of compartmentalization comes with > many drawbacks, so you may be facing hefty work of cutting any > other interferences stemming from pre-chrooting assumptions of > what is a singleton on the system, incl. sockets etc. > > 2. use modern enough libqb (v1.0.2+) and use > > touch /etc/libqb/force-filesystem-sockets > >on both host and within the container (assuming those two locations >are fully disjoint, i.e., not an overlay-based reuse), you should >then be able to share the respective reified sockets simply by >sharing the pertaining directory (normally /var/run it seems) > >- if indeed a directory as generic as /var/run is involved, > it may also lead to unexpected interferences, so the more > minimalistic the container is, the better I think > (or you can recompile libqb and play with path mapping > in container configuration to achieve smoother plug-in) > > Then, pacemaker utilities would hopefully work across the container > boundaries just as if they were fully native, hence hawk shall as > well. > > Let us know how far you'll get and where we can colletively join you > in your attempts, I don't think we had such experience disseminated > here. I know for sure I haven't ever tried this in practice, some > one else here could have. Also, there may be a lot of fun with various > Linux Security Modules like SELinux. That system is out of sorts right now, but will give the filesystem sockets a try. Many thanks! Cheers, Dejan > -- > Jan (Poki) > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] how to connect to the cluster from a docker container
On Tue, 2019-08-06 at 14:03 +0200, Jan Pokorný wrote: > On 06/08/19 13:36 +0200, Jan Pokorný wrote: > > On 06/08/19 10:37 +0200, Dejan Muhamedagic wrote: > > > Hawk runs in a docker container on one of the cluster nodes (the > > > nodes run Debian and apparently it's rather difficult to install > > > hawk on a non-SUSE distribution, hence docker). Now, how to > > > connect to the cluster? Hawk uses the pacemaker command line > > > tools such as cibadmin. I have a vague recollection that there is > > > a way to connect over tcp/ip, but, if that is so, I cannot find > > > any documentation about it. I think one of the solutions Jan suggested would be best, but what you're likely remembering is remote-tls-port: https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Administration/#s-remote-connection However that only works for the CIB, so anything that needed to contact other daemons wouldn't work. > > > > I think that what you are after is one of: > > > > 1. have docker runtime for the particular container have the > > abstract > >Unix sockets shared from the host (--network=host? don't > > remember > >exactly) > > > >- apparently, this weak style of compartmentalization comes with > > many drawbacks, so you may be facing hefty work of cutting any > > other interferences stemming from pre-chrooting assumptions of > > what is a singleton on the system, incl. sockets etc. > > > > 2. use modern enough libqb (v1.0.2+) and use > > > > touch /etc/libqb/force-filesystem-sockets > > > >on both host and within the container (assuming those two > > locations > >are fully disjoint, i.e., not an overlay-based reuse), you > > should > >then be able to share the respective reified sockets simply by > >sharing the pertaining directory (normally /var/run it seems) > > > >- if indeed a directory as generic as /var/run is involved, > > it may also lead to unexpected interferences, so the more > > minimalistic the container is, the better I think > > (or you can recompile libqb and play with path mapping > > in container configuration to achieve smoother plug-in) > > Oh, and there's additional prerequisite for both to at least > theoretically work -- 1:1 sharing of /dev/shm (which may also > be problematic in a sense). > > > Then, pacemaker utilities would hopefully work across the container > > boundaries just as if they were fully native, hence hawk shall as > > well. > > > > Let us know how far you'll get and where we can colletively join > > you > > in your attempts, I don't think we had such experience disseminated > > here. I know for sure I haven't ever tried this in practice, some > > one else here could have. Also, there may be a lot of fun with > > various > > Linux Security Modules like SELinux. > > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] how to connect to the cluster from a docker container
On 06/08/19 13:36 +0200, Jan Pokorný wrote: > On 06/08/19 10:37 +0200, Dejan Muhamedagic wrote: >> Hawk runs in a docker container on one of the cluster nodes (the >> nodes run Debian and apparently it's rather difficult to install >> hawk on a non-SUSE distribution, hence docker). Now, how to >> connect to the cluster? Hawk uses the pacemaker command line >> tools such as cibadmin. I have a vague recollection that there is >> a way to connect over tcp/ip, but, if that is so, I cannot find >> any documentation about it. > > I think that what you are after is one of: > > 1. have docker runtime for the particular container have the abstract >Unix sockets shared from the host (--network=host? don't remember >exactly) > >- apparently, this weak style of compartmentalization comes with > many drawbacks, so you may be facing hefty work of cutting any > other interferences stemming from pre-chrooting assumptions of > what is a singleton on the system, incl. sockets etc. > > 2. use modern enough libqb (v1.0.2+) and use > > touch /etc/libqb/force-filesystem-sockets > >on both host and within the container (assuming those two locations >are fully disjoint, i.e., not an overlay-based reuse), you should >then be able to share the respective reified sockets simply by >sharing the pertaining directory (normally /var/run it seems) > >- if indeed a directory as generic as /var/run is involved, > it may also lead to unexpected interferences, so the more > minimalistic the container is, the better I think > (or you can recompile libqb and play with path mapping > in container configuration to achieve smoother plug-in) Oh, and there's additional prerequisite for both to at least theoretically work -- 1:1 sharing of /dev/shm (which may also be problematic in a sense). > Then, pacemaker utilities would hopefully work across the container > boundaries just as if they were fully native, hence hawk shall as > well. > > Let us know how far you'll get and where we can colletively join you > in your attempts, I don't think we had such experience disseminated > here. I know for sure I haven't ever tried this in practice, some > one else here could have. Also, there may be a lot of fun with various > Linux Security Modules like SELinux. -- Jan (Poki) pgpqEk3Xjsvnq.pgp Description: PGP signature ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] how to connect to the cluster from a docker container
Hello Dejan, nice to see you around, On 06/08/19 10:37 +0200, Dejan Muhamedagic wrote: > Hawk runs in a docker container on one of the cluster nodes (the > nodes run Debian and apparently it's rather difficult to install > hawk on a non-SUSE distribution, hence docker). Now, how to > connect to the cluster? Hawk uses the pacemaker command line > tools such as cibadmin. I have a vague recollection that there is > a way to connect over tcp/ip, but, if that is so, I cannot find > any documentation about it. I think that what you are after is one of: 1. have docker runtime for the particular container have the abstract Unix sockets shared from the host (--network=host? don't remember exactly) - apparently, this weak style of compartmentalization comes with many drawbacks, so you may be facing hefty work of cutting any other interferences stemming from pre-chrooting assumptions of what is a singleton on the system, incl. sockets etc. 2. use modern enough libqb (v1.0.2+) and use touch /etc/libqb/force-filesystem-sockets on both host and within the container (assuming those two locations are fully disjoint, i.e., not an overlay-based reuse), you should then be able to share the respective reified sockets simply by sharing the pertaining directory (normally /var/run it seems) - if indeed a directory as generic as /var/run is involved, it may also lead to unexpected interferences, so the more minimalistic the container is, the better I think (or you can recompile libqb and play with path mapping in container configuration to achieve smoother plug-in) Then, pacemaker utilities would hopefully work across the container boundaries just as if they were fully native, hence hawk shall as well. Let us know how far you'll get and where we can colletively join you in your attempts, I don't think we had such experience disseminated here. I know for sure I haven't ever tried this in practice, some one else here could have. Also, there may be a lot of fun with various Linux Security Modules like SELinux. -- Jan (Poki) pgpLs2h1wuNFz.pgp Description: PGP signature ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
[ClusterLabs] how to connect to the cluster from a docker container
Hi, Hawk runs in a docker container on one of the cluster nodes (the nodes run Debian and apparently it's rather difficult to install hawk on a non-SUSE distribution, hence docker). Now, how to connect to the cluster? Hawk uses the pacemaker command line tools such as cibadmin. I have a vague recollection that there is a way to connect over tcp/ip, but, if that is so, I cannot find any documentation about it. Cheers, Dejan ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
