Re: [openstack-dev] [kolla][tc] add service discovery, proxysql, vault, fabio and FQDN endpoints
On 10/11/2018 01:08 PM, Zane Bitter wrote: On 10/10/18 1:35 PM, Jay Pipes wrote: +tc topic On 10/10/2018 11:49 AM, Fox, Kevin M wrote: Sorry. Couldn't quite think of the name. I was meaning, openstack project tags. I think having a tag that indicates the project is no longer using SELECT FOR UPDATE (and thus is safe to use multi-writer Galera) is an excellent idea, Kevin. ++ I would support such a tag, especially if it came with detailed instructions on how to audit your code to make sure you are not doing this with sqlalchemy. (Bonus points for a flake8 plugin that can be enabled in the gate.) I can contribute to such a tag's documentation, but I don't currently have the bandwidth to start and shepherd it. (One question for clarification: is this actually _required_ to use multi-writer Galera? My previous recollection was that it was possible, but inefficient, to use SELECT FOR UPDATE safely as long as you wrote a lot of boilerplate to restart the transaction if it failed.) Certainly not. There is just a higher occurrence of the deadlock error in question when using SELECT FOR UPDATE versus using a compare-and-swap technique that does things like this: UPDATE tbl SET field = value, generation = generation + 1 WHERE generation = $expected_generation; The vast majority of cases I've seen where the deadlock occurred were during Rally tests, which were just brute-forcing breakage points and not particularly reflecting a real-world usage pattern. So, in short, yes, it's perfectly safe and fine to use Galera in a multi-writer setup from the get-go with most OpenStack projects. It's just that *some* OpenStack projects of later releases have fewer code areas that aggravate the aforementioned deadlock conditions with Galera in multi-writer mode. Best, -jay -jay From: Jay Pipes [jaypi...@gmail.com] Sent: Tuesday, October 09, 2018 12:22 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints On 10/09/2018 03:10 PM, Fox, Kevin M wrote: Oh, this does raise an interesting question... Should such information be reported by the projects up to users through labels? Something like, "percona_multimaster=safe" Its really difficult for folks to know which projects can and can not be used that way currently. Are you referring to k8s labels/selectors? or are you referring to project tags (you know, part of that whole Big Tent thing...)? -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [kolla][tc] add service discovery, proxysql, vault, fabio and FQDN endpoints
My understanding is it is still safeish to use when you deal with it right. it causes a transaction abort if the race condition ever hits, and you can keep retrying until your commit makes it. So, there are two issues here: 1. its a more rare kind of abort, so unless you are testing and retrying, it can cause operations to fail in a way the user might notice needlessly. This is bad. It should be tested for in the gate. 2. in highly contended systems, it can be a performance issue. This is less bad then #1. For certain codes, it may never be a problem. Thanks, Kevin From: Zane Bitter [zbit...@redhat.com] Sent: Thursday, October 11, 2018 10:08 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [kolla][tc] add service discovery, proxysql, vault, fabio and FQDN endpoints On 10/10/18 1:35 PM, Jay Pipes wrote: > +tc topic > > On 10/10/2018 11:49 AM, Fox, Kevin M wrote: >> Sorry. Couldn't quite think of the name. I was meaning, openstack >> project tags. > > I think having a tag that indicates the project is no longer using > SELECT FOR UPDATE (and thus is safe to use multi-writer Galera) is an > excellent idea, Kevin. ++ I would support such a tag, especially if it came with detailed instructions on how to audit your code to make sure you are not doing this with sqlalchemy. (Bonus points for a flake8 plugin that can be enabled in the gate.) (One question for clarification: is this actually _required_ to use multi-writer Galera? My previous recollection was that it was possible, but inefficient, to use SELECT FOR UPDATE safely as long as you wrote a lot of boilerplate to restart the transaction if it failed.) > -jay > >> >> From: Jay Pipes [jaypi...@gmail.com] >> Sent: Tuesday, October 09, 2018 12:22 PM >> To: openstack-dev@lists.openstack.org >> Subject: Re: [openstack-dev] [kolla] add service discovery, proxysql, >> vault, fabio and FQDN endpoints >> >> On 10/09/2018 03:10 PM, Fox, Kevin M wrote: >>> Oh, this does raise an interesting question... Should such >>> information be reported by the projects up to users through labels? >>> Something like, "percona_multimaster=safe" Its really difficult for >>> folks to know which projects can and can not be used that way currently. >> >> Are you referring to k8s labels/selectors? or are you referring to >> project tags (you know, part of that whole Big Tent thing...)? >> >> -jay >> >> __ >> >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __ >> >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [kolla][tc] add service discovery, proxysql, vault, fabio and FQDN endpoints
On 10/10/18 1:35 PM, Jay Pipes wrote: +tc topic On 10/10/2018 11:49 AM, Fox, Kevin M wrote: Sorry. Couldn't quite think of the name. I was meaning, openstack project tags. I think having a tag that indicates the project is no longer using SELECT FOR UPDATE (and thus is safe to use multi-writer Galera) is an excellent idea, Kevin. ++ I would support such a tag, especially if it came with detailed instructions on how to audit your code to make sure you are not doing this with sqlalchemy. (Bonus points for a flake8 plugin that can be enabled in the gate.) (One question for clarification: is this actually _required_ to use multi-writer Galera? My previous recollection was that it was possible, but inefficient, to use SELECT FOR UPDATE safely as long as you wrote a lot of boilerplate to restart the transaction if it failed.) -jay From: Jay Pipes [jaypi...@gmail.com] Sent: Tuesday, October 09, 2018 12:22 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints On 10/09/2018 03:10 PM, Fox, Kevin M wrote: Oh, this does raise an interesting question... Should such information be reported by the projects up to users through labels? Something like, "percona_multimaster=safe" Its really difficult for folks to know which projects can and can not be used that way currently. Are you referring to k8s labels/selectors? or are you referring to project tags (you know, part of that whole Big Tent thing...)? -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [kolla][tc] add service discovery, proxysql, vault, fabio and FQDN endpoints
+tc topic On 10/10/2018 11:49 AM, Fox, Kevin M wrote: Sorry. Couldn't quite think of the name. I was meaning, openstack project tags. I think having a tag that indicates the project is no longer using SELECT FOR UPDATE (and thus is safe to use multi-writer Galera) is an excellent idea, Kevin. ++ -jay From: Jay Pipes [jaypi...@gmail.com] Sent: Tuesday, October 09, 2018 12:22 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints On 10/09/2018 03:10 PM, Fox, Kevin M wrote: Oh, this does raise an interesting question... Should such information be reported by the projects up to users through labels? Something like, "percona_multimaster=safe" Its really difficult for folks to know which projects can and can not be used that way currently. Are you referring to k8s labels/selectors? or are you referring to project tags (you know, part of that whole Big Tent thing...)? -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
