Re: [Wikitech-l] Datacenter switchover and switchback
I am sorry to hear that. It looks like something that we will have to take into account for the next switchovers. That being said, we had deliberations across the involved teams months ago to come up with those exact dates and have been communicating them via at least SoS since 2018-08-01. I am curious about something though. How does the deployment train (cause that's what we are talking about) impact the software release exactly ? On Wed, Oct 3, 2018 at 6:19 PM C. Scott Ananian wrote: > > Oct 8 seems to be a particularly bad time to freeze the train given that we > are forking for the MW 1.32 release on Oct 15, and a lot of folks have > last-minute things they want to get into the release (eg deprecations, etc). > --scott > > On Thu, Aug 30, 2018 at 10:57 AM Pine W wrote: > > > +1 to DJ's question about timing. Also, one might wish to be mindful of > > the number of recent trains that were supposed to be boring but involved > > interesting surprises; this makes me wonder whether trains that one thinks > > will be boring are actually OK in this circumstance even if they turn out > > to be "interesting". > > > > Pine > > ( https://meta.wikimedia.org/wiki/User:Pine ) > > > > > > > > Original message From: Derk-Jan Hartman < > > d.j.hartman+wmf...@gmail.com> Date: 8/30/18 2:54 AM (GMT-08:00) To: > > Wikimedia developers Subject: Re: > > [Wikitech-l] Datacenter switchover and switchback > > While I think these regular switches are a very good idea, from an outside > > perspective I do have to question a process that puts a significant plug in > > the velocity of various teams working on major projects (esp. in a time of > > year that could probably be seen as one of the most productive). What are > > plans to reduce the disruption of this exercise in the future ? > > > > DJ > > > > On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo > > wrote: > > > > > Let me explain the rationale of the bellow request for clarification: > > > > > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > > > > > Hello: > > > > > > > > >For the duration of the switchover (1 month), deployers are kindly > > > > >requested to refrain from large db schema changes and avoid deploying > > > > >any kind of new feature that requires creation of tables. > > > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > > > > > > > During the failover, some schema changes will be finalized on the current > > > active datacenter (plus some major server and network maintenance may be > > > done)- our request is mostly to refrain from quickly enabling those large > > > new unlocked features (e.g. the ongoing comment refactoring, actor/user > > > refactoring, Multi Content Revision, JADE, major wikidata or structured > > > comons structure changes, new extensions not ever deployed to the > > cluster, > > > etc.) at the same time than the ongoing maintenance to reduce variables > > of > > > things that can go bad- enabling those features may be unblocked during > > the > > > switchover time, but we ask you to hold until being back on the current > > > active datacenter. Basically, ask yourself if you are enabling a large > > new > > > core feature or want to start a heavy-write maintenance script and there > > is > > > a chance you will need DBA/system support. Sadly, we had some instances > > of > > > this happening last year and we want to explicitly discourage this during > > > these 2 weeks. > > > > > > In own my opinion, enabling existing features on smaller projects (size > > > here is in amount of server resources, not that they are less important) > > is > > > equivalent to a swat change, and I am not against it happening. I would > > ask > > > contributors to use their best judgement on every case, and ask people on > > > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > > > My plea is to not enable major structural changes during that time may > > > affect thousands of edits per minute. Swat-like changes and "boring" :-) > > > trains are ok. > > > > > > For new wiki creations I would prefer if those were delayed but CC #DBA s > > > on the phabricator task to check with us. > > > ___ > > > Wikitech-l mailing list > > > Wikitech-l@lists.wikimedia.org > > > https://lists.wikimedia.org/m
Re: [Wikitech-l] Datacenter switchover and switchback
Oct 8 seems to be a particularly bad time to freeze the train given that we are forking for the MW 1.32 release on Oct 15, and a lot of folks have last-minute things they want to get into the release (eg deprecations, etc). --scott On Thu, Aug 30, 2018 at 10:57 AM Pine W wrote: > +1 to DJ's question about timing. Also, one might wish to be mindful of > the number of recent trains that were supposed to be boring but involved > interesting surprises; this makes me wonder whether trains that one thinks > will be boring are actually OK in this circumstance even if they turn out > to be "interesting". > > Pine > ( https://meta.wikimedia.org/wiki/User:Pine ) > > > > Original message From: Derk-Jan Hartman < > d.j.hartman+wmf...@gmail.com> Date: 8/30/18 2:54 AM (GMT-08:00) To: > Wikimedia developers Subject: Re: > [Wikitech-l] Datacenter switchover and switchback > While I think these regular switches are a very good idea, from an outside > perspective I do have to question a process that puts a significant plug in > the velocity of various teams working on major projects (esp. in a time of > year that could probably be seen as one of the most productive). What are > plans to reduce the disruption of this exercise in the future ? > > DJ > > On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo > wrote: > > > Let me explain the rationale of the bellow request for clarification: > > > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > > > Hello: > > > > > > >For the duration of the switchover (1 month), deployers are kindly > > > >requested to refrain from large db schema changes and avoid deploying > > > >any kind of new feature that requires creation of tables. > > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > > > > During the failover, some schema changes will be finalized on the current > > active datacenter (plus some major server and network maintenance may be > > done)- our request is mostly to refrain from quickly enabling those large > > new unlocked features (e.g. the ongoing comment refactoring, actor/user > > refactoring, Multi Content Revision, JADE, major wikidata or structured > > comons structure changes, new extensions not ever deployed to the > cluster, > > etc.) at the same time than the ongoing maintenance to reduce variables > of > > things that can go bad- enabling those features may be unblocked during > the > > switchover time, but we ask you to hold until being back on the current > > active datacenter. Basically, ask yourself if you are enabling a large > new > > core feature or want to start a heavy-write maintenance script and there > is > > a chance you will need DBA/system support. Sadly, we had some instances > of > > this happening last year and we want to explicitly discourage this during > > these 2 weeks. > > > > In own my opinion, enabling existing features on smaller projects (size > > here is in amount of server resources, not that they are less important) > is > > equivalent to a swat change, and I am not against it happening. I would > ask > > contributors to use their best judgement on every case, and ask people on > > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > > My plea is to not enable major structural changes during that time may > > affect thousands of edits per minute. Swat-like changes and "boring" :-) > > trains are ok. > > > > For new wiki creations I would prefer if those were delayed but CC #DBA s > > on the phabricator task to check with us. > > ___ > > Wikitech-l mailing list > > Wikitech-l@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- (http://cscott.net) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
Reminder: The switchback is next week. On Wed, Aug 29, 2018 at 8:28 PM Alexandros Kosiaris wrote: > > Hello everyone, > > This is to inform you that there will be a datacenter switchover and > switchback on the next few weeks. The timeline's are > > Services: Tuesday, September 11th 2018 14:30 UTC > Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC > Traffic: Tuesday, September 11th 2018 19:00 UTC > MediaWiki: Wednesday, September 12th 2018: 14:00 UTC > > Switchback: > > Traffic: Wednesday, October 10th 2018 09:00 UTC > MediaWiki: Wednesday, October 10th 2018: 14:00 UTC > Services: Thursday, October 11th 2018 14:30 UTC > Media storage/Swift: Thursday, October 11th 2018 15:00 UTC > > For the duration of the switchover (1 month), deployers are kindly > requested to refrain from large db schema changes and avoid deploying > any kind of new feature that requires creation of tables. > There will be a train freeze in the week of Sept 10th and Oct 8th. > > The net effect of the switchover and switchback for volunteers is > expected to be some minutes of inability to save an edit. For readers, > everything will be as usual. > > The tracking task for interested parties is > https://phabricator.wikimedia.org/T199073 > > Regards, > > -- > Alexandros Kosiaris -- Alexandros Kosiaris ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
On Thu, Aug 30, 2018 at 12:55 PM Derk-Jan Hartman wrote: > > While I think these regular switches are a very good idea, from an outside > perspective I do have to question a process that puts a significant plug in > the velocity of various teams working on major projects (esp. in a time of > year that could probably be seen as one of the most productive). That is absolutely true. > What are > plans to reduce the disruption of this exercise in the future ? There are indeed plans about making this easier, more automated and shorter in duration. Just to name a few, mediawiki now no longer requires deployments to switch datacenters but rather relies on a state key in the etcd database, there is a library + cookbooks to automate the currently automateable steps, there is work to update the documentation and make it update to date, accurate and more usable by more people and at shorter notices and so on. That being said, it's never gonna be free, but the toil is significant to make us want to reduce and that what we aim for. > > DJ > > On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo wrote: > > > Let me explain the rationale of the bellow request for clarification: > > > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > > > Hello: > > > > > > >For the duration of the switchover (1 month), deployers are kindly > > > >requested to refrain from large db schema changes and avoid deploying > > > >any kind of new feature that requires creation of tables. > > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > > > > During the failover, some schema changes will be finalized on the current > > active datacenter (plus some major server and network maintenance may be > > done)- our request is mostly to refrain from quickly enabling those large > > new unlocked features (e.g. the ongoing comment refactoring, actor/user > > refactoring, Multi Content Revision, JADE, major wikidata or structured > > comons structure changes, new extensions not ever deployed to the cluster, > > etc.) at the same time than the ongoing maintenance to reduce variables of > > things that can go bad- enabling those features may be unblocked during the > > switchover time, but we ask you to hold until being back on the current > > active datacenter. Basically, ask yourself if you are enabling a large new > > core feature or want to start a heavy-write maintenance script and there is > > a chance you will need DBA/system support. Sadly, we had some instances of > > this happening last year and we want to explicitly discourage this during > > these 2 weeks. > > > > In own my opinion, enabling existing features on smaller projects (size > > here is in amount of server resources, not that they are less important) is > > equivalent to a swat change, and I am not against it happening. I would ask > > contributors to use their best judgement on every case, and ask people on > > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > > My plea is to not enable major structural changes during that time may > > affect thousands of edits per minute. Swat-like changes and "boring" :-) > > trains are ok. > > > > For new wiki creations I would prefer if those were delayed but CC #DBA s > > on the phabricator task to check with us. > > ___ > > Wikitech-l mailing list > > Wikitech-l@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Alexandros Kosiaris ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
Wiki creation relies on experienced deployers and ops, I would expect they all know. On Thu, 30 Aug 2018, 16:49 Pine W, wrote: > A couple of additional points came to mind. > > 1. Blocking the creation of new wikis sounds like it could be a big deal. > I know little about the process for getting new wikis approved and > launched, but I hope that the folks who are regularly involved in these > processes have been advised of the situation. > > 2. In my previous email I may have revealed my level of ignorance about > the deployment train. Hopefully I didn't come across as presuming to "sound > smart" about that subject. I know that I am in the presence of experts on > that subject. > > Regards, > > Pine > ( https://meta.wikimedia.org/wiki/User:Pine ) > > > > Original message From: Pine W > Date: 8/30/18 7:57 AM (GMT-08:00) To: Wikimedia developers < > wikitech-l@lists.wikimedia.org> Subject: Re: [Wikitech-l] Datacenter > switchover and switchback > +1 to DJ's question about timing. Also, one might wish to be mindful of > the number of recent trains that were supposed to be boring but involved > interesting surprises; this makes me wonder whether trains that one thinks > will be boring are actually OK in this circumstance even if they turn out > to be "interesting". > > Pine > ( https://meta.wikimedia.org/wiki/User:Pine ) > > > > Original message From: Derk-Jan Hartman < > d.j.hartman+wmf...@gmail.com> Date: 8/30/18 2:54 AM (GMT-08:00) To: > Wikimedia developers Subject: Re: > [Wikitech-l] Datacenter switchover and switchback > While I think these regular switches are a very good idea, from an outside > perspective I do have to question a process that puts a significant plug in > the velocity of various teams working on major projects (esp. in a time of > year that could probably be seen as one of the most productive). What are > plans to reduce the disruption of this exercise in the future ? > > DJ > > On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo > wrote: > > > Let me explain the rationale of the bellow request for clarification: > > > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > > > Hello: > > > > > > >For the duration of the switchover (1 month), deployers are kindly > > > >requested to refrain from large db schema changes and avoid deploying > > > >any kind of new feature that requires creation of tables. > > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > > > > During the failover, some schema changes will be finalized on the current > > active datacenter (plus some major server and network maintenance may be > > done)- our request is mostly to refrain from quickly enabling those large > > new unlocked features (e.g. the ongoing comment refactoring, actor/user > > refactoring, Multi Content Revision, JADE, major wikidata or structured > > comons structure changes, new extensions not ever deployed to the > cluster, > > etc.) at the same time than the ongoing maintenance to reduce variables > of > > things that can go bad- enabling those features may be unblocked during > the > > switchover time, but we ask you to hold until being back on the current > > active datacenter. Basically, ask yourself if you are enabling a large > new > > core feature or want to start a heavy-write maintenance script and there > is > > a chance you will need DBA/system support. Sadly, we had some instances > of > > this happening last year and we want to explicitly discourage this during > > these 2 weeks. > > > > In own my opinion, enabling existing features on smaller projects (size > > here is in amount of server resources, not that they are less important) > is > > equivalent to a swat change, and I am not against it happening. I would > ask > > contributors to use their best judgement on every case, and ask people on > > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > > My plea is to not enable major structural changes during that time may > > affect thousands of edits per minute. Swat-like changes and "boring" :-) > > trains are ok. > > > > For new wiki creations I would prefer if those were delayed but CC #DBA s > > on the phabricator task to check with us. > > ___ > > Wikitech-l mailing list > > Wikitech-l@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
A couple of additional points came to mind. 1. Blocking the creation of new wikis sounds like it could be a big deal. I know little about the process for getting new wikis approved and launched, but I hope that the folks who are regularly involved in these processes have been advised of the situation. 2. In my previous email I may have revealed my level of ignorance about the deployment train. Hopefully I didn't come across as presuming to "sound smart" about that subject. I know that I am in the presence of experts on that subject. Regards, Pine ( https://meta.wikimedia.org/wiki/User:Pine ) Original message From: Pine W Date: 8/30/18 7:57 AM (GMT-08:00) To: Wikimedia developers Subject: Re: [Wikitech-l] Datacenter switchover and switchback +1 to DJ's question about timing. Also, one might wish to be mindful of the number of recent trains that were supposed to be boring but involved interesting surprises; this makes me wonder whether trains that one thinks will be boring are actually OK in this circumstance even if they turn out to be "interesting". Pine ( https://meta.wikimedia.org/wiki/User:Pine ) Original message From: Derk-Jan Hartman Date: 8/30/18 2:54 AM (GMT-08:00) To: Wikimedia developers Subject: Re: [Wikitech-l] Datacenter switchover and switchback While I think these regular switches are a very good idea, from an outside perspective I do have to question a process that puts a significant plug in the velocity of various teams working on major projects (esp. in a time of year that could probably be seen as one of the most productive). What are plans to reduce the disruption of this exercise in the future ? DJ On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo wrote: > Let me explain the rationale of the bellow request for clarification: > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > Hello: > > > > >For the duration of the switchover (1 month), deployers are kindly > > >requested to refrain from large db schema changes and avoid deploying > > >any kind of new feature that requires creation of tables. > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > During the failover, some schema changes will be finalized on the current > active datacenter (plus some major server and network maintenance may be > done)- our request is mostly to refrain from quickly enabling those large > new unlocked features (e.g. the ongoing comment refactoring, actor/user > refactoring, Multi Content Revision, JADE, major wikidata or structured > comons structure changes, new extensions not ever deployed to the cluster, > etc.) at the same time than the ongoing maintenance to reduce variables of > things that can go bad- enabling those features may be unblocked during the > switchover time, but we ask you to hold until being back on the current > active datacenter. Basically, ask yourself if you are enabling a large new > core feature or want to start a heavy-write maintenance script and there is > a chance you will need DBA/system support. Sadly, we had some instances of > this happening last year and we want to explicitly discourage this during > these 2 weeks. > > In own my opinion, enabling existing features on smaller projects (size > here is in amount of server resources, not that they are less important) is > equivalent to a swat change, and I am not against it happening. I would ask > contributors to use their best judgement on every case, and ask people on > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > My plea is to not enable major structural changes during that time may > affect thousands of edits per minute. Swat-like changes and "boring" :-) > trains are ok. > > For new wiki creations I would prefer if those were delayed but CC #DBA s > on the phabricator task to check with us. > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
+1 to DJ's question about timing. Also, one might wish to be mindful of the number of recent trains that were supposed to be boring but involved interesting surprises; this makes me wonder whether trains that one thinks will be boring are actually OK in this circumstance even if they turn out to be "interesting". Pine ( https://meta.wikimedia.org/wiki/User:Pine ) Original message From: Derk-Jan Hartman Date: 8/30/18 2:54 AM (GMT-08:00) To: Wikimedia developers Subject: Re: [Wikitech-l] Datacenter switchover and switchback While I think these regular switches are a very good idea, from an outside perspective I do have to question a process that puts a significant plug in the velocity of various teams working on major projects (esp. in a time of year that could probably be seen as one of the most productive). What are plans to reduce the disruption of this exercise in the future ? DJ On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo wrote: > Let me explain the rationale of the bellow request for clarification: > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > Hello: > > > > >For the duration of the switchover (1 month), deployers are kindly > > >requested to refrain from large db schema changes and avoid deploying > > >any kind of new feature that requires creation of tables. > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > During the failover, some schema changes will be finalized on the current > active datacenter (plus some major server and network maintenance may be > done)- our request is mostly to refrain from quickly enabling those large > new unlocked features (e.g. the ongoing comment refactoring, actor/user > refactoring, Multi Content Revision, JADE, major wikidata or structured > comons structure changes, new extensions not ever deployed to the cluster, > etc.) at the same time than the ongoing maintenance to reduce variables of > things that can go bad- enabling those features may be unblocked during the > switchover time, but we ask you to hold until being back on the current > active datacenter. Basically, ask yourself if you are enabling a large new > core feature or want to start a heavy-write maintenance script and there is > a chance you will need DBA/system support. Sadly, we had some instances of > this happening last year and we want to explicitly discourage this during > these 2 weeks. > > In own my opinion, enabling existing features on smaller projects (size > here is in amount of server resources, not that they are less important) is > equivalent to a swat change, and I am not against it happening. I would ask > contributors to use their best judgement on every case, and ask people on > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > My plea is to not enable major structural changes during that time may > affect thousands of edits per minute. Swat-like changes and "boring" :-) > trains are ok. > > For new wiki creations I would prefer if those were delayed but CC #DBA s > on the phabricator task to check with us. > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
While I think these regular switches are a very good idea, from an outside perspective I do have to question a process that puts a significant plug in the velocity of various teams working on major projects (esp. in a time of year that could probably be seen as one of the most productive). What are plans to reduce the disruption of this exercise in the future ? DJ On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo wrote: > Let me explain the rationale of the bellow request for clarification: > > On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > > > Hello: > > > > >For the duration of the switchover (1 month), deployers are kindly > > >requested to refrain from large db schema changes and avoid deploying > > >any kind of new feature that requires creation of tables. > > >There will be a train freeze in the week of Sept 10th and Oct 8th. > > > During the failover, some schema changes will be finalized on the current > active datacenter (plus some major server and network maintenance may be > done)- our request is mostly to refrain from quickly enabling those large > new unlocked features (e.g. the ongoing comment refactoring, actor/user > refactoring, Multi Content Revision, JADE, major wikidata or structured > comons structure changes, new extensions not ever deployed to the cluster, > etc.) at the same time than the ongoing maintenance to reduce variables of > things that can go bad- enabling those features may be unblocked during the > switchover time, but we ask you to hold until being back on the current > active datacenter. Basically, ask yourself if you are enabling a large new > core feature or want to start a heavy-write maintenance script and there is > a chance you will need DBA/system support. Sadly, we had some instances of > this happening last year and we want to explicitly discourage this during > these 2 weeks. > > In own my opinion, enabling existing features on smaller projects (size > here is in amount of server resources, not that they are less important) is > equivalent to a swat change, and I am not against it happening. I would ask > contributors to use their best judgement on every case, and ask people on > the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. > My plea is to not enable major structural changes during that time may > affect thousands of edits per minute. Swat-like changes and "boring" :-) > trains are ok. > > For new wiki creations I would prefer if those were delayed but CC #DBA s > on the phabricator task to check with us. > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
Let me explain the rationale of the bellow request for clarification: On Wed, Aug 29, 2018 at 11:30 PM MA wrote: > Hello: > > >For the duration of the switchover (1 month), deployers are kindly > >requested to refrain from large db schema changes and avoid deploying > >any kind of new feature that requires creation of tables. > >There will be a train freeze in the week of Sept 10th and Oct 8th. During the failover, some schema changes will be finalized on the current active datacenter (plus some major server and network maintenance may be done)- our request is mostly to refrain from quickly enabling those large new unlocked features (e.g. the ongoing comment refactoring, actor/user refactoring, Multi Content Revision, JADE, major wikidata or structured comons structure changes, new extensions not ever deployed to the cluster, etc.) at the same time than the ongoing maintenance to reduce variables of things that can go bad- enabling those features may be unblocked during the switchover time, but we ask you to hold until being back on the current active datacenter. Basically, ask yourself if you are enabling a large new core feature or want to start a heavy-write maintenance script and there is a chance you will need DBA/system support. Sadly, we had some instances of this happening last year and we want to explicitly discourage this during these 2 weeks. In own my opinion, enabling existing features on smaller projects (size here is in amount of server resources, not that they are less important) is equivalent to a swat change, and I am not against it happening. I would ask contributors to use their best judgement on every case, and ask people on the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. My plea is to not enable major structural changes during that time may affect thousands of edits per minute. Swat-like changes and "boring" :-) trains are ok. For new wiki creations I would prefer if those were delayed but CC #DBA s on the phabricator task to check with us. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Datacenter switchover and switchback
Hello: >For the duration of the switchover (1 month), deployers are kindly >requested to refrain from large db schema changes and avoid deploying >any kind of new feature that requires creation of tables. >There will be a train freeze in the week of Sept 10th and Oct 8th. Does this also mean that we should not deploy new extensions to wikis such as Babel, Echo, ShortUrl or Translate (to mention some examples) where we also need to create the relevant extension tables on the wiki in question for them to work? I guess also no new wiki creations will be allowed in that timespan? Thanks for the clarification. Regards. -- M. A. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Datacenter switchover and switchback
Hello everyone, This is to inform you that there will be a datacenter switchover and switchback on the next few weeks. The timeline's are Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC Switchback: Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC For the duration of the switchover (1 month), deployers are kindly requested to refrain from large db schema changes and avoid deploying any kind of new feature that requires creation of tables. There will be a train freeze in the week of Sept 10th and Oct 8th. The net effect of the switchover and switchback for volunteers is expected to be some minutes of inability to save an edit. For readers, everything will be as usual. The tracking task for interested parties is https://phabricator.wikimedia.org/T199073 Regards, -- Alexandros Kosiaris ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
