[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Addshore]

Addshore closed this task as "Resolved".

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE, Addshore
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[gerritbot]

gerritbot added a comment.


  Change 603570 **abandoned** by Addshore:
  [mediawiki/core@master] Document that addSite won't work well without 
apc.enable_cli enabled
  
  Reason:
  See https://gerrit.wikimedia.org/r/c/661412
  
  https://gerrit.wikimedia.org/r/603570

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE, gerritbot
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[ReleaseTaggerBot]

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.31; 2021-02-16).

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE, ReleaseTaggerBot
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[gerritbot]

gerritbot added a comment.


  Change 661412 **merged** by jenkins-bot:
  [mediawiki/core@master] Add notice about local-server cache to addSite.php
  
  https://gerrit.wikimedia.org/r/661412

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE, gerritbot
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Addshore]

Addshore added a comment.


  > So I think the task description is a bit misguided
  
  Yep, I think this is one of the issues with trying to over specify the thing 
up front before it is looked at.
  This ticket should have just been left describing the perceived problem and 
desired result without all of the other parts.

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE, Addshore
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE updated the task description.

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  I updated the mw.org documentation 

 and uploaded the above change to hopefully clear this up a bit. I don’t think 
there’s much else we can do here. (Making the script print a warning doesn’t 
feel right to me, if that warning would have to be printed every time the 
script is used.)

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[gerritbot]

gerritbot added a comment.


  Change 661412 had a related patch set uploaded (by Lucas Werkmeister (WMDE); 
owner: Lucas Werkmeister (WMDE)):
  [mediawiki/core@master] Add notice about local-server cache to addSite.php
  
  https://gerrit.wikimedia.org/r/661412

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE, gerritbot
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  Hm, but this seems to be the //intended// behavior of the sites store?
  
  name=ServiceWiring.php
'SiteStore' => function ( MediaWikiServices $services ) : SiteStore {
$rawSiteStore = new DBSiteStore( $services->getDBLoadBalancer() 
);

$cache = $services->getLocalServerObjectCache();
if ( $cache instanceof EmptyBagOStuff ) {
$cache = ObjectCache::getLocalClusterInstance();
}

return new CachingSiteStore( $rawSiteStore, $cache );
},
  
  This specifically uses the local-server cache 
(`ObjectCache::makeLocalServerCache()`: APCu or WinCache), and only falls back 
to the cluster-local cache (i.e. `$wgMainCacheType`) if the other is not 
available. And indeed, I can reproduce this issue even with `$wgMainCacheType = 
CACHE_MEMCACHED` in my local settings.
  
  (Aside: if I understand the code correctly, by default `addSite.php` will 
actually send the new sites to the `$wgMainCacheType` cache, because 
`ObjectCache::makeLocalServerCache()` will notice that `apc.enable_cli` is off, 
and the site store will therefore fall back to the main cache. But web requests 
will then ignore the main cache and get sites from the server-local cache 
instead.)
  
  So I guess the “reload the webserver” advice should not only be given if 
`$wgMainCacheType = CACHE_ACCEL`, but actually… always?

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  In T269099#6780349 , 
@Lucas_Werkmeister_WMDE wrote:
  
  >> The script does run successfully and adds the site to the database, but 
the site is not visible as the cache is populated with the old data.
  >
  > One way to check this could be to compare the results of 
`api.php?action=sitematrix` (using Extension:SiteMatrix 
) with 
`maintenance/exportSites.php`
  
  SiteMatrix is apparently painful to set up, but I was able to instead verify 
the issue with the following JS code on a wiki page:
  
void mw.loader.using('mw.config.values.wbSiteDetails').then(() => { for 
(const [name, site] of Object.entries(mw.config.get('wbSiteDetails'))) if 
(name.startsWith('testwiki_T269099_')) console.log(site); })
  
  And then after running `php maintenance/addSite.php testwiki_T269099_1 wiki` 
(_2, _3, etc.), `php maintenance/exportSites.php php://stdout | grep 
testwiki_T269099_` would show the new sites, but the JS-site module would not. 
(I also tweaked `SitesModule::getScript()` to directly return 
`$this->makeScript( $languageCode )`, bypassing the cache for the 
ResourceLoader module script.)
  
  However, this problem didn’t go away after setting `apc.enable_cli` to `1` in 
the `php.ini` file. And if I understand APC(u) correctly, this makes sense: 
according to this StackOverflow answer 
, PHP worker processes of a web 
server can sometimes share the same cache depending on how they’re set up, but 
their cache can never be shared with CLI scripts. APC(u) is an in-process 
cache, and maintenance scripts running as separate processes will get their own 
cache.
  
  So I think the task description is a bit misguided – the problem is not 
really `apc.enable_cli` specifically, but APC(u) in general. If MediaWiki is 
configured to use `CACHE_ACCEL`, then maintenance scripts can’t update or purge 
that cache. So I think we should check for that condition in the maintenance 
script, and if an APC(u) cache is configured, then the best advice we can give 
is probably to reload the webserver. (I tested that `systemctl reload apache2` 
made the sites show up with the above JS snippet, but the advice in the 
maintenance script should probably not be systemd- or Apache-specific.)

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE updated the task description.

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE claimed this task.
Lucas_Werkmeister_WMDE moved this task from To Do (prioritised from top to 
bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

WORKBOARD
  https://phabricator.wikimedia.org/project/board/3539/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  > The script does run successfully and adds the site to the database, but the 
site is not visible as the cache is populated with the old data.
  
  One way to check this could be to compare the results of 
`api.php?action=sitematrix` (using Extension:SiteMatrix 
) with 
`maintenance/exportSites.php`

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Akuckartz, Hook696, Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, 
CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, 
Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, 
Mbch331, Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] T269099: [8hrs] Investigate the effects of apc.enable_cli on the addSite.php maintenance script

[noarave]

noarave renamed this task from "Investigate the effects of  apc.enable_cli  on 
the addSite.php maintenance script" to "[8hrs] Investigate the effects of  
apc.enable_cli  on the addSite.php maintenance script".

TASK DETAIL
  https://phabricator.wikimedia.org/T269099

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: noarave
Cc: Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, 
Iflorez, Kent7301, alaa_wmde, joker88john, DannyS712, CucyNoiD, Nandana, 
Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, 
GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, 
rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, 
Rxy, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs