Any news on this? > == Next steps == > > We want to complete our effort and mirror the biggest Wikipedia > projects. Unfortunately, we have reached the limits of a traditional > usage of wmflabs. We need more quota and to experiment with the NFS > storage because an x-large instance in not able to mirror more than > 1.5 millions of articles at a time. How might that be made possible?
Per https://phabricator.wikimedia.org/T91853#1113701 and https://phabricator.wikimedia.org/T91853#1129560 , francium seems a storage server rather than a processing server: it can't be used for mwoffliner tasks. https://phabricator.wikimedia.org/T57503 is a separate task. Emmanuel, I think the Labs instances are only meant to store the data they're currently processing, while the ZIM files as such keep ending up on http://download.kiwix.org/ + mirrors: right? Andrew, if so the best option IMHO is to increase the local disk quota of (at least) one of the instances from 160 to 500 GB. Then Emmanuel can start testing an en.wiki run and see how it goes. virt1011, currently used by mwoffliner2, seems to afford it. https://ganglia.wikimedia.org/latest/graph_all_periods.php?c=Virtualization%20cluster%20eqiad&h=virt1011.eqiad.wmnet&r=hour&z=default&jr=&js=&st=1427785147&v=1188.316&m=disk_free&vl=GB&ti=Disk%20Space%20Available&z=large
Nemo _______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
