Federico, Francium is meant as a processing and storage server. The HTML dump part is relatively light on CPU and naturally heavy on storage. ZIM based on this will be more CPU-consuming, but should still be doable with a single hardware node.
The current status is that the hardware is up, but we don't have shell access to the server yet. I hope that this can be resolved soon. Gabriel On Tue, Mar 31, 2015 at 12:05 AM, Federico Leva (Nemo) <[email protected]> wrote: > Any news on this? > > > == Next steps == > > > > We want to complete our effort and mirror the biggest Wikipedia > > projects. Unfortunately, we have reached the limits of a traditional > > usage of wmflabs. We need more quota and to experiment with the NFS > > storage because an x-large instance in not able to mirror more than > > 1.5 millions of articles at a time. How might that be made possible? > > Per https://phabricator.wikimedia.org/T91853#1113701 and > https://phabricator.wikimedia.org/T91853#1129560 , francium seems a > storage server rather than a processing server: it can't be used for > mwoffliner tasks. https://phabricator.wikimedia.org/T57503 is a separate > task. > Emmanuel, I think the Labs instances are only meant to store the > data they're currently processing, while the ZIM files as such keep ending > up on http://download.kiwix.org/ + mirrors: right? > Andrew, if so the best option IMHO is to increase the local disk > quota of (at least) one of the instances from 160 to 500 GB. Then Emmanuel > can start testing an en.wiki run and see how it goes. virt1011, currently > used by mwoffliner2, seems to afford it. https://ganglia.wikimedia.org/ > latest/graph_all_periods.php?c=Virtualization%20cluster% > 20eqiad&h=virt1011.eqiad.wmnet&r=hour&z=default&jr=&js= > &st=1427785147&v=1188.316&m=disk_free&vl=GB&ti=Disk% > 20Space%20Available&z=large > > Nemo > > > _______________________________________________ > Labs-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/labs-l >
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
