So much good news in a single email! Should have saved up some of it to dole out over time :)
Thanks Luca and Analytics! On Thu, Mar 12, 2020, 13:42 Nuria Ruiz <[email protected]> wrote: > Hello, > > >We deployed jupyterhub on stat1004 and stat1006, > So we are all clear on what this implies it means that disk space > constrains in jupyter notebooks are no longer an issue. The stats machines > have much more disk available than > the notebook hosts. That being said that answer to larger workloads on > jupyter is to run those in hadoop rather than locally. We have done some > work to facilitate running distributed jobs from jupyter in hadoop and that > work will continue next quarter. > > > > > On Thu, Mar 12, 2020 at 11:29 AM Luca Toscano <[email protected]> > wrote: > >> Hi everybody, >> >> some news from the Analytics team: >> >> - The Kerberos ticket expiry time has been bumped to 48h. You can >> kdestroy/kinit to get the new settings :) >> >> - There are new memory and cpu limits on all stat/notebook hosts, that >> should automatically kill big jobs that cause too much memory pressure. CPU >> cores are also limited to 90% of the available ones to leave space for >> system daemons. This should help a lot in avoiding recurrent alarms to the >> SRE team (and me reaching out to some of you as consequence!) and it should >> be a more fair system for everybody. In order to apply these new settings >> I'd need to shutdown/start all the notebooks running on notebook1003/1004, >> but I didn't do it since I didn't want to impact any work. If you could >> please take care of stopping/starting your notebooks it would be really >> appreciated :) >> >> - We deployed jupyterhub on stat1004 and stat1006, ready for general use! >> This should help in avoiding the small home size problem that many of you >> are experiencing on notebook1003/1004. We are also working on setting up >> jupyterhub on stat1005, with updated dependencies (jupyterhub 1.1.0, toree >> 0.3.0, etc.. full list in >> https://gerrit.wikimedia.org/r/#/c/analytics/jupyterhub/deploy/+/577761/1/frozen-requirements.txt). >> The plan is to eventually have the same version on all stat boxes (no >> timeline yet). We didn't deploy jupyterhub on stat1007 due to some puppet >> code refactoring in progress, but we hope to do it next quarter. >> >> - A new stat host (stat1008) will be ready for general use soon. It hosts >> a GPU like stat1005. >> >> If you have questions/doubts/etc.. please feel free to follow up with me >> or any member of the Analytics team on #wikimedia-analytics :) >> >> Luca (on behalf of the Analytics team) >> _______________________________________________ >> Research-Internal mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/research-internal >> > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
