On Wed, Oct 7, 2020 at 4:16 PM Roy Smith <[email protected]> wrote: > > > On Oct 7, 2020, at 4:04 PM, Bryan Davis <[email protected]> wrote: > > > > Use a `webservice python3.7 shell` session as your dev/test > > environment and you will a) get the same python version as the > > "production" container, and b) move your dev/test workload off of the > > limited resources of the bastion server and onto the more scalable > > Kubernetes cluster. > > BTDT. It was unusable. People can search the list archives if they're > interested in details. At this point, I can't even get in: > > > WARNING: No explict backend provided. > > Using default of 'kubernetes' > > For help refer to <https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web> > > runtime: failed to create new OS thread (have 3 already; errno=11) > > runtime: may need to increase max user processes (ulimit -u) > > fatal error: newosproc
This error indicates that you are running out of process quota on the bastion server due to the number of processes that you are attempting to run in parallel. We have fairly strict limits placed on each user to help prevent overwhelming the bastions. The quota is typically enough to run 2-3 small applications. I can see that you are currently logged into the dev.toolforge.org bastion (tools-sgebastion-08.tools.eqiad.wmflabs) where you have a tmux session running with four bash shells and a python process that appears to be some sort of TCP log event sync. This last process is almost certainly pushing your thread quota to the limit. > I understand that k8s is scalable, and the way of the future, and that > upgrades need to be prioritized, etc. But the current setup is not an > effective development environment. This may actually be the root of some of your frustrations. Toolforge is not meant to be a replacement for a development environment on your local laptop or other server. It is intended primarily to be a runtime environment for tools. Attempting to use the bastions as your primary development environment is going to be a struggle against the various quotas and limits that have been placed on the bastions in an attempt to keep them available and responsive for people who need to start/stop/restart their tools and check log output. You might be interested in things like <https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_to_the_database_replicas_from_your_own_computer> to help you find a way to develop and test your software outside of Toolforge. Bryan -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Wikimedia Cloud Services mailing list [email protected] (formerly [email protected]) https://lists.wikimedia.org/mailman/listinfo/cloud
