On Tue, Jun 22, 2021 at 11:56 AM Danny Sauer <[email protected]> wrote:
> You can certainly run multiple instances of the content server. It just > needs a connection to the database and access to the storage. > Agreed, you could deploy additional content servers and have your nginx/apache load balance them. > Have you tuned the number of worker processes in Gunicorn? It defaults to > 1, but should almost certainly be increased for any sort of volume. > https://docs.gunicorn.org/en/stable/settings.html#worker-processes > Pulp changed the default gunicorn worker processes to 8 maybe a release or two ago. See the `pulp_content_workers` variable in the installer here https://pulp-installer.readthedocs.io/en/latest/roles/pulp_content/#role-variables > > There are several moving pieces, but that's really all I had to touch here. > > --Danny > With pulpcore==3.14 there is a significant performance improvement being reviewed now https://pulp.plan.io/issues/8805 . In addition to resolving it with methods like ^, when 3.14 comes out (scheduled for June 29th) it would be great if you could report on if the improvements helped you. > > On Tue, Jun 22, 2021 at 10:34 AM Bin Li (BLOOMBERG/ 120 PARK) < > [email protected]> wrote: > >> We recently add more clients to use the pulp content server. The >> processes run out the file descriptor first. We then increased both nginx >> and pulp-content by creating a override.conf >> /etc/systemd/system/pulpcore-content.service.d # cat override.conf >> [Service] >> LimitNOFILE=65536 >> >> and updated nginx.conf >> # Gunicorn docs suggest this value. >> worker_processes 1; >> events { >> worker_connections 10000; # increase if you have lots of clients >> accept_mutex off; # set to 'on' if nginx worker_processes > 1 >> } >> >> worker_rlimit_nofile 20000; >> >> >> Now we are keep getting this error. >> 2021/06/22 11:26:36 [error] 78373#0: *112823 upstream timed out (110: >> Connection timed out) while connecting to upstream, client: >> >> It looks like pulp-content server cannot keep up with requests. Is there >> anything we could do to increase the performance of the content server? >> _______________________________________________ >> Pulp-list mailing list >> [email protected] >> https://listman.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
