How many instances of Redis are involved? Is every pulpcore-api instance and pulpcore-worker instance pointing to the same redis instance? This is necessary for the work to be routed correctly.
Pulpcore currently uses redis-py, which does not support connecting to a Redis Cluster[0]. However, we should investigate if it's viable to switch to using redis-py-cluster[1]. [0] https://github.com/andymccurdy/redis-py/issues/931 [1] https://github.com/Grokzen/redis-py-cluster On Thu, Feb 13, 2020, 6:28 PM Bin Li (BLOOMBERG/ 120 PARK) < [email protected]> wrote: > Hi Brian, > I did a quick test on a active passive pulp 3.1 setup. Two pulp servers > are pointing to the same external postgres database. Only one server is > active at any time. Redis queue resides on the localhost. The /var/lib/pulp > are synced from primary to the contingency host. > After I shutdown primary host, I was able to bring up the contingency pulp > server and created a repo. Deleting any repo stuck in a waiting state. Then > I started primary host and shutdown contingency host, I was able to delete > repos I created on the contingency host but all previous delete job > continually stuck in the waiting state. > I am wonder if anything I could do to make this work on contingency host > or this setup is not going to work? > > Thanks > > > From: [email protected] At: 01/03/20 12:01:44 > To: [email protected] > Subject: Pulp-list Digest, Vol 122, Issue 1 > > Send Pulp-list mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.redhat.com/mailman/listinfo/pulp-list > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Pulp-list digest..." > > > Today's Topics: > > 1. Re: pulp3 High availability and disaster recovery (Brian Bouterse) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 2 Jan 2020 16:10:29 -0500 > From: Brian Bouterse <[email protected]> > To: JASON STELZER <[email protected]> > Cc: pulp-list <[email protected]> > Subject: Re: [Pulp-list] pulp3 High availability and disaster recovery > Message-ID: > <caacvrtgdycjxco3tr50wub1j2suc6g9q1_yqjvdsys_t44q...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Sorry for the late reply. Each component of Pulp itself can be deployed in > HA configurations. Of the services Pulp's processes depend on, Redis is the > one service that can't run as a full cluster because RQ doesn't support > that yet, so the best you can do is a hot-spare Redis that auto-fails over. > That isn't graceful failover so when traffic routes to your hot-spare Redis > it has to data and doesn't have the tasking system's data. Those Pulp tasks > would be cancelled, and Pulp would be immediately ready to accept new tasks > so they could be resubmitted, e.g. Katello resubmits some job failures I > believe. > > More docs about this are here: > https://docs.pulpproject.org/components.html#architecture-and-deploying > More questions are welcome; sorry for the slow response. If you can see any > way to improve the docs and want to get involved, PRs are welcome! > > -Brian > > > On Mon, Nov 18, 2019 at 7:37 AM JASON STELZER <[email protected]> > wrote: > > > For what it is worth, at heart pulp3 is a django app. So, following the > > advice for HA and django apps generally works. A lot of it is driven by > the > > particulars of your use case. > > > > My use case is a little different than yours I'm sure. But in terms of HA > > for now I'm good with a balancer and nodes in multiple azs, an RDS db > with > > failover, and regular db backups. > > > > In my case, the pulp3 server is far enough behind the scenes that even if > > there were to be a several hour outage, the impact would be minimal. > YMMV. > > > > Others can chime in with pulp3 specifics. > > > > On Fri, Nov 15, 2019 at 11:41 AM Bin Li (BLOOMBERG/ 120 PARK) < > > [email protected]> wrote: > > > >> Does pulp3 support active/active or active/passive configuration? What > is > >> the strategy to restore the pulp3 service on a different server if the > >> primary is down? Do we have any documentation on this topic? > >> > >> Thanks > >> _______________________________________________ > >> Pulp-list mailing list > >> [email protected] > >> https://www.redhat.com/mailman/listinfo/pulp-list > > > > > > > > -- > > J. > > _______________________________________________ > > Pulp-list mailing list > > [email protected] > > https://www.redhat.com/mailman/listinfo/pulp-list > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > < > https://www.redhat.com/archives/pulp-list/attachments/20200102/4cc40982/atta > chment.html> > > ------------------------------ > > _______________________________________________ > Pulp-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/pulp-list > > End of Pulp-list Digest, Vol 122, Issue 1 > ***************************************** > > > _______________________________________________ > Pulp-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
