On 06-Jan-17 2:07 PM, Gary Lee wrote:
> Hi Praveen
>
> I’m a little concerned about extending the queue size from 200 to 500, in
> terms of load on IMM. Should I push the patch with just 200?
> I think trimming only half the queue should solve the original issue.
I am ok if the test case is passing.
Regarding the load on IMM. I just brought two controllers up. On second
controller I am seeing in amfd traces:
Jan 6 14:24:51.839865 osafamfd [3696:src/amf/amfd/imm.cc:0394] TR
Emptying imm job queue of size:291
Jan 6 14:24:51.839870 osafamfd [3696:src/amf/amfd/imm.cc:0457] >> empty
Jan 6 14:24:51.839925 osafamfd [3696:src/amf/amfd/imm.cc:0463] << empty
So in a two node system queue size reaches to 291. In a deployed system
with big cluster size and with good number of applications running will
further make it very high value. Although this value is during initial
cluster formation, but such a high value can reach in a running cluster
also (some payloads reboots simultaneously). I think IMM must be able to
handle it.
Thanks,
Praveen
>
> Then we can improve with your patch etc..
>
> Thanks
> Gary
>
> -----Original Message-----
> From: praveen malviya <praveen.malv...@oracle.com>
> Organization: Oracle Corporation
> Date: Thursday, 5 January 2017 at 4:21 pm
> To: Gary Lee <gary....@dektech.com.au>, <hans.nordeb...@ericsson.com>,
> <minh.c...@dektech.com.au>, <nagendr...@oracle.com>
> Cc: <opensaf-devel@lists.sourceforge.net>
> Subject: Re: [PATCH 1 of 1] amfd: increase max job queue size at standby
> [#2228]
>
> Hi Gary,
> Ack from me (code review only). I think MAX_JOB_SIZE_AT_STANDBY could be
> made configurable using amfd.conf so that it can be fine tuned based on
> the cluster size and no of applications.
>
> While working on #1141 (defect), I had prepared an enhancement patch
> also that will remove limitation on size. Idea was, when active amfd
> finishes all the jobs and job queue becomes empty then it will send a
> checkpoint msg to standby to flush its job queue. I will raise a ticket
> and publish it for 5.2.
>
> Thanks,
> Praveen
>
>
> On 04-Jan-17 9:15 AM, Gary Lee wrote:
> > src/amf/amfd/imm.cc | 28 +++++++++++++++++++++++-----
> > src/amf/amfd/imm.h | 2 ++
> > 2 files changed, 25 insertions(+), 5 deletions(-)
> >
> >
> > After a cold sync, the standby job queue size can exceed the current
> limit of 200.
> > This results in the emptying of the entire queue at standby.
> >
> > Potentially, some IMM updates are not done, as all jobs will be deleted
> during
> > a failover if the size is exceeded. This is especially important for
> updates that
> > the old 'active' could not perform in time during the failover.
> >
> > This patch attempts to reduce the occurrence of this by increasing the
> max size,
> > and only trimming the queue to half the maximum size. The updates that
> must be done
> > are likely to be located towards the end of the queue, and these are
> kept.
> >
> > diff --git a/src/amf/amfd/imm.cc b/src/amf/amfd/imm.cc
> > --- a/src/amf/amfd/imm.cc
> > +++ b/src/amf/amfd/imm.cc
> > @@ -126,7 +126,7 @@ static char *StrDup(const char *s)
> > std::strcpy(c,s);
> > return c;
> > }
> > -uint32_t const MAX_JOB_SIZE_AT_STANDBY = 200;
> > +uint32_t const MAX_JOB_SIZE_AT_STANDBY = 500;
> >
> > //
> > Job::~Job()
> > @@ -390,10 +390,15 @@ Job* Fifo::dequeue()
> > */
> > void check_and_flush_job_queue_standby_amfd(void)
> > {
> > - if (Fifo::size() >= MAX_JOB_SIZE_AT_STANDBY) {
> > - TRACE("Emptying imm job queue of size:%u",Fifo::size());
> > - Fifo::empty();
> > - }
> > + TRACE_ENTER();
> > +
> > + if (Fifo::size() >= MAX_JOB_SIZE_AT_STANDBY) {
> > + const uint32_t new_size = MAX_JOB_SIZE_AT_STANDBY / 2;
> > + LOG_WA("Reducing job queue of size:%u to
> %u",Fifo::size(),new_size);
> > + Fifo::trim_to_size(new_size);
> > + }
> > +
> > + TRACE_LEAVE();
> > }
> >
> > //
> > @@ -468,6 +473,19 @@ uint32_t Fifo::size()
> > return job_.size();
> > }
> >
> > +void Fifo::trim_to_size(const uint32_t size)
> > +{
> > + Job *ajob;
> > +
> > + TRACE_ENTER();
> > +
> > + while (job_.size() > size && (ajob = dequeue()) != nullptr) {
> > + delete ajob;
> > + }
> > +
> > + TRACE_LEAVE();
> > +}
> > +
> > //
> > std::queue<Job*> Fifo::job_;
> > //
> > diff --git a/src/amf/amfd/imm.h b/src/amf/amfd/imm.h
> > --- a/src/amf/amfd/imm.h
> > +++ b/src/amf/amfd/imm.h
> > @@ -151,6 +151,8 @@ public:
> > static void empty();
> >
> > static uint32_t size();
> > +
> > + static void trim_to_size(const uint32_t size);
> > private:
> > static std::queue<Job*> job_;
> > };
> >
>
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
