On 06-Jan-17 2:07 PM, Gary Lee wrote: > Hi Praveen > > I’m a little concerned about extending the queue size from 200 to 500, in > terms of load on IMM. Should I push the patch with just 200? > I think trimming only half the queue should solve the original issue. I am ok if the test case is passing.
Regarding the load on IMM. I just brought two controllers up. On second controller I am seeing in amfd traces: Jan 6 14:24:51.839865 osafamfd [3696:src/amf/amfd/imm.cc:0394] TR Emptying imm job queue of size:291 Jan 6 14:24:51.839870 osafamfd [3696:src/amf/amfd/imm.cc:0457] >> empty Jan 6 14:24:51.839925 osafamfd [3696:src/amf/amfd/imm.cc:0463] << empty So in a two node system queue size reaches to 291. In a deployed system with big cluster size and with good number of applications running will further make it very high value. Although this value is during initial cluster formation, but such a high value can reach in a running cluster also (some payloads reboots simultaneously). I think IMM must be able to handle it. Thanks, Praveen > > Then we can improve with your patch etc.. > > Thanks > Gary > > -----Original Message----- > From: praveen malviya <praveen.malv...@oracle.com> > Organization: Oracle Corporation > Date: Thursday, 5 January 2017 at 4:21 pm > To: Gary Lee <gary....@dektech.com.au>, <hans.nordeb...@ericsson.com>, > <minh.c...@dektech.com.au>, <nagendr...@oracle.com> > Cc: <opensaf-devel@lists.sourceforge.net> > Subject: Re: [PATCH 1 of 1] amfd: increase max job queue size at standby > [#2228] > > Hi Gary, > Ack from me (code review only). I think MAX_JOB_SIZE_AT_STANDBY could be > made configurable using amfd.conf so that it can be fine tuned based on > the cluster size and no of applications. > > While working on #1141 (defect), I had prepared an enhancement patch > also that will remove limitation on size. Idea was, when active amfd > finishes all the jobs and job queue becomes empty then it will send a > checkpoint msg to standby to flush its job queue. I will raise a ticket > and publish it for 5.2. > > Thanks, > Praveen > > > On 04-Jan-17 9:15 AM, Gary Lee wrote: > > src/amf/amfd/imm.cc | 28 +++++++++++++++++++++++----- > > src/amf/amfd/imm.h | 2 ++ > > 2 files changed, 25 insertions(+), 5 deletions(-) > > > > > > After a cold sync, the standby job queue size can exceed the current > limit of 200. > > This results in the emptying of the entire queue at standby. > > > > Potentially, some IMM updates are not done, as all jobs will be deleted > during > > a failover if the size is exceeded. This is especially important for > updates that > > the old 'active' could not perform in time during the failover. > > > > This patch attempts to reduce the occurrence of this by increasing the > max size, > > and only trimming the queue to half the maximum size. The updates that > must be done > > are likely to be located towards the end of the queue, and these are > kept. > > > > diff --git a/src/amf/amfd/imm.cc b/src/amf/amfd/imm.cc > > --- a/src/amf/amfd/imm.cc > > +++ b/src/amf/amfd/imm.cc > > @@ -126,7 +126,7 @@ static char *StrDup(const char *s) > > std::strcpy(c,s); > > return c; > > } > > -uint32_t const MAX_JOB_SIZE_AT_STANDBY = 200; > > +uint32_t const MAX_JOB_SIZE_AT_STANDBY = 500; > > > > // > > Job::~Job() > > @@ -390,10 +390,15 @@ Job* Fifo::dequeue() > > */ > > void check_and_flush_job_queue_standby_amfd(void) > > { > > - if (Fifo::size() >= MAX_JOB_SIZE_AT_STANDBY) { > > - TRACE("Emptying imm job queue of size:%u",Fifo::size()); > > - Fifo::empty(); > > - } > > + TRACE_ENTER(); > > + > > + if (Fifo::size() >= MAX_JOB_SIZE_AT_STANDBY) { > > + const uint32_t new_size = MAX_JOB_SIZE_AT_STANDBY / 2; > > + LOG_WA("Reducing job queue of size:%u to > %u",Fifo::size(),new_size); > > + Fifo::trim_to_size(new_size); > > + } > > + > > + TRACE_LEAVE(); > > } > > > > // > > @@ -468,6 +473,19 @@ uint32_t Fifo::size() > > return job_.size(); > > } > > > > +void Fifo::trim_to_size(const uint32_t size) > > +{ > > + Job *ajob; > > + > > + TRACE_ENTER(); > > + > > + while (job_.size() > size && (ajob = dequeue()) != nullptr) { > > + delete ajob; > > + } > > + > > + TRACE_LEAVE(); > > +} > > + > > // > > std::queue<Job*> Fifo::job_; > > // > > diff --git a/src/amf/amfd/imm.h b/src/amf/amfd/imm.h > > --- a/src/amf/amfd/imm.h > > +++ b/src/amf/amfd/imm.h > > @@ -151,6 +151,8 @@ public: > > static void empty(); > > > > static uint32_t size(); > > + > > + static void trim_to_size(const uint32_t size); > > private: > > static std::queue<Job*> job_; > > }; > > > > > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel