Hi Bob, I realize you guys have your hands busy and deep in the new release(s) of fossology, but I just wanted to see if this was anywhere near completion (v1.3 release)?
I'm installing our new server we got in last week and it's a monster!!! However, with the postgres CPU utilization pegged most of the time (like I mentioned in this thread) I wanted to know if installing the 1.3 release would be possible for our RHEL5 64-bit monster-server. I'm probably asking for the world here but again, just curious as far as how close and/or timeline for this? I'm going to install whatever latest release is out right now which appears to still be fossology-1.1.0-1.el5.x86_64.rpm for me. We were just hoping since we invested quite a bit of money into this new server, we'd like to see the improvements on both the server-side and hopefully any fixes on software side to help the psql stuff out. No way of getting an early release v1.3 right? :-) -Regards, Gibran On Tue, Dec 22, 2009 at 6:52 AM, Gobeille, Robert <[email protected]>wrote: > Hi Gibran, > The scheduler does have some serious issues (I've seen what you are > describing). This really became apparent in the release we are working on > (1.2) where the new license analysis agent is 20x faster than the one you > are using. I've been spending a lot of time on it but the solutions are > pretty involved. Right now we are planning on the 1.3 release to be where > we fix the scheduler (and other issues). > > For now, > > 1) Are you only interested in license analysis? If so, only ever start the > license agent, not all the agents. In other words, choose License Analysis > but NOT MIME-type Analysis, Metadata Analysis, or Spec File Analysis. > These other agents (especially pkgmettagetta does a LOT of database > writes). > > 2) Since you already have these other agents running (you mention > pkgmetagetta), go to the job queue display (Jobs -> Queue -> Details) and > delete that job. This is probably the one causing the majority of your > problems. > > 3) Check your Scheduler.conf to make sure it is not trying to run too many > simultaneous processes. If you want someone else to look at it, send it to > the list. > > Bob Gobeille > > > On Dec 21, 2009, at 6:12 PM, Furosh One wrote: > > Hey guys and gal, > > I'm not sure if this is truly a problem but my collegue uploaded a rather > large amount of data to be processed and it appears to have been uploaded > and scheduled correctly, however it seems like the server has postgres > pegged at around 100% for the majority of the time. > > I was reading and searching your archives to see if I could find anyone > else who had the same issue and only found this: "Error while uploading" on > Fri, 22 May 2009 08:56:39 -0700 > > After looking at some of the areas Bob suggested this guy check I thought I > should look into each of those too. > > > >1) your scheduler is in some loop > >Do a top and see if fossology_scheduler is using lots of cpu time. > > This doesn't apply other than it's the Postgres process that is at 100% > > > >2) you need to do a vacuum analyze on your database > > It seems like the DB is doing a lot being at 100%, could I do this while > its at 100%. It has 2 CPU's so (1)=100% and (2)= around 20% utilization. > > > >3) you are memory starved. > > We seem to be okay on memory, this server currently has 4GB RAM. > > >4) your postgresql statement timeout is set very low > > >In postgresql.conf the "statement_timeout" variable > > Our statement_timeout is #'d commented out, > #statement_timeout = 0 > > Should I change this? > > >5) your disk performance is poor (perhaps due to other processes) > > This doesn't seem to be an issue at the moment for us. > > > In addition, the agents seem to spit off a small process like pkgmetagetta > or licinspect every once in a while but ends really quick. > I also look at the Scheduler / Status and it's updating and showing running > on localhost and agents, but never seems to get to the filter_license or > bsam_engine parts. > > > Anyway, these questions weren't to me directly but was trying to look for > help in your archive and thought I'd send you some info to go on to see if > you had any suggestions or recommendation. > Please let me know what I could check/do to correct this. I don't know much > about DB optimization or administration on the database. > > > -Regards, > Gibran > > <ATT00001..txt> > >
_______________________________________________ fossology mailing list [email protected] http://fossology.org/mailman/listinfo/fossology
