[FOSSology] Cygwin Analysis causing scheduler restarts
Good Morning Everyone. On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? Any suggestions, please? Thanks, Thanks, Ray W. CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed and may contain confidential and privileged information protected by law. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of the e-mail is strictly prohibited. Please notify the sender immediately by return e-mail and delete all copies from your system. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Cygwin Analysis causing scheduler restarts
Did your disk fill up? Take a look at the log file to see if you can identify what was being processed when the scheduler died. Mary From: fossology-boun...@fossology.org [mailto:fossology-boun...@fossology.org] On Behalf Of Westphal, Raymond W Sent: Monday, May 23, 2011 6:32 AM To: fossology@fossology.org Subject: [FOSSology] Cygwin Analysis causing scheduler restarts Importance: High Good Morning Everyone. On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? Any suggestions, please? Thanks, Thanks, Ray W. CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed and may contain confidential and privileged information protected by law. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of the e-mail is strictly prohibited. Please notify the sender immediately by return e-mail and delete all copies from your system. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Cygwin Analysis causing scheduler restarts
Hi Ray, I just grabbed the source from the cygwin cvs, tared and uploaded it. On the system I used it took 7 minutes. http://repo.fossology.org/simpleIndex.php?mod=nomoslicenseshow=detailupload=148item=50608165 We have seen the scheduler problem you describe before. I think, as long as you aren't hitting some system resource, stopping the scheduler and restarting it might take care of the problem (btw, v2.0 has a new scheduler). Bob On May 23, 2011, at 6:31 AM, Westphal, Raymond W wrote: On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Cygwin Analysis causing scheduler restarts
Good Morning Bob/Good Morning Mary. Here is the info. from the log just prior to where the scheduler started its cycle of not responding and then restarting. 2011-05-20 13:51:46 scheduler[8691] : Child[1] 'agent=unpack host=localhost ' state=FREEING(2) @ Fri May 20 13:51:46 2011 2011-05-20 13:51:46 scheduler[8691] : Child[1] 'agent=unpack host=localhost ' state=FREE(1) @ Fri May 20 13:51:46 2011 2011-05-20 13:51:48 scheduler[8691] : Child[2] 'agent=adj2nest host=localhost ' state=SPAWNED(4) @ Fri May 20 13:51:48 2011 2011-05-20 13:51:48 scheduler[8691] : Child[2] 'agent=adj2nest host=localhost ' state=READY(5) @ Fri May 20 13:51:48 2011 2011-05-20 13:57:42 scheduler[8693] : *** Scheduler not responding: killing and restarting *** 2011-05-20 13:57:42 scheduler[8693] : *** Exiting fossology-scheduler PID 8691 QUIT *** 2011-05-20 13:58:46 scheduler[8693] : *** Scheduler restarted successfully by fo_watchdog *** 2011-05-20 13:58:46 scheduler[13944] : Log opened 2011-05-20 13:58:46 scheduler[13944] : Scheduler started. Build version: 1.2.1~3507, exported. I don't know of any resource issues. Disk did not fill up although it grew to 93% with about 2.5GB remaining. I think the ununpack completed. CPU and memory were apparently not an issue. The load average did start to climb slowly after the scheduler kept failing. I think the restarts of the scheduler leave select queries running and generates new queries. So when you watch top you can see the 4 or 5 select queries as the top processes. Bob - I presume your system is running 1.4.0. Can I do an rpm -Uvh to the 1.3.0 Fossology? I noticed the 1.4.0 rpm is for Red Hat 6. We are not running any Red Hat 6 yet. You think the 1.4.0 rpm will run on Red Hat 5? Thanks for the quick response. Ray W. From: Bob Gobeille [mailto:bob.gobei...@hp.com] Sent: Monday, May 23, 2011 9:22 AM To: Westphal, Raymond W Cc: fossology@fossology.org Subject: Re: [FOSSology] Cygwin Analysis causing scheduler restarts Importance: High Hi Ray, I just grabbed the source from the cygwin cvs, tared and uploaded it. On the system I used it took 7 minutes. http://repo.fossology.org/simpleIndex.php?mod=nomoslicenseshow=detailupload=148item=50608165 We have seen the scheduler problem you describe before. I think, as long as you aren't hitting some system resource, stopping the scheduler and restarting it might take care of the problem (btw, v2.0 has a new scheduler). Bob On May 23, 2011, at 6:31 AM, Westphal, Raymond W wrote: On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed and may contain confidential and privileged information protected by law. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of the e-mail is strictly prohibited. Please notify the sender immediately by return e-mail and delete all copies from your system. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Cygwin Analysis causing scheduler restarts
Hi Ray, Thanks for telling us about the missing RHEL5 packages. They're there now :) When I've seen this type of behavior log file messaging, it usually means the unpack was not responding to the scheduler because it was having trouble unpacking a particular file OR the db was having difficulty processing the requests resulting for the unpack. The postgresql log file might have more clues. BTW, did you turn on autovacuum as we discussed last week? Upgrading (rpm -Uvh) is probably the fastest way around this current problem. Mary From: fossology-boun...@fossology.org [mailto:fossology-boun...@fossology.org] On Behalf Of Westphal, Raymond W Sent: Monday, May 23, 2011 8:46 AM To: fossology@fossology.org Subject: Re: [FOSSology] Cygwin Analysis causing scheduler restarts Good Morning Bob/Good Morning Mary. Here is the info. from the log just prior to where the scheduler started its cycle of not responding and then restarting. 2011-05-20 13:51:46 scheduler[8691] : Child[1] 'agent=unpack host=localhost ' state=FREEING(2) @ Fri May 20 13:51:46 2011 2011-05-20 13:51:46 scheduler[8691] : Child[1] 'agent=unpack host=localhost ' state=FREE(1) @ Fri May 20 13:51:46 2011 2011-05-20 13:51:48 scheduler[8691] : Child[2] 'agent=adj2nest host=localhost ' state=SPAWNED(4) @ Fri May 20 13:51:48 2011 2011-05-20 13:51:48 scheduler[8691] : Child[2] 'agent=adj2nest host=localhost ' state=READY(5) @ Fri May 20 13:51:48 2011 2011-05-20 13:57:42 scheduler[8693] : *** Scheduler not responding: killing and restarting *** 2011-05-20 13:57:42 scheduler[8693] : *** Exiting fossology-scheduler PID 8691 QUIT *** 2011-05-20 13:58:46 scheduler[8693] : *** Scheduler restarted successfully by fo_watchdog *** 2011-05-20 13:58:46 scheduler[13944] : Log opened 2011-05-20 13:58:46 scheduler[13944] : Scheduler started. Build version: 1.2.1~3507, exported. I don't know of any resource issues. Disk did not fill up although it grew to 93% with about 2.5GB remaining. I think the ununpack completed. CPU and memory were apparently not an issue. The load average did start to climb slowly after the scheduler kept failing. I think the restarts of the scheduler leave select queries running and generates new queries. So when you watch top you can see the 4 or 5 select queries as the top processes. Bob - I presume your system is running 1.4.0. Can I do an rpm -Uvh to the 1.3.0 Fossology? I noticed the 1.4.0 rpm is for Red Hat 6. We are not running any Red Hat 6 yet. You think the 1.4.0 rpm will run on Red Hat 5? Thanks for the quick response. Ray W. From: Bob Gobeille [mailto:bob.gobei...@hp.com] Sent: Monday, May 23, 2011 9:22 AM To: Westphal, Raymond W Cc: fossology@fossology.org Subject: Re: [FOSSology] Cygwin Analysis causing scheduler restarts Importance: High Hi Ray, I just grabbed the source from the cygwin cvs, tared and uploaded it. On the system I used it took 7 minutes. http://repo.fossology.org/simpleIndex.php?mod=nomoslicenseshow=detailupload=148item=50608165 We have seen the scheduler problem you describe before. I think, as long as you aren't hitting some system resource, stopping the scheduler and restarting it might take care of the problem (btw, v2.0 has a new scheduler). Bob On May 23, 2011, at 6:31 AM, Westphal, Raymond W wrote: On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed and may contain confidential and privileged information protected by law. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of the e-mail is strictly prohibited. Please notify the sender immediately by return e-mail and delete all copies from your system. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Cygwin Analysis causing scheduler restarts
Hi Mary. I just download the RHEL5 rpm! Hooray! Yes the autovacuum has been configured and appears to be working. Here is an interesting and often repeated section from the postgresql-Sat.log . LOG: could not send data to client: Broken pipe LOG: unexpected EOF on client connection LOG: autovacuum: processing database fossology LOG: could not send data to client: Broken pipe LOG: unexpected EOF on client connection LOG: autovacuum: processing database postgres ERROR: relation license_481 already exists Regarding the upgrade: I'll backup the database (filesystem level backup plus a dump of the database. Then can I simply perform an rpm -Uvh fossology-1.3.0-1.el5.x86_64.rpm followed by a rpm -Uvh fossology-1.4.0-1.el5.x86_64.rpm ? Thanks, Ray W. From: Laser, Mary [mailto:mary.la...@hp.com] Sent: Monday, May 23, 2011 10:19 AM To: Westphal, Raymond W; fossology@fossology.org Subject: RE: [FOSSology] Cygwin Analysis causing scheduler restarts Hi Ray, Thanks for telling us about the missing RHEL5 packages. They're there now :) When I've seen this type of behavior log file messaging, it usually means the unpack was not responding to the scheduler because it was having trouble unpacking a particular file OR the db was having difficulty processing the requests resulting for the unpack. The postgresql log file might have more clues. BTW, did you turn on autovacuum as we discussed last week? Upgrading (rpm -Uvh) is probably the fastest way around this current problem. Mary From: fossology-boun...@fossology.org [mailto:fossology-boun...@fossology.org] On Behalf Of Westphal, Raymond W Sent: Monday, May 23, 2011 8:46 AM To: fossology@fossology.org Subject: Re: [FOSSology] Cygwin Analysis causing scheduler restarts Good Morning Bob/Good Morning Mary. Here is the info. from the log just prior to where the scheduler started its cycle of not responding and then restarting. 2011-05-20 13:51:46 scheduler[8691] : Child[1] 'agent=unpack host=localhost ' state=FREEING(2) @ Fri May 20 13:51:46 2011 2011-05-20 13:51:46 scheduler[8691] : Child[1] 'agent=unpack host=localhost ' state=FREE(1) @ Fri May 20 13:51:46 2011 2011-05-20 13:51:48 scheduler[8691] : Child[2] 'agent=adj2nest host=localhost ' state=SPAWNED(4) @ Fri May 20 13:51:48 2011 2011-05-20 13:51:48 scheduler[8691] : Child[2] 'agent=adj2nest host=localhost ' state=READY(5) @ Fri May 20 13:51:48 2011 2011-05-20 13:57:42 scheduler[8693] : *** Scheduler not responding: killing and restarting *** 2011-05-20 13:57:42 scheduler[8693] : *** Exiting fossology-scheduler PID 8691 QUIT *** 2011-05-20 13:58:46 scheduler[8693] : *** Scheduler restarted successfully by fo_watchdog *** 2011-05-20 13:58:46 scheduler[13944] : Log opened 2011-05-20 13:58:46 scheduler[13944] : Scheduler started. Build version: 1.2.1~3507, exported. I don't know of any resource issues. Disk did not fill up although it grew to 93% with about 2.5GB remaining. I think the ununpack completed. CPU and memory were apparently not an issue. The load average did start to climb slowly after the scheduler kept failing. I think the restarts of the scheduler leave select queries running and generates new queries. So when you watch top you can see the 4 or 5 select queries as the top processes. Bob - I presume your system is running 1.4.0. Can I do an rpm -Uvh to the 1.3.0 Fossology? I noticed the 1.4.0 rpm is for Red Hat 6. We are not running any Red Hat 6 yet. You think the 1.4.0 rpm will run on Red Hat 5? Thanks for the quick response. Ray W. From: Bob Gobeille [mailto:bob.gobei...@hp.com] Sent: Monday, May 23, 2011 9:22 AM To: Westphal, Raymond W Cc: fossology@fossology.org Subject: Re: [FOSSology] Cygwin Analysis causing scheduler restarts Importance: High Hi Ray, I just grabbed the source from the cygwin cvs, tared and uploaded it. On the system I used it took 7 minutes. http://repo.fossology.org/simpleIndex.php?mod=nomoslicenseshow=detailupload=148item=50608165 We have seen the scheduler problem you describe before. I think, as long as you aren't hitting some system resource, stopping the scheduler and restarting it might take care of the problem (btw, v2.0 has a new scheduler). Bob On May 23, 2011, at 6:31 AM, Westphal, Raymond W wrote: On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed and may contain confidential and privileged information protected by law. If you received this e-mail
Re: [FOSSology] Cygwin Analysis causing scheduler restarts
Hello Bob, I learned from the user that the zip file he analyzes is built based upon options he chooses from a Cygwin menu. He believes it contains both source and binary files. So it is very different from the file you analyzed. It is 450MB! I'm going to try the upgrade to Fossology 1.4.0 and see what happens. Thanks, Ray W. From: Bob Gobeille [mailto:bob.gobei...@hp.com] Sent: Monday, May 23, 2011 9:22 AM To: Westphal, Raymond W Cc: fossology@fossology.org Subject: Re: [FOSSology] Cygwin Analysis causing scheduler restarts Importance: High Hi Ray, I just grabbed the source from the cygwin cvs, tared and uploaded it. On the system I used it took 7 minutes. http://repo.fossology.org/simpleIndex.php?mod=nomoslicenseshow=detailupload=148item=50608165 We have seen the scheduler problem you describe before. I think, as long as you aren't hitting some system resource, stopping the scheduler and restarting it might take care of the problem (btw, v2.0 has a new scheduler). Bob On May 23, 2011, at 6:31 AM, Westphal, Raymond W wrote: On Friday my customer uploaded Cygwin for analysis. I think it was a zip file. The job ran from 11:23am until I deleted it at around 8am on Sunday. I had to delete it because it caused Fossology scheduler to keep restarting. And that was causing the load average on the server to slowly climb. Has anyone had any experience with analyzing Cygwin? CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed and may contain confidential and privileged information protected by law. If you received this e-mail in error, any review, use, dissemination, distribution, or copying of the e-mail is strictly prohibited. Please notify the sender immediately by return e-mail and delete all copies from your system. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology