Hi Bob, Understood. That will be great if FOSSology can run multiple agents in parallel on one archive. Also I agree with what you said in http://www.fossology.org/issues/3909 "So I'm raising the priority. It is still marked for 2.5 but I don't expect it to receive any time until 2.6"
-Larry From: Gobeille, Robert Sent: Wednesday, February 12, 2014 7:59 AM To: Shi, Yao-Bin (Larry, GBU-HPSW, Cloud and OS) Cc: Cirujano Cuesta, Silvano; [email protected] Subject: Re: [FOSSology] Multiple Nomos instances Hi Larry, I do not want to speak for Silvano, but we have other users requesting this as well. The single reason is speed. You submit a large archive to be scanned and it can take many hours (even days for a large distro). This is frustrating on multicore machines that still have many cycles they could be using. At least one user that I know of uses the standalone nomos and copyright and scripts them so they can have multiple parallel streams processing a single archive file. The original design of the 2.0 scheduler allowed the scheduler to send multiple agents lists of files to process. This is how we planned on multithreading the agent execution on a single upload. However, this implementation was not completed. The current scheduler does well running simultaneous agents on multiple uploads, but to multitask on a single upload. This would be a great improvement for fossology. Bob Gobeille On Feb 11, 2014, at 5:47 AM, Shi, Yao-Bin (Larry, GBU-HPSW, Cloud and OS) <[email protected]<mailto:[email protected]>> wrote: Hi Silvano, What is your purpose for parallelization of nomos? Run multiple nomos instances in parallel stably is not a easy task. One reason why we do not add this feature is because newer nomos is(should be) more accurate than the older one, so just schedule the latest nomos. If you just want to compare the license report on different version of nomos, there is one workaround: 1. Unpack one package to one folder through ununpack agent from command line. 2. Run different version of nomos from command line on that folder to get license report individually, then compare them. If this workaround works for your goal and do not how to use this workaround, please let me know, will give you an example in detail. -Larry From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Gobeille, Robert Sent: Tuesday, February 04, 2014 1:12 AM To: Cirujano Cuesta, Silvano Cc: [email protected]<mailto:[email protected]> Subject: Re: [FOSSology] Multiple Nomos instances Hi Silvano, We took the parallelization out in 2.0 because we were trying to make the scheduler more robust. Also, the implementation was not efficient and involved a great deal of communication overhead. We did have a plan for a more efficient scheme in 2.0 but it was never implemented. We need to give this some more priority. Thanks for reminding us of the importance of this feature. I'm going to raise the priority and make you a "watcher" on the issue so you will see any progress. Thanks, Bob Gobeille On Feb 3, 2014, at 2:41 AM, Cirujano Cuesta, Silvano <[email protected]<mailto:[email protected]>> wrote: Hi Bob, Thank you for the information. I was expecting to get parallelization as FOSSology 1.x does it (one job can be split among different agents), but I already noticed that it works as you explained. A colleague of mine already pointed out to me, that such an enhancement is already planed [1]. [1] http://www.fossology.org/issues/3909 Silvano From: Gobeille, Robert [mailto:[email protected]] Sent: Freitag, 31. Januar 2014 21:48 To: Cirujano Cuesta, Silvano Cc: [email protected]<mailto:[email protected]> Subject: Re: [FOSSology] Multiple Nomos instances Hi Silvano, That should be all you need, though you might want to try an alternate localhost line: [HOSTS] localhost = localhost AGENT_DIR 10 All the agents are currently single threaded, so only one nomos will run on any upload. However, if you have multiple simultaneous uploads, then multiple nomos's should start to service each upload. Bob Gobeille On Jan 24, 2014, at 8:29 AM, Cirujano Cuesta, Silvano wrote: I'm trying to get multiple Nomos instances running in parallel in a single machine configuration, but without success. The block [HOSTS] of the file /usr/local/etc/fossology/fossology.conf contains this entry: localhost = localhost /usr/local/etc/fossology 10 And in the block [default] of the file /usr/local/etc/fossology/mods-enabled/nomos/nomos.conf this is the entry: max = -1 I also tried modifying those values, restarting the server... As far as I understood the documentation, those two are the only parameters that control how many Nomos instances can run in parallel. Am I doing something wrong? Regards, Silvano _______________________________________________ fossology mailing list [email protected]<mailto:[email protected]> http://lists.fossology.org/mailman/listinfo/fossology _______________________________________________ fossology mailing list [email protected]<mailto:[email protected]> http://lists.fossology.org/mailman/listinfo/fossology
_______________________________________________ fossology mailing list [email protected] http://lists.fossology.org/mailman/listinfo/fossology
