Re: [FOSSology] Survey - for next version of FOSSology
On Aug 10, 2009, at 12:20 PM, Matt Taggart wrote: Are you saying you would rather move it to a separate package than remove it from the main package? I'm saying it should remain the the upstream fossology tarball, but for Debian I can put it in a separate package that won't be required (the fossology-agents package will only Recommends instead of Depends). BTW one other thing that needs to be fixed for this to happen is that I'm not sure the selftest agent is able to deal with 'missing' agents. Yes, selftest does need to be changed for this. There may be some other code that has a concept of default agents which would also need to change. In the future, selftest should read the db to find the agents to test. For now, it's easy to remove pkgmetagetta from selftest (so there would be no test for it). I'd still like to know if anyone is using pkgmetagetta. Are you saying that we should keep it because someone somewhere might use the data even though fossology doesn't? Yes. FOSSology is still sort of a one-trick-pony, but I think the metadata agent is something that fits into the bigger picture. I agree. We did it mostly as an example to show our potential. But now I'm thinking that we shouldn't have released it until we actually use the data. Right now it's a cost with no benefit unless people are doing direct db queries or are counting on a UI to use it in the future. The only feedback I've gotten so far is from folks who do not know what this agent does. So far we have the following options: 1) Do nothing. Leave as is. 2) Make pkgmetagetta a separate package (requires minor code changes and pkgmetagetta will never do selftest even if installed) 3) Leave packaging alone, but make same minor code changes as in 2, so that pkgmetagetta is NEVER run. I don't feel strongly about this but am asking for feedback to find out if users would like a change. Votes taggart: option 1 or 2 bobg: option 1, 2, or 3 ;-) Bob Gobeille b...@fossology.org ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Survey - for next version of FOSSology
I agree. We did it mostly as an example to show our potential. But now I'm thinking that we shouldn't have released it until we actually use the data. Right now it's a cost with no benefit unless people are doing direct db queries or are counting on a UI to use it in the future. The only feedback I've gotten so far is from folks who do not know what this agent does. So far we have the following options: 1) Do nothing. Leave as is. 2) Make pkgmetagetta a separate package (requires minor code changes and pkgmetagetta will never do selftest even if installed) 3) Leave packaging alone, but make same minor code changes as in 2, so that pkgmetagetta is NEVER run. I don't feel strongly about this but am asking for feedback to find out if users would like a change. Votes taggart: option 1 or 2 bobg: option 1, 2, or 3 ;-) Mary: option 2 or 3 However, I think selftest should always be performed if an agent is installed. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Q regarding the email notification in ver 1.1
On August 10, 2009 Bob Gobeille wrote: On Aug 10, 2009, at 2:25 PM, Todd Beverly wrote: I recently install fossology v 1.1 and made the decision to not use the /repo/ sub directory in Apache. I got everything to work, except the link in the email message always returns http://$hostname/repo/?...I tracked this down to routine /usr/local/bin/fo_notify, which reads file Db.conf to extract the database server name, calculate the fully qualified hostname and then concantenates http:// $hostname and /repo/?mod=showjobshistory=1upload=$upload_id If no one has worked on this, I wouldn't mind changing fo_notify to accept an optional string that defines the URL prefix (everything up to the ?). My original thought was to look for an optional file Url.conf for the prefix and then look for Db.conf if Url.conf isn't found. I was wondering if there was a more logical place for this option (or maybe I misconfigured the original installation?) That's a mistake for fo_notify to hardcode in the path. I like your solution. Maybe a better place than a Url.conf could be Fossology.conf. Then we would have a catch all conf file for this and whatever other user configuration variables we need to set. The conf file would default to http://$hostname/repo/ I took the liberty of filing this as a new bug, so that we do not lose track of it! http://bugs.linux-foundation.org/show_bug.cgi?id=380 Dan ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] unpack size/time
Bob, I agree with you, the best user experience would come from backup and restore of the entire repository. In our case, however, there's a big problem: We do not have enough unallocated disk space on the RFO cluster to implement this. So I don't see a problem in recommending the full repo backup method, but in our case we will not be able to implement this, and will have to use the gold files-only method. Dan -Original Message- From: fossology-boun...@fossology.org [mailto:fossology-boun...@fossology.org] On Behalf Of Gobeille, Robert Sent: Monday, August 10, 2009 2:18 PM To: Ma, Dong (vinc...@gdcc-bj-most) Cc: fossology@fossology.org Subject: Re: [FOSSology] unpack size/time Hi Vincent, As per my previous email, this is probably not a great way to back up a fossology database. But it's good functionality to have because we could add a feature that allows users to remove all non-reused files except gold to save disk space. I don't see any bad ramifications except for the user experience (waiting for unpack). The code changes needed in unpack are minimal. All the places in the UI that read files will have to change. So you don't want to create this backup solution until all the code changes are done. I think you also need to provide user instructions to backup the entire repo. That is: stop scheduler pg_dumpall myfile backup myfile backup repo gold, files, and license start scheduler Bob On Aug 10, 2009, at 1:31 AM, Ma, Dong (vinc...@gdcc-bj-most) wrote: Hi Bob, If no dependencies with the unpacked files at the backup point, I think we should not do unpack on the fly at the restore time. We just restore the gold files and give the user's opinion to unpack the gold files as they needed. Is this opinion make sense? Or this will bring some bad ramifications? Thanks, Vincent -Original Message- From: Gobeille, Robert Sent: Friday, August 07, 2009 10:56 PM To: Ma, Dong (vinc...@gdcc-bj-most) Cc: fossology@fossology.org Subject: Re: unpack size/time On Aug 7, 2009, at 2:20 AM, Ma, Dong (vinc...@gdcc-bj-most) wrote: Hi Bob, Also have 2 more questions talk with you: 1. Backup all repo files, the incremental disk backup time is not an issue, but the restore process will also cost so much time when restore all the repo file every time(one day or more). I just consider this situation: a. only backup gold(and license) files b. when restore, only unpack the files which have dependency with backup point running jobs (if no dependency with running jobs, will not unpack any gold files in restore) See previous email. There are no dependencies from the interrupted unpack. c. unpack other files with user's demand after restore I am not sure the percentage of 'unpack the files which have dependency with backup point running jobs' with whole repo unpack files, if very little unpacked files should be unpacked in restore process, the time cost will not the issue. May be the time will less than restore all the repo file. I hope this question not confuse you. Restore only the gold (and license) files would typically be much faster than restoring all the repo files. Se my previous email in this thread for times. 2. About ' Allow users to archive uploads.', I have a question is if users have this requirement? If user want this feature of delete the unpacked files? My thought is the users don't care about unpacked files occupied how many disk space and want do delete them if disk spaces is enough. So far, this has not come up as a user requirement. The only thing I've seen from users is the need to delete the entire upload from the database and repository. And we already do that. Bob Gobeille ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Fossology installation troubles : ERROR: Unable to initialize.
Dear Mary, I have read this in the Postegresql wiki page of the French Ubuntu community about the release 8.3 : Il est important de remarquer que lors de l'installation, les bases de données sont créées en unicode et qu'à cause de cela, il ne sera pas possible de créer une base de données dans un autre encodage. that means in English : It's important to notice that during the installation, the database schemes are created with the unicode encoding and due to this, it will not possible to create an other scheme with an other encoding. Best regards, Jonathan 2009/8/10 Laser, Mary mary.la...@hp.com Hi Jonathan, Thanks for the update. I spent several hours Friday examining the PostgreSQL log file you sent me and could not find the source of your problem. I don't think it's due to encoding but, I'll look specifically for that issue. I will also try to recreate the problem you describe below Postgresql doesn't like the database creation with an encoding different from UTF-8, and the fossology database uses the SQL-ASCII encoding. Thanks again, Mary -- *From:* Jonathan Parès [mailto:jonathan.pa...@gmail.com] *Sent:* Sunday, August 09, 2009 9:10 PM *To:* Laser, Mary *Cc:* fossology@fossology.org *Subject:* Re: [FOSSology] Fossology installation troubles : ERROR: Unable to initialize. Dear Mary, I have decided to install CentOS-5.3 as virtual machine on my PC. Fossology is working like a charm now. I would like helping you to fix this problem on Ubuntu 8.04, but I have done a backup on my virtual machine and now I don't have fossology installed on it anymore. But it seems that the 8.3 release of Postgresql doesn't like the database creation with an encoding different from UTF-8, and the fossology database uses the SQL-ASCII encoding ... This information must be confirmed, may be the problem comes from this. Best regards, Jonathan 2009/8/6 Laser, Mary mary.la...@hp.com Hi Jonathan, The message Applying database schema comes from the function ApplySchema in /usr/share/fossology/www/plugins/core-schema.php. Immediately after this is printed, we check for the existence of the core-schema.dat file (also in /usr/share/fossology/www/plugins/) followed by a validity check. If these 2 tests pass, the core-schema.dat file is read and applied to the database. Since we are not seeing any messages in the postgresql log file indicating the schema is being applied, I'd guess that one of the 2 previous tests failed. Please check to see that the core-schema.dat file is present, readable and valid. You can check it against this copy: http://fossology.svn.sourceforge.net/viewvc/fossology/trunk/fossology/ui/plugins/core-schema.dat?revision=2095view=markup . Mary -- *From:* Laser, Mary *Sent:* Wednesday, August 05, 2009 9:26 AM *To:* 'Jonathan Parès' *Cc:* fossology@fossology.org *Subject:* RE: [FOSSology] Fossology installation troubles : ERROR: Unable to initialize. Hi Jonathan, Thank you for this additional information. I will take a look at fossinit to see why the error is occurring. Mary -- *From:* Jonathan Parès [mailto:jonathan.pa...@gmail.com] *Sent:* Wednesday, August 05, 2009 1:27 AM *To:* Laser, Mary *Cc:* fossology@fossology.org *Subject:* Re: [FOSSology] Fossology installation troubles : ERROR: Unable to initialize. Dear Mary, It seems it's a problem of script and not of my database. The postgresql log file is not updated/modified by the execution of the fo-postinstall script. Consequently I think it's a problem of script, inside the fo-postinstall script or the other ones called by the fo-postinstall. Following the result of the running with the -x parameter of the script : virtual...@ubuntu-8-04-vbx:~$ sudo bash -x /usr/local/lib/fossology/fo-postinstall ++ getopt -o adwseoh --long agent,database,web,web-only,scheduler,scheduler-only,everything,overwrite,help -n fo-postinstall -- + OPTS=' --' + '[' 0 '!=' 0 ']' + eval set -- ' --' ++ set -- -- + '[' ' --' = ' --' -o ' --' = ' -o --' ']' + EVERYTHING=1 + true + case $1 in + shift + break + '[' ']' + '[' ']' + '[' 1 ']' + echo '*** Running postinstall for everything ***' *** Running postinstall for everything *** + AGENT=1 + DATABASE=1 + WEBONLY=1 + SCHEDULERONLY=1 ++ id -u + '[' 0 '!=' 0 ']' + '[' 1 ']' + /usr/local/lib/fossology/dbcreate *** Setting up the FOSSology database *** NOTE: fossology database already exists, not creating *** Checking for plpgsql support *** NOTE: plpgsql already exists in fossology database, good + '[' 1 ']' + '[' -e /usr/local/etc/fossology/RepPath.conf ']' ++ cat /usr/local/etc/fossology/RepPath.conf + REPO=/srv/fossology/repository + echo '*** Creating user and group ***' *** Creating user and group *** + grep -q '^fossy:' /etc/group + echo 'NOTE: group '\''fossy'\'' already exists, good.' NOTE: group
