Hi Kotrappa, We are working on a couple of different tools here at UNO that may address what you are asking. In particular:
We are creating a command line interface. The interface will (in short) (1) unpack packages, (2) check file SHA1 off cache, (3) return license information for found SHA1s, (4) send files with no license information found to FOSSology (as tar), (5) combine results from cache and FOSSology into final SPDX. Preliminary forms of this code have been contributed to The Yocto project and can be found here: http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/spdx.bbclass?id=a0904066865c9792033d6c87c270966113b6ae66 This may not precisely solve your issue but it is the production of an SPDX document as part of an automated process, using caching to help reduce the load on FOSSology. In any case, we are advancing this work and could talk about it more if you would like. If you are looking to compare two different SPDX documents (i.e. two different versions of the same package), we are also developing a tool for that but it is the comparison of post-production of SPDX docs. Regards, Matt -- Mutual of Omaha Associate Professor of Information Systems University of Nebraska at Omaha Vita <http://myweb.unomaha.edu/~mgermonprez/> Open Communities Lab <http://ocrl.unomaha.edu/> NSF Grant on Open Communities <http://1.usa.gov/17mbd1Z> On Fri, Feb 28, 2014 at 6:50 AM, <[email protected]> wrote: > Hi Venkata, > > > > We need an option to retrieve .sdpx file results for the already scanned > files by command line. > > Because we are automating to scan big file size ( around 1gb or so) and > using wget option to scan and get .spdx results, nearly it takes around > 20-30 mins for 1GB file. And also we also need those files to exist in > database so that we can compare results during next time scan. > > > > Is there any work around or it is required to write new PHP page for this > requirement? > > > > Best regards > > Kotrappa. > > > > > > *From:* Venkata Krishna Penmatsa [mailto:[email protected]] > *Sent:* Monday, February 24, 2014 9:46 PM > *To:* Kotrappa DeverShetty (WT01 - Manufacturing & Hi Tech) > *Cc:* Matt Germonprez; [email protected]; Liang Cao; > [email protected]; Akanksha Singh (WT01 - Manufacturing & Hi > Tech); Ravishankar T.S (WT01 - Manufacturing & Hi Tech) > *Subject:* Re: fossology Digest, Vol 75, Issue 18 > > > > Hi Kotrappa, > > > > Command line does only real time scan of the subject packages and we don't > have option to retrieve .spdx file results of already scanned packages. > > But the web UI(the website) allows us to select scanned package and > generate spdx docs. > > > > Thanks, > > Venkata K Penmatsa > > Open Source Research Lab > > University of Nebraska at Omaha > > > > On Mon, Feb 24, 2014 at 12:34 AM, <[email protected]> wrote: > > Hi Venkata, > > > > Thanks for the information. > > > > I have one more query. I think following command sends the local file ( > --post-file =../time-1.7.tar) to webserver, waits for file to be scanned > and gets .spdx results from the webserver. > > Do we have an option where a big file is already uploaded to fossology > server, and it has been already scanned, I need to just get the .spdx file > results from the server by command line? > > wget -qO - --no-check-certificate --post-file=./time-1.7.tar --timeout=0 " > http://localhost/repo/?mod=spdx_license_once&noCopyright=false&jsonOutput=false&fullSPDXFlag=false&packageNameInLog=time-1.7.tar"; > > time-1.7.tar.spdx > > > > Best regards > > Kotrappa. > > > > > > *From:* Venkata Krishna Penmatsa [mailto:[email protected]] > *Sent:* Friday, February 21, 2014 8:26 PM > > > *To:* Kotrappa DeverShetty (WT01 - Manufacturing & Hi Tech) > *Cc:* Matt Germonprez; [email protected]; Liang Cao; > [email protected]; Akanksha Singh (WT01 - Manufacturing & Hi > Tech); Ravishankar T.S (WT01 - Manufacturing & Hi Tech) > *Subject:* Re: fossology Digest, Vol 75, Issue 18 > > > > Hi Kotrappa, > > > > The reason is extracted license(the licenses not been defined in SPDX > license list) is indexed as "LicenseRef-" in full spdx document. > > In your case, you can find "No_license_found" license is defined as > "LicenseRef-3" in [email protected]_localhost.spdx. > > > > Thanks, > > Venkata K Penmatsa > > Open Source Research Lab > > University of Nebraska > > > > On Fri, Feb 21, 2014 at 5:25 AM, <[email protected]> wrote: > > Hi Venkata, > > > > Thanks for information and clarification. > > > > Sorry, there was a mistake in localhost address instead of https, it > should be http. With https, always connection to localhost failed with wget > command. > > > > Now I could get .spdx out put from localhost. > > Please find the .spdx and logs files for both [fullSPDXFlag] flag set as > true and false, but why if fullSPDXFlag set to flase gives .spdx out put > with "No License Found" entries for all the files which have license if > [fullSPDXFlag] is set to true. > > > > Best regards > > Kotrappa > > > > > > *From:* Venkata Krishna Penmatsa [mailto:[email protected]] > *Sent:* Friday, February 21, 2014 3:07 AM > *To:* Kotrappa DeverShetty (WT01 - Manufacturing & Hi Tech) > *Cc:* Matt Germonprez; [email protected]; Liang Cao; > [email protected]; Akanksha Singh (WT01 - Manufacturing & Hi > Tech); Ravishankar T.S (WT01 - Manufacturing & Hi Tech) > *Subject:* Re: fossology Digest, Vol 75, Issue 18 > > > > Hi Kotrappa, > > > > We are looking into the SPDX document output issue and it would really > help us if you could provide logs while generating the SPDX document. > > > > I would also suggest you to try using > *https://localhost/repo/?mod=spdx_license_once&noCopyright=true&jsonOutput=true&fullSPDXFlag=true&packageNameInLog=time-1.7.tar > <https://localhost/repo/?mod=spdx_license_once&noCopyright=true&jsonOutput=true&fullSPDXFlag=true&packageNameInLog=time-1.7.tar>*. > Because > the SPDXFlag should be true to generate an SPDX document. > > > > Regarding your other questions: > > > > 1. Do you support fossology+spdx module as a package, which is easier to > upgrade by sudo apt-get command? Instead of everytime manually > uninstall/installing the module. > > > > This is a great suggestion and one that we have been considering. We will > keep you posted on any progress in this area and would welcome any support > as well. > > > > 2. Why fossology+spdx module does not have version information? > > > > Thank you very much for the suggestion and we have updated the version > information. Please find the VERSION file at:* > https://github.com/spdx-tools/fossology-spdx > <https://github.com/spdx-tools/fossology-spdx>* > > > > Thanks, > > Venkata K Penmatsa > > Open Source Research Lab > > University of Nebraska at Omaha > > > > > > > > On Thu, Feb 20, 2014 at 12:20 PM, Liang Cao <[email protected]> wrote: > > > > ---------- Forwarded message ---------- > From: <[email protected]> > Date: Thu, Feb 20, 2014 at 9:46 AM > Subject: fossology Digest, Vol 75, Issue 18 > To: [email protected] > > > Send fossology mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.fossology.org/mailman/listinfo/fossology > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of fossology digest..." > > > Today's Topics: > > 1. Re: query regarding fossology+spdx command line support > ([email protected]) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Feb 2014 15:45:26 +0000 > From: <[email protected]> > To: <[email protected]>, <[email protected]> > Cc: [email protected], [email protected], > [email protected] > Subject: Re: [FOSSology] query regarding fossology+spdx command line > support > Message-ID: > <[email protected]> > Content-Type: text/plain; charset="us-ascii" > > Hi Matt, > > Thanks for the information > I was able to get .spdx output from below command line from > fossologyspdx.ist.unomaha.edu server. > > wget -qO - --no-check-certificate --post-file=./time-1.7.tar --timeout=0 " > https://fossologyspdx.ist.unomaha.edu/?mod=spdx_license_once&noCopyright=false&jsonOutput=false&fullSPDXFlag=false&packageNameInLog=time-1.7.tar"; > > time-1.7.tar.spdx > > On fresh Ubuntu 13.10 system, today I installed Fossology 2.4.0, with > latest SPDX module as available from > https://github.com/spdx-tools/fossology-spdx by downloading zip file on > right hand side of the page. I wanted to try same command as above on > localhost server,but I did not get any output of spdx in localhost. > > wget -qO - --no-check-certificate --post-file=./time-1.7.tar --timeout=0 " > https://localhost/repo/?mod=spdx_license_once&noCopyright=false&jsonOutput=false&fullSPDXFlag=false&packageNameInLog=time-1.7.tar"; > > time-1.7.tar.spdx > > Is there something I am missing here. I assume the latest spdx source is > supporting [fullSPDXFlag]. > > Other general queries I do have are below. > > > 1. Do you support fossology+spdx module as an package, which is > easier to upgrade by sudo apt-get command? instead of everytime manually > uninstall/installing the module. > > 2. Why fossology+spdx module does not have version information? > > > > Best regards > Kotrappa. > > > > > From: Matt Germonprez [mailto:[email protected]] > Sent: Saturday, February 08, 2014 6:39 PM > To: Gobeille, Robert > Cc: Kotrappa DeverShetty (WT01 - Manufacturing & Hi Tech); > [email protected]; Akanksha Singh (WT01 - Manufacturing & Hi > Tech); Ravishankar T.S (WT01 - Manufacturing & Hi Tech) > Subject: Re: [FOSSology] query regarding fossology+spdx command line > support > > Hi Kotrappa, > > Liang Cao has added an option to generate a full SPDX document in TAG > format from the command line. He has also provided a nice overview of how > to work with the source. > > The source is pushed to here: > > https://github.com/spdx-tools/fossology-spdx/blob/master/src/spdx/ui/spdx_license_once.php > > The documentation is here: > https://github.com/spdx-tools/fossology-spdx/wiki/Fossology-spdx-web-api > > The option for [fullSPDXFlag] is added. > [fullSPDXFlag]: true/false. Only when this option is set to "true", low > definition version of the full SPDX contents are output. Skipping this > option equals setting it to "false." This option should be set to "true" > when you want to generate an SPDX document from the command line. > > You could generate a mypackagename.spdx by running a command like the > following: > wget -qO - --no-check-certificate --post-file=./[mypackagename] > --timeout=0 " > https://domain/?mod=spdx_license_once&noCopyright=false&jsonOutput=false&fullSPDXFlag=true&packageNameInLog=[mypackagename] > < > https://domain/?mod=spdx_license_once&noCopyright=false&jsonOutput=false&fullSPDXFlag=true&packageNameInLog=%5bmypackagename%5d>" > > [mypackagename].spdx > > for example: > > wget -qO - --no-check-certificate --post-file=./time-1.7.tar --timeout=0 " > https://fossologyspdx.ist.unomaha.edu/?mod=spdx_license_once&noCopyright=false&jsonOutput=false&fullSPDXFlag=false&packageNameInLog=time-1.7.tar"; > > time-1.7.tar.spdx > > If you have any questions, please feel free to contact me or Liang. We are > happy to help. > > Regards, > Matt Germonprez and Liang Cao > > -- > Mutual of Omaha Associate Professor of Information Systems > University of Nebraska at Omaha > Vita<http://myweb.unomaha.edu/~mgermonprez/> > Open Communities Lab<http://ocrl.unomaha.edu/> > NSF Grant on Open Communities<http://1.usa.gov/17mbd1Z> > > On Fri, Feb 7, 2014 at 10:07 AM, Gobeille, Robert <[email protected] > <mailto:[email protected]>> wrote: > On Feb 7, 2014, at 4:14 AM, <[email protected]<mailto: > [email protected]>> <[email protected]<mailto: > [email protected]>> wrote: > We are using Fossology on a local sever for scanning some of packages for > License/copyright info. > Recently we have installed Fossology+SPDX module on a local PC, and this > gives spdx results as well. > > Spdx.org<http://Spdx.org> gives open source tools to convert .rdf to > .spdx, .spdx to .rdf, .xls to.rdf etc. > Reference http://spdx.org/spdx-tools/tools-from-the-spdx-workgroup > > We would like to know after Fossology+SPDX scans a package and gives > results in sdpx format, > Is there any command line tools to execute on command prompt in Local PC > using localhost server running Fossology agents to get results in spdx > format. > ( Please note I cannot use Web Interface version of > https://fossologyspdx.ist.unomaha.edu/?mod=Default because packages > cannot be uploaded to public) > > I mean, I should be able to get results something like mypackage_name.spdx > or mypackage_name.rdf which complies with SPDX format > specified in spdx.org<http://spdx.org>, which I can use as an input to > spdx open source tools for conversion, comparison etc. > > Hi Kotrappa, > > I'm confused why you mention that you cannot use > fossologyspdx.ist.unomaha.edu<http://fossologyspdx.ist.unomaha.edu> since > you have installed the spdx module on your own local machine. You > shouldn't have to use the unomaha machine since you have it installed > locally. > > Since you have installed the FOSSology+SPDX module on your local PC, then > you can create spdx files (tag files). That option is the default but is > specified in the "Output File Type" pull down on the SPDX Edit screen. So > though we call it a .tag file, I think that is the same as the .spdx file. > > The command line (web api) doc is at: > > https://github.com/spdx-tools/fossology-spdx/wiki/Fossology-SPDX-Web-API > > However, this does not generate the full .spdx (tag) output. > > If I have not understood your question completely, please ask again. > Liang Cao is the author of the SPDX module and he is on this list as well. > > Thanks, > Bob Gobeille > > _______________________________________________ > fossology mailing list > [email protected]<mailto:[email protected]> > http://lists.fossology.org/mailman/listinfo/fossology > > > > > > The information contained in this electronic message and any attachments > to this message are intended for the exclusive use of the addressee(s) and > may contain proprietary, confidential or privileged information. If you are > not the intended recipient, you should not disseminate, distribute or copy > this e-mail. Please notify the sender immediately and destroy all copies of > this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient > should check this email and any attachments for the presence of viruses. > The company accepts no liability for any damage caused by any virus > transmitted by this email. > > www.wipro.com > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.fossology.org/pipermail/fossology/attachments/20140220/d5b54887/attachment.html > > > > ------------------------------ > > _______________________________________________ > fossology mailing list > [email protected] > http://lists.fossology.org/mailman/listinfo/fossology > > > End of fossology Digest, Vol 75, Issue 18 > ***************************************** > > > > > > The information contained in this electronic message and any attachments > to this message are intended for the exclusive use of the addressee(s) and > may contain proprietary, confidential or privileged information. If you are > not the intended recipient, you should not disseminate, distribute or copy > this e-mail. Please notify the sender immediately and destroy all copies of > this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient > should check this email and any attachments for the presence of viruses. > The company accepts no liability for any damage caused by any virus > transmitted by this email. > > www.wipro.com > > > > The information contained in this electronic message and any attachments > to this message are intended for the exclusive use of the addressee(s) and > may contain proprietary, confidential or privileged information. If you are > not the intended recipient, you should not disseminate, distribute or copy > this e-mail. Please notify the sender immediately and destroy all copies of > this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient > should check this email and any attachments for the presence of viruses. > The company accepts no liability for any damage caused by any virus > transmitted by this email. > > www.wipro.com > > > > The information contained in this electronic message and any attachments > to this message are intended for the exclusive use of the addressee(s) and > may contain proprietary, confidential or privileged information. If you are > not the intended recipient, you should not disseminate, distribute or copy > this e-mail. Please notify the sender immediately and destroy all copies of > this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient > should check this email and any attachments for the presence of viruses. > The company accepts no liability for any damage caused by any virus > transmitted by this email. > > www.wipro.com >
_______________________________________________ fossology mailing list [email protected] http://lists.fossology.org/mailman/listinfo/fossology
