Re: Rubygem packages SPDX status
Hi all, I have fixed up the script a bit and sprinkled some more processing on top, so I come back with more precise data (still rpm-specs as of 2022-11-14, as with previous email) The state is as follows: * Total of rubygem packages in Fedora: ~~~ $ ls rubygem-*.spec | wc -l 495 ~~~ * Total of packages I ran script against (-1 line for the header): ~~~ $ cat rubygems_fedora_spdx_state.csv | wc -l 487 ~~~ * Ruby gems where /Fedora License field/ and /gem2rpm/ output matches and /license-validate/ says they are OK SPDX: *291/495* https://fedorapeople.org/cgit/jackorp/public_git/spdx_rubygems.git/tree/rubygems_fedora_valid_no_action.csv?id=cdbdc41a5ae532df73c76277a939e456214bb0f9 * Ruby gems where only /license-validate/ says they are OK SPDX, but licenses may or may not match between Fedora and upstream: *334/495* https://fedorapeople.org/cgit/jackorp/public_git/spdx_rubygems.git/tree/rubygems_fedora_valid_spdx.csv?id=cdbdc41a5ae532df73c76277a939e456214bb0f9 * Ruby gems where the "or" and "and" are converted to "OR" and "AND" and /license-validate/ says they are OK SPDX: *337/495* ~~~ $ cat rubygems_try_convert_conjunctions.csv | grep -E "(true|false);0" | wc -l 337 ~~~ https://fedorapeople.org/cgit/jackorp/public_git/spdx_rubygems.git/tree/rubygems_try_convert_conjunctions.csv?id=cdbdc41a5ae532df73c76277a939e456214bb0f9 I will follow up with RPM specs from this week later. Regards, Jarek On 11/21/22 14:25, Jarek Prokop wrote: Hi, I have been working on the validation of Rubygem licenses with the SPDX format. all work done so far lives in my fedorapeople space: https://fedorapeople.org/cgit/jackorp/public_git/spdx_rubygems.git/tree/ It is WIP, including the scripts. (I got a bit sidetracked with validating MIT variations.) Note that you have to be in the directory where `rpm-specs-latest.tar.xz` was upacked into. From the total of about 500 rubygems (and with my incomplete script) we have 232 SPDX licenses: ~~~ $ cat current_ok_rb.csv | wc -l 232 ~~~ The real count is probably higher, as some gems are simply MIT or have correct SPDX, however are older so they do not have the license metadata included that I can compare to. See this list for complete info: https://fedorapeople.org/cgit/jackorp/public_git/spdx_rubygems.git/tree/rbgem_license_check_out.csv You can see the `current_ok_rb.csv` in the same repository. So far, I have validated mostly MIT (as those are the easiest to get right), I will need to fix the script before my next update to account for other licenses too. If your gem has multiple valid SPDX license identifiers, but are in conjunction using "and" "or" instead of "AND" "OR", the script will be able to catch that and I'll make a PR to the gem with the converted form should that be the case. There are also a few gems excluded from the search as they do not have a gem in Fedora's sources cache. These gems are: rubygem-morph-cli rubygem-krb5-auth rubygem-asciidoctor rubygem-rgen rubygem-net-irc I have not run the script against them, they are explicitly excluded, since so far I am only comparing the RPM Spec against gem2rpm output. == Plan == I plan to do this in 2 main phases: 1) Initial pass This is just a pass through the packages to note all packages that are already SPDX compatible. 2a) Inspecting and fixing This probably will be mostly manual work to identify correct licenses for gems that do not have license in the gemspec metadata. Also convert Callaway convention to the correct SPDX identifier. This cannot be reliably automated for example the BSD license can be converted to 4 different SPDX license identifiers. However, tooling can give an idea of what the licensing probably is. At that point I will be mainly looking on Fedora spec's license, whether it has valid SPDX or not. 2b) Validate MIT licenses MIT license has multiple variants. While it mostly seems that the gem MIT licenses are under what SPDX also considers MIT, I'd like to validate that assumption. == How to help == Make sure your rubygem package has a license in Fedora's specfile that is a valid SPDX license identifier. You can also take a look at the gems where license does not match between gem2rpm and fedora (RubyGems should use SPDX, even though it is older version, that's a good pointer for us) and fix them. On closing note, be wary of licenses like "LGPL-2.1+", it is valid, however deprecated by current SPDX version and the `license-validate` tool won't accept it. See https://spdx.org/licenses/ for complete license list including the list of deprecated identifiers. Thanks, Jarek ___ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:
Re: Packaging Ruby gem documentation
On 11/1/22 20:53, Vít Ondruch wrote: Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a): We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours. Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method. Thanks for this. Is it ok to build upon this incrementally? License information is needed for all included components, even fonts. So an extra license tag should be added to all Ruby packages that produce rdoc documentation if it will be troublesome to unbundle fonts relatively soon. Impact on css is primarily where the fonts are, but css and javascript files are much smaller than the font files. Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO. BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic. Vít [1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36b0a76e632/f/macros.rubygems#_31 ___ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: gem install fails on rawhide
Dne 12. 11. 22 v 21:04 Jun Aruga (he / him) napsal(a): Hello, I found the gem install fails on the Ruby on the rawhide. I reported it to the package-notes RPM. https://bugzilla.redhat.com/show_bug.cgi?id=2142119 https://bugzilla.redhat.com/show_bug.cgi?id=2142119#c1 Maybe a temporary workaround we can do soon is to set the macro below in the ruby.spec. ``` %undefine _package_note_file ``` IOW revert this revert: https://src.fedoraproject.org/rpms/ruby/c/588a4ae9f02928d7bedbcf46a739d36b0a76e632?branch=rawhide Not sure if I really tested the installation of the native gems prior enabling the package notes. Probably not, because I thought that it will be similar to other `-specs=` directives, but this is much worse. Oh my, sorry And this is now also in F37: https://src.fedoraproject.org/rpms/ruby/c/588a4ae9f02928d7bedbcf46a739d36b0a76e632?branch=f37 Although this was not build yet. I will sort it out. Vít But essentially we need to do something in the `ruby.spec` not to expose the redhat-rpm-config RPM or Red Hat specific RPMs such as package-notes RPM. Possible ways are below. A. Modify rbconfig.rb by such as `sed` command. B. Add another logic to create the `reconfig.rb` by `make rbconfig.rb` with extension flags, and ship it in the Ruby RPM. C. Update the configure script with proper arguments or command options. https://src.fedoraproject.org/rpms/ruby/pull-request/117 Feel free to take this task. OpenPGP_signature Description: OpenPGP digital signature ___ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Packaging Ruby gem documentation
Dne 22. 11. 22 v 9:21 Benson Muite napsal(a): On 11/1/22 20:53, Vít Ondruch wrote: Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a): We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours. Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method. Thanks for this. Is it ok to build upon this incrementally? This would be just distraction. I'll rather finally spend time trying to find the ultimate solution then fixing licenses where this is know issue since forever, but was never really pressing. And if you really want some quick and dirty solution, then it would be better to just drop the fonts instead of adding license tags. This would result in slightly broken documentation, which would frankly nobody noticed. Less dirty solution would be to drop the fonts and require them into the -doc packages. There is even possible to not generate the documentation at all or drop it and not ship it. As you can see, there is many opportunities for bad workarounds and I don't really want to dig this rabbit hole. Vít License information is needed for all included components, even fonts. So an extra license tag should be added to all Ruby packages that produce rdoc documentation if it will be troublesome to unbundle fonts relatively soon. Impact on css is primarily where the fonts are, but css and javascript files are much smaller than the font files. Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO. BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic. Vít [1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36b0a76e632/f/macros.rubygems#_31 ___ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue OpenPGP_signature Description: OpenPGP digital signature ___ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
