Re: [Rpm-maint] %setup for RubyGems

2016-11-01 Thread Vít Ondruch 


Dne 1.11.2016 v 10:26 Pascal Terjan napsal(a):
> On 1 November 2016 at 09:01, Vít Ondruch  wrote:
>>
>> Dne 1.11.2016 v 09:08 Pascal Terjan napsal(a):
>>> On 1 November 2016 at 07:53, Vít Ondruch  wrote:
 Dne 1.11.2016 v 00:04 Pascal Terjan napsal(a):
> One big problem with having the gemspec in the parent directory is
> that we can't easily apply patches to it.
 You are right. Never thought about it. But I would not call it "big"
 problem ;)
>>> Well not being able to use %autopatch / %autosetup / ... is annoying
>>>
> This is a problem for about a dozen packages with invalid gemspec
> (duplicate dependencies, empty author, ...).
 Do you have some examples please?
>>> 2 random examples:
>>>
>>> + /usr/bin/gem build ../ruby-openid-apps-discovery-1.2.0.gemspec
>>> WARNING:  See http://guides.rubygems.org/specification-reference/ for help
>>> ERROR:  While executing gem ... (Gem::InvalidSpecificationException)
>>> authors may not be empty
>>>
>>> + /usr/bin/gem build ../journey-1.0.4.gemspec
>>> WARNING:  licenses is empty, but is recommended.  Use a license
>>> abbreviation from:
>>> http://opensource.org/licenses/alphabetical
>>> WARNING:  open-ended dependency on racc (>= 1.4.6, development) is not
>>> recommended
>>>   if racc is semantically versioned, use:
>>> add_development_dependency 'racc', '~> 1.4', '>= 1.4.6'
>>> WARNING:  open-ended dependency on json (>= 0, development) is not 
>>> recommended
>>>   if json is semantically versioned, use:
>>> add_development_dependency 'json', '~> 0'
>>> WARNING:  See http://guides.rubygems.org/specification-reference/ for help
>>> ERROR:  While executing gem ... (Gem::InvalidSpecificationException)
>>> duplicate dependency on rdoc (~> 3.10, development), (~> 3.11) use:
>>> add_runtime_dependency 'rdoc', '~> 3.10', '~> 3.11'
>>> error: Bad exit status from
>>> /tmp/ruby/ruby-journey/BUILDROOT/rpm-tmp.umx65z (%build)
>>>
>>> We can probably kill some of those packages as they seem dead upstream
>>>
 Do I understand correctly that you actually take some gem, do "gem spec
 --ruby" and you can't rebuild such gem with "gem build"?

>>> Yes
>>>
>>> There are also a few other cases when we have patches, mostly to allow
>>> newer versions of some dependencies
>> So let me make a few remarks.
>>
>> * The patches won't be applicable not just because the .gemspec is
>> generated in parent directory, but because they are versioned now. Hence
>> the patch needs changes with version updates.
>>
>> * But anyway, I don't think patching the generated .gemspec is good
>> idea. You used to patch the .gemspec, because you used mixture of
>> upstream .gemspec with generated .gemspec. I think it is reasonably OK,
>> to patch the upstream .gemspec (but then you still have to fight with
>> things such as "git ls-files"), but I don't think it is good idea to
>> patch the generated .gemspec, since even minor changes in RubyGems might
>> change the formatting. For example if I am not mistaken, RubyGems in
>> Ruby 2.4 apply #freeze method on every string, which will render your
>> current patches unusable.
> True that's a problem
>
>> * The two packages you pointed out are really obsolete. Journey was
>> subsumed into Rails long time ago, while the ruby-openid-apps-discovery
>> is dead upstream. I don't think you will encounter issues you described
>> with any up-2-date gem.
> Yes, until they change something in gem which again breaks more recent ones
>
> In the previous example, ruby-openid-apps-discovery had last commit in
> July 2012, rubygems 2.0 which broke it was released in February 2013.
> At that time we had to patch a larger number of packages.
>
> I can imagine a number of recent gems breaking again in the future
> that we would need to patch.

I agree, but this should be RubyGems upstream discussion IMO, because
they are breaking backward compatibility not just to us.

>
>> * Yes, changing dependencies is valid use case. I typically preferred to
>> use sed for this purpose, since it is more flexible then applying
>> patches. But recently, I introduced macros for this purpose [1, 2] into
>> Fedora. I believe they are superior to any other approach.
> Yes I had considered that but never took the time to write them, I may
> steal yours :)

Feel free to steal it

BTW speaking of patching gem, we once developed gem-patch RubyGems plugin:

https://rubygems.org/gems/gem-patch

May be it might come handy one day


Vít
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] %setup for RubyGems

2016-11-01 Thread Vít Ondruch 


Dne 1.11.2016 v 09:08 Pascal Terjan napsal(a):
> On 1 November 2016 at 07:53, Vít Ondruch  wrote:
>>
>> Dne 1.11.2016 v 00:04 Pascal Terjan napsal(a):
>>> One big problem with having the gemspec in the parent directory is
>>> that we can't easily apply patches to it.
>> You are right. Never thought about it. But I would not call it "big"
>> problem ;)
> Well not being able to use %autopatch / %autosetup / ... is annoying
>
>>> This is a problem for about a dozen packages with invalid gemspec
>>> (duplicate dependencies, empty author, ...).
>> Do you have some examples please?
> 2 random examples:
>
> + /usr/bin/gem build ../ruby-openid-apps-discovery-1.2.0.gemspec
> WARNING:  See http://guides.rubygems.org/specification-reference/ for help
> ERROR:  While executing gem ... (Gem::InvalidSpecificationException)
> authors may not be empty
>
> + /usr/bin/gem build ../journey-1.0.4.gemspec
> WARNING:  licenses is empty, but is recommended.  Use a license
> abbreviation from:
> http://opensource.org/licenses/alphabetical
> WARNING:  open-ended dependency on racc (>= 1.4.6, development) is not
> recommended
>   if racc is semantically versioned, use:
> add_development_dependency 'racc', '~> 1.4', '>= 1.4.6'
> WARNING:  open-ended dependency on json (>= 0, development) is not recommended
>   if json is semantically versioned, use:
> add_development_dependency 'json', '~> 0'
> WARNING:  See http://guides.rubygems.org/specification-reference/ for help
> ERROR:  While executing gem ... (Gem::InvalidSpecificationException)
> duplicate dependency on rdoc (~> 3.10, development), (~> 3.11) use:
> add_runtime_dependency 'rdoc', '~> 3.10', '~> 3.11'
> error: Bad exit status from
> /tmp/ruby/ruby-journey/BUILDROOT/rpm-tmp.umx65z (%build)
>
> We can probably kill some of those packages as they seem dead upstream
>
>> Do I understand correctly that you actually take some gem, do "gem spec
>> --ruby" and you can't rebuild such gem with "gem build"?
>>
> Yes
>
> There are also a few other cases when we have patches, mostly to allow
> newer versions of some dependencies

So let me make a few remarks.

* The patches won't be applicable not just because the .gemspec is
generated in parent directory, but because they are versioned now. Hence
the patch needs changes with version updates.

* But anyway, I don't think patching the generated .gemspec is good
idea. You used to patch the .gemspec, because you used mixture of
upstream .gemspec with generated .gemspec. I think it is reasonably OK,
to patch the upstream .gemspec (but then you still have to fight with
things such as "git ls-files"), but I don't think it is good idea to
patch the generated .gemspec, since even minor changes in RubyGems might
change the formatting. For example if I am not mistaken, RubyGems in
Ruby 2.4 apply #freeze method on every string, which will render your
current patches unusable.

* The two packages you pointed out are really obsolete. Journey was
subsumed into Rails long time ago, while the ruby-openid-apps-discovery
is dead upstream. I don't think you will encounter issues you described
with any up-2-date gem.

* Yes, changing dependencies is valid use case. I typically preferred to
use sed for this purpose, since it is more flexible then applying
patches. But recently, I introduced macros for this purpose [1, 2] into
Fedora. I believe they are superior to any other approach.


Vít



[1]
http://pkgs.fedoraproject.org/cgit/rpms/ruby.git/commit/?id=4fb1d9c02d92ae2f7ecaf1b644695ed94fc12978
[2]
https://lists.fedoraproject.org/archives/list/ruby-...@lists.fedoraproject.org/thread/K5DXS4N3BDSPRIHZOL4WGL2IIBOYXMXF/

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] %setup for RubyGems

2016-10-19 Thread Vít Ondruch 


Dne 17.10.2016 v 16:34 Pascal Terjan napsal(a):
> On 17 October 2016 at 15:12, Vít Ondruch  wrote:
>>
>> Dne 17.10.2016 v 15:37 Pascal Terjan napsal(a):
>>> On 17 October 2016 at 14:31, Vít Ondruch  wrote:
 Dne 17.10.2016 v 15:07 Thierry Vignaud napsal(a):
> On 17 October 2016 at 12:06, Vít Ondruch  wrote:
>> Dne 17.10.2016 v 11:30 Thierry Vignaud napsal(a):
>>> On 17 October 2016 at 10:10, Panu Matilainen  
>>> wrote:
> What is the chance to get [1, 2] into the release? I mildly remember,
> that once I was offered to get this patch into Fedora, but that never
> materialized and now it is almost a year. I don't think this is
> controversial change which should make anything break.
>
> Thx for considering.
>
>
> Vít
>
>
> [1] https://github.com/rpm-software-management/rpm/pull/27
> [2]
>
> https://github.com/rpm-software-management/rpm/commit/89d1dd0a7c63c7497d334e9f240ce7e36ca89434
 Hmm, that has actually been in Mageia for over a year so it's certainly
 gotten its share of soak-time (so at least it's not breaking anything 
 else)
 and people are probably depending on it in Mageia so it'd be a 
 reasonable
 candidate.
>>> Actually, it's been here at least in Mageia from much more earlier:
>>> http://svnweb.mageia.org/packages/cauldron/rpm/current/SOURCES/rpm-4.6.1-setup-rubygems.patch?view=markup=343
>>>
>>> I think the original patch went in in October 2010, previously we were
>>> using a separate %gem_unpack macro
>>>
>>> But it's not the same implementation as the one that has been merged
>>> in master. Mageia one is:
>>> http://svnweb.mageia.org/packages/cauldron/rpm/current/SOURCES/rpm-4.12.90-setup-rubygems.patch?revision=860276=markup
>>>
>>> But if it works the same, Mageia will be happy to drop one more patch 
>>> :-)
>> They are a bit different indeed. These are two main differences I can 
>> see:
>>
>> 1) The upstream patch is using 'gem' command to unpack the sources
>> instead of using 'tar'. The advantage of this approach is that it should
>> be always able to unpack every gem. Please note that historically, the
>> .gem format was internally different and the format might change again
>> in the future. The disadvantage is the dependency on external tool, but
>> this is just soft dependency, since RPM can handle the missing 'gem'
>> command gracefully.
>>
>> 2) There is provided the %{gem_name}.gemspec file alongside the unpacked
>> code, which is in Fedora used to repackage patched gem, prior 
>> installation.
> As it turns out, there's at least one other difference which breaks
> build for us:
> eg for ruby-gemcutter
> (http://svnweb.mageia.org/packages/cauldron/ruby-gemcutter/current/SPECS/ruby-gemcutter.spec?revision=947391=markup)
> our implementation creates eg:
> BUILD/ruby-gemcutter-0.7.1
> whereas the patch merged upstream creates eg:
> BUILD/gemcutter-0.7.1
>
> This breaks the couple packages I tried.
> This would need to patch the macros we ships with ruby-RubyGems
> (see attached rubygems.macros.diff)
 So in reality you did not get the sources calling the %setup macro, you
 actually unpacked just the first level archive and unpacking of the
 second level archive is handled by %gem_setup, right? The 'gem unpack'
 in upstream patch does this in single step.

 And I'd say that you should be able to simplify the %gem_setup macro
 even more, since the lines:

 ```
  if [ ! -f %{gem_name}.gemspec ]; then \
  %{_bindir}/gem specification -l --ruby %{SOURCE0} > 
 %{gem_name}.gemspec \
  fi \
 ```

 are covered by RPM now, as can be seen in your log:
>>> Yes that's good
>>>
 ```
 ++ /usr/bin/gem spec 
 /home/tv/mga/pkgs/N/ruby-gemcutter/SOURCES/gemcutter-0.7.1.gem --ruby
 ```

 vs

 ```
  + /usr/bin/gem specification -l --ruby 
 /home/tv/mga/pkgs/N/ruby-gemcutter/SOURCES/gemcutter-0.7.1.gem
 ```


 Actually, the log itself would be more useful, since I suspect that in
 the %build section, you rebuild the gem, which is using the .gemspec
 file somewhere, but that was stripped out from the diff.
>>> Yes and actually we have the code twice for the gemspec...
>>>
>>> -14: gem_build
>>> if [ ! -f %{gem_name}.gemspec ]; then
>>>%{_bindir}/gem specification -l --ruby %{SOURCE0} > 
>>> %{gem_name}.gemspec
>>> fi
>>> %{_bindir}/gem build %{gem_name}.gemspec
>> Yes, I can see that in [1] now. You should be good to simplify the
>> %gem_build to:
>>
>> ```
>> %gem_build \
>> %{_bindir}/gem build ../%{gem_name}-%{version}.gemspec

Re: [Rpm-maint] %setup for RubyGems

2016-10-17 Thread Vít Ondruch 


Dne 17.10.2016 v 15:37 Pascal Terjan napsal(a):
> On 17 October 2016 at 14:31, Vít Ondruch  wrote:
>>
>> Dne 17.10.2016 v 15:07 Thierry Vignaud napsal(a):
>>> On 17 October 2016 at 12:06, Vít Ondruch  wrote:
 Dne 17.10.2016 v 11:30 Thierry Vignaud napsal(a):
> On 17 October 2016 at 10:10, Panu Matilainen  
> wrote:
>>> What is the chance to get [1, 2] into the release? I mildly remember,
>>> that once I was offered to get this patch into Fedora, but that never
>>> materialized and now it is almost a year. I don't think this is
>>> controversial change which should make anything break.
>>>
>>> Thx for considering.
>>>
>>>
>>> Vít
>>>
>>>
>>> [1] https://github.com/rpm-software-management/rpm/pull/27
>>> [2]
>>>
>>> https://github.com/rpm-software-management/rpm/commit/89d1dd0a7c63c7497d334e9f240ce7e36ca89434
>> Hmm, that has actually been in Mageia for over a year so it's certainly
>> gotten its share of soak-time (so at least it's not breaking anything 
>> else)
>> and people are probably depending on it in Mageia so it'd be a reasonable
>> candidate.
> Actually, it's been here at least in Mageia from much more earlier:
> http://svnweb.mageia.org/packages/cauldron/rpm/current/SOURCES/rpm-4.6.1-setup-rubygems.patch?view=markup=343
>
> I think the original patch went in in October 2010, previously we were
> using a separate %gem_unpack macro
>
> But it's not the same implementation as the one that has been merged
> in master. Mageia one is:
> http://svnweb.mageia.org/packages/cauldron/rpm/current/SOURCES/rpm-4.12.90-setup-rubygems.patch?revision=860276=markup
>
> But if it works the same, Mageia will be happy to drop one more patch :-)
 They are a bit different indeed. These are two main differences I can see:

 1) The upstream patch is using 'gem' command to unpack the sources
 instead of using 'tar'. The advantage of this approach is that it should
 be always able to unpack every gem. Please note that historically, the
 .gem format was internally different and the format might change again
 in the future. The disadvantage is the dependency on external tool, but
 this is just soft dependency, since RPM can handle the missing 'gem'
 command gracefully.

 2) There is provided the %{gem_name}.gemspec file alongside the unpacked
 code, which is in Fedora used to repackage patched gem, prior installation.
>>> As it turns out, there's at least one other difference which breaks
>>> build for us:
>>> eg for ruby-gemcutter
>>> (http://svnweb.mageia.org/packages/cauldron/ruby-gemcutter/current/SPECS/ruby-gemcutter.spec?revision=947391=markup)
>>> our implementation creates eg:
>>> BUILD/ruby-gemcutter-0.7.1
>>> whereas the patch merged upstream creates eg:
>>> BUILD/gemcutter-0.7.1
>>>
>>> This breaks the couple packages I tried.
>>> This would need to patch the macros we ships with ruby-RubyGems
>>> (see attached rubygems.macros.diff)
>> So in reality you did not get the sources calling the %setup macro, you
>> actually unpacked just the first level archive and unpacking of the
>> second level archive is handled by %gem_setup, right? The 'gem unpack'
>> in upstream patch does this in single step.
>>
>> And I'd say that you should be able to simplify the %gem_setup macro
>> even more, since the lines:
>>
>> ```
>>  if [ ! -f %{gem_name}.gemspec ]; then \
>>  %{_bindir}/gem specification -l --ruby %{SOURCE0} > 
>> %{gem_name}.gemspec \
>>  fi \
>> ```
>>
>> are covered by RPM now, as can be seen in your log:
> Yes that's good
>
>> ```
>> ++ /usr/bin/gem spec 
>> /home/tv/mga/pkgs/N/ruby-gemcutter/SOURCES/gemcutter-0.7.1.gem --ruby
>> ```
>>
>> vs
>>
>> ```
>>  + /usr/bin/gem specification -l --ruby 
>> /home/tv/mga/pkgs/N/ruby-gemcutter/SOURCES/gemcutter-0.7.1.gem
>> ```
>>
>>
>> Actually, the log itself would be more useful, since I suspect that in
>> the %build section, you rebuild the gem, which is using the .gemspec
>> file somewhere, but that was stripped out from the diff.
> Yes and actually we have the code twice for the gemspec...
>
> -14: gem_build
> if [ ! -f %{gem_name}.gemspec ]; then
>%{_bindir}/gem specification -l --ruby %{SOURCE0} > %{gem_name}.gemspec
> fi
> %{_bindir}/gem build %{gem_name}.gemspec

Yes, I can see that in [1] now. You should be good to simplify the
%gem_build to:

```
%gem_build \
%{_bindir}/gem build ../%{gem_name}-%{version}.gemspec
```

These were the changes I proposed for Fedora [2].

Please note that the .gemspec file one directory level above the sources
has the advantage, that it does not collide with .gemspec, which might
be shipped with the gem itself and it is always completely different
from the generated .gemspec.


Vít



[1]

[Rpm-maint] %setup for RubyGems

2016-10-17 Thread Vít Ondruch 


Dne 17.10.2016 v 15:07 Thierry Vignaud napsal(a):
> On 17 October 2016 at 12:06, Vít Ondruch  wrote:
>> Dne 17.10.2016 v 11:30 Thierry Vignaud napsal(a):
>>> On 17 October 2016 at 10:10, Panu Matilainen  
>>> wrote:
> What is the chance to get [1, 2] into the release? I mildly remember,
> that once I was offered to get this patch into Fedora, but that never
> materialized and now it is almost a year. I don't think this is
> controversial change which should make anything break.
>
> Thx for considering.
>
>
> Vít
>
>
> [1] https://github.com/rpm-software-management/rpm/pull/27
> [2]
>
> https://github.com/rpm-software-management/rpm/commit/89d1dd0a7c63c7497d334e9f240ce7e36ca89434
 Hmm, that has actually been in Mageia for over a year so it's certainly
 gotten its share of soak-time (so at least it's not breaking anything else)
 and people are probably depending on it in Mageia so it'd be a reasonable
 candidate.
>>> Actually, it's been here at least in Mageia from much more earlier:
>>> http://svnweb.mageia.org/packages/cauldron/rpm/current/SOURCES/rpm-4.6.1-setup-rubygems.patch?view=markup=343
>>>
>>> I think the original patch went in in October 2010, previously we were
>>> using a separate %gem_unpack macro
>>>
>>> But it's not the same implementation as the one that has been merged
>>> in master. Mageia one is:
>>> http://svnweb.mageia.org/packages/cauldron/rpm/current/SOURCES/rpm-4.12.90-setup-rubygems.patch?revision=860276=markup
>>>
>>> But if it works the same, Mageia will be happy to drop one more patch :-)
>> They are a bit different indeed. These are two main differences I can see:
>>
>> 1) The upstream patch is using 'gem' command to unpack the sources
>> instead of using 'tar'. The advantage of this approach is that it should
>> be always able to unpack every gem. Please note that historically, the
>> .gem format was internally different and the format might change again
>> in the future. The disadvantage is the dependency on external tool, but
>> this is just soft dependency, since RPM can handle the missing 'gem'
>> command gracefully.
>>
>> 2) There is provided the %{gem_name}.gemspec file alongside the unpacked
>> code, which is in Fedora used to repackage patched gem, prior installation.
> As it turns out, there's at least one other difference which breaks
> build for us:
> eg for ruby-gemcutter
> (http://svnweb.mageia.org/packages/cauldron/ruby-gemcutter/current/SPECS/ruby-gemcutter.spec?revision=947391=markup)
> our implementation creates eg:
> BUILD/ruby-gemcutter-0.7.1
> whereas the patch merged upstream creates eg:
> BUILD/gemcutter-0.7.1
>
> This breaks the couple packages I tried.
> This would need to patch the macros we ships with ruby-RubyGems
> (see attached rubygems.macros.diff)

So in reality you did not get the sources calling the %setup macro, you
actually unpacked just the first level archive and unpacking of the
second level archive is handled by %gem_setup, right? The 'gem unpack'
in upstream patch does this in single step.

And I'd say that you should be able to simplify the %gem_setup macro
even more, since the lines:

```
 if [ ! -f %{gem_name}.gemspec ]; then \
 %{_bindir}/gem specification -l --ruby %{SOURCE0} > 
%{gem_name}.gemspec \
 fi \
```

are covered by RPM now, as can be seen in your log:

```
++ /usr/bin/gem spec 
/home/tv/mga/pkgs/N/ruby-gemcutter/SOURCES/gemcutter-0.7.1.gem --ruby
```

vs

```
 + /usr/bin/gem specification -l --ruby 
/home/tv/mga/pkgs/N/ruby-gemcutter/SOURCES/gemcutter-0.7.1.gem
```


Actually, the log itself would be more useful, since I suspect that in
the %build section, you rebuild the gem, which is using the .gemspec
file somewhere, but that was stripped out from the diff.



Vít

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint