Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Pirate Praveen]

On Mon, 10 Jul 2017 01:22:39 +0530 Pirate Praveen
 wrote:
> make[2]: *** No rule to make target
> '/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2/src/ruby/ext/grpc/libs/opt/libgrpc.a'.
>  Stop.

This is happening because Makefile was not copied to
/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2/
even though grpc.gemspec includes the Makefile.

after adding the following line to debian/rules,
export DH_RUBY_GEM_INSTALL_WHITELIST_APPEND=Makefile

build is succeeding



signature.asc
Description: OpenPGP digital signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Pirate Praveen]

On Sun, 9 Jul 2017 11:30:24 -0300 Antonio Terceiro 
wrote:
> Thinking about it, we kind of already support this: when you use
> --gem-install, this typo of thing should Just Work™.
> 
> > For now, I suggest that you
> > 
> > - drop your ruby-grpc
> > - send a patch to the existing grpc source to produce a ruby-grcp binary
> >   from it
> 

I have created this patch and submitted it (please review)
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867541#12 But it still
needs a fix for this bug to work (see below for problem with
--gem-install).

Note: I saw your comment about --gem-install only after submitting the
patch.

> I suggest that you use --gem-install when sending your patch to grpc, it
> should do the right thing.
> 

But it somehow fails to consider
https://anonscm.debian.org/cgit/pkg-ruby-extras/ruby-grpc.git/tree/debian/patches/no-embed-libs.patch
and fails (which works if I remove --gem-install option)

Building native extensions.  This could take a while...
current directory:
/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2/src/ruby/ext/grpc
/usr/bin/ruby2.3 -r ./siteconf20170710-20691-1hxb754.rb extconf.rb
Building internal gRPC into
/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2/src/ruby/ext/grpc/libs/opt
make[2]: Entering directory
'/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2'
make[2]: *** No rule to make target
'/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2/src/ruby/ext/grpc/libs/opt/libgrpc.a'.
 Stop.
make[2]: Leaving directory
'/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2'
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.
You may
need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/$(RUBY_BASE_NAME)2.3
ERROR:  Error installing /tmp/d20170710-20670-8tpnrd/grpc-1.3.2.gem:
ERROR: Failed to build gem native extension.

Building has failed. See above output for more information on the
failure.
extconf failed, exit code 1

Gem files will remain installed in
/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/gems/grpc-1.3.2
for inspection.
Results logged to
/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0/extensions/x86_64-linux/2.3.0/grpc-1.3.2/gem_make.out
/usr/lib/ruby/vendor_ruby/gem2deb.rb:56:in `run': /usr/bin/ruby2.3 -S
gem install --config-file /dev/null --verbose --local --verbose
--no-rdoc --no-ri --ignore-dependencies --install-dir
debian/ruby-grpc/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0
/tmp/d20170710-20670-8tpnrd/grpc-1.3.2.gem (Gem2Deb::CommandFailed)
from /usr/lib/ruby/vendor_ruby/gem2deb/gem_installer.rb:189:in `gem'
from /usr/lib/ruby/vendor_ruby/gem2deb/gem_installer.rb:103:in `block
in install_files_and_build_extensions'
from /usr/lib/ruby/vendor_ruby/gem2deb/gem_installer.rb:51:in `each'
from /usr/lib/ruby/vendor_ruby/gem2deb/gem_installer.rb:51:in
`install_files_and_build_extensions'
from /usr/lib/ruby/vendor_ruby/gem2deb/dh_ruby.rb:84:in `block in 
install'
from /usr/lib/ruby/vendor_ruby/gem2deb/dh_ruby.rb:82:in `each'
from /usr/lib/ruby/vendor_ruby/gem2deb/dh_ruby.rb:82:in `install'
from /usr/bin/dh_ruby:94:in `'
dh_auto_install: dh_ruby --install
/home/pravi/forge/debian/gitlab/grpc-1.3.2/debian/tmp returned exit code 1
debian/rules:30: recipe for target 'override_dh_auto_install' failed
make[1]: *** [override_dh_auto_install] Error 1
make[1]: Leaving directory '/home/pravi/forge/debian/gitlab/grpc-1.3.2'
debian/rules:40: recipe for target 'binary' failed



signature.asc
Description: OpenPGP digital signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Antonio Terceiro]

On Sat, Jul 08, 2017 at 11:15:04AM -0300, Antonio Terceiro wrote:
> Control: severity -1 wishlist
> Control: retitle -1 gem2deb: support installing Ruby code not in lib/
> 
> On Sat, Jul 08, 2017 at 01:48:19PM +0530, Pirate Praveen wrote:
> > On 07/08/2017 11:53 AM, Pirate Praveen wrote:
> > > On 07/08/2017 07:44 AM, Antonio Terceiro wrote:
> > >> if the pure ruby libraries were not being installed, you would get a
> > >> LoadError (i.e. a `require` statement failed), but that's not what is
> > >> happening here.
> > > For tests, the pure ruby libs are available in the source tree.
> > > 
> > > If I move the src/ruby/lib folder (I have locally installed ruby-grpc
> > > deb), then it fails with `require': cannot load such file -- grpc
> > > (LoadError)
> > 
> > I think gem2deb expects libraries only in lib.
> 
> From what I see from the source code, grpc is a library with multiple
> language bindings; packaging it as a Ruby package is wrong. there is
> even already a grpc source package in Debian. you should send a patch to
> that package to generate a ruby-grpc binary package from there, instead
> of creating a new ruby-specific package.
> 
> your ruby-grpc has a lot of embedded code copies under third_party/,
> which has been already solved in grpc source package.
> 
> also, we can't possibly make gem2deb "just work" in such packages
> because they deviate a lot from what a standard ruby package looks like.
> 
> > s.require_paths = %w( src/ruby/lib src/ruby/bin src/ruby/pb ) mentions
> > it in grpc.gemspec but I don't know if gem2deb can reply on that.
> 
> gem2deb could be made to work with it, but I don't expect to get to it
> any time soon.

Thinking about it, we kind of already support this: when you use
--gem-install, this typo of thing should Just Work™.

> For now, I suggest that you
> 
> - drop your ruby-grpc
> - send a patch to the existing grpc source to produce a ruby-grcp binary
>   from it

I suggest that you use --gem-install when sending your patch to grpc, it
should do the right thing.



signature.asc
Description: PGP signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Antonio Terceiro]

Control: severity -1 wishlist
Control: retitle -1 gem2deb: support installing Ruby code not in lib/

On Sat, Jul 08, 2017 at 01:48:19PM +0530, Pirate Praveen wrote:
> On 07/08/2017 11:53 AM, Pirate Praveen wrote:
> > On 07/08/2017 07:44 AM, Antonio Terceiro wrote:
> >> if the pure ruby libraries were not being installed, you would get a
> >> LoadError (i.e. a `require` statement failed), but that's not what is
> >> happening here.
> > For tests, the pure ruby libs are available in the source tree.
> > 
> > If I move the src/ruby/lib folder (I have locally installed ruby-grpc
> > deb), then it fails with `require': cannot load such file -- grpc
> > (LoadError)
> 
> I think gem2deb expects libraries only in lib.

From what I see from the source code, grpc is a library with multiple
language bindings; packaging it as a Ruby package is wrong. there is
even already a grpc source package in Debian. you should send a patch to
that package to generate a ruby-grpc binary package from there, instead
of creating a new ruby-specific package.

your ruby-grpc has a lot of embedded code copies under third_party/,
which has been already solved in grpc source package.

also, we can't possibly make gem2deb "just work" in such packages
because they deviate a lot from what a standard ruby package looks like.

> s.require_paths = %w( src/ruby/lib src/ruby/bin src/ruby/pb ) mentions
> it in grpc.gemspec but I don't know if gem2deb can reply on that.

gem2deb could be made to work with it, but I don't expect to get to it
any time soon. For now, I suggest that you

- drop your ruby-grpc
- send a patch to the existing grpc source to produce a ruby-grcp binary
  from it


signature.asc
Description: PGP signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Pirate Praveen]

On 07/08/2017 01:48 PM, Pirate Praveen wrote:
> On 07/08/2017 11:53 AM, Pirate Praveen wrote:
>> On 07/08/2017 07:44 AM, Antonio Terceiro wrote:
>>> if the pure ruby libraries were not being installed, you would get a
>>> LoadError (i.e. a `require` statement failed), but that's not what is
>>> happening here.
>> For tests, the pure ruby libs are available in the source tree.
>>
>> If I move the src/ruby/lib folder (I have locally installed ruby-grpc
>> deb), then it fails with `require': cannot load such file -- grpc
>> (LoadError)
> 
> I think gem2deb expects libraries only in lib.
> 
> s.require_paths = %w( src/ruby/lib src/ruby/bin src/ruby/pb ) mentions
> it in grpc.gemspec but I don't know if gem2deb can reply on that.
> 

I tried adding X-DhRuby-Root: src/ruby but then I got this failure

┌──┐
│ Install files
  │
└──┘

install -d /<>/debian/ruby-grpc/usr/bin
install -D -m755 /<>/src/ruby/bin/apis
/<>/debian/ruby-grpc/usr/bin/apis
install: omitting directory '/<>/src/ruby/bin/apis'
/usr/lib/ruby/vendor_ruby/gem2deb.rb:56:in `run': install -D -m755
/<>/src/ruby/bin/apis
/<>/debian/ruby-grpc/usr/bin/apis (Gem2Deb::CommandFailed)
from /usr/lib/ruby/vendor_ruby/gem2deb/installer.rb:172:in `block in
install_files'
from /usr/lib/ruby/vendor_ruby/gem2deb/installer.rb:169:in `each'
from /usr/lib/ruby/vendor_ruby/gem2deb/installer.rb:169:in 
`install_files'
from /usr/lib/ruby/vendor_ruby/gem2deb/installer.rb:30:in
`install_files_and_build_extensions'
from /usr/lib/ruby/vendor_ruby/gem2deb/dh_ruby.rb:84:in `block in 
install'
from /usr/lib/ruby/vendor_ruby/gem2deb/dh_ruby.rb:82:in `each'
from /usr/lib/ruby/vendor_ruby/gem2deb/dh_ruby.rb:82:in `install'
from /usr/bin/dh_ruby:94:in `'
dh_auto_install: dh_ruby --install /<>/debian/ruby-grpc
returned exit code 1
debian/rules:6: recipe for target 'binary-arch' failed
make: *** [binary-arch] Error 1
dpkg-buildpackage: error: fakeroot debian/rules binary-arch gave error
exit status 2

Build finished at 2017-07-08T09:04:37Z

Finished




signature.asc
Description: OpenPGP digital signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Pirate Praveen]

On 07/08/2017 11:53 AM, Pirate Praveen wrote:
> On 07/08/2017 07:44 AM, Antonio Terceiro wrote:
>> if the pure ruby libraries were not being installed, you would get a
>> LoadError (i.e. a `require` statement failed), but that's not what is
>> happening here.
> For tests, the pure ruby libs are available in the source tree.
> 
> If I move the src/ruby/lib folder (I have locally installed ruby-grpc
> deb), then it fails with `require': cannot load such file -- grpc
> (LoadError)

I think gem2deb expects libraries only in lib.

s.require_paths = %w( src/ruby/lib src/ruby/bin src/ruby/pb ) mentions
it in grpc.gemspec but I don't know if gem2deb can reply on that.



signature.asc
Description: OpenPGP digital signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Antonio Terceiro]

Control: tag -1 + moreinfo

On Fri, Jul 07, 2017 at 10:43:20PM +0530, Pirate Praveen wrote:
> package: gem2deb
> version: 0.34
> 
> gem2deb is not installing the pure ruby libraries of grpc (ruby-grpc is
> in alioth) and tests fail.

yes it is installing the pure ruby libraries

> ──┐
> │ Run tests for ruby2.3 from debian/ruby-tests.rake
>   │
> └──┘
> 
> RUBYLIB=/<>/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/ruby/vendor_ruby/2.3.0:.
> GEM_PATH=debian/ruby-grpc/usr/share/rubygems-integration/2.3.0:/var/lib/gems/2.3.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0:/usr/share/rubygems-integration/2.3.0:/usr/share/rubygems-integration/all
> ruby2.3 -S rake -f debian/ruby-tests.rake
> /usr/bin/ruby2.3 /usr/bin/rspec --pattern
> ./src/ruby/spec/\*\*/\*_spec.rb --format documentation
> /<>/src/ruby/lib/grpc/errors.rb:48:in `':
> uninitialized constant GRPC::Core (NameError)

if the pure ruby libraries were not being installed, you would get a
LoadError (i.e. a `require` statement failed), but that's not what is
happening here.

gem2deb does install the Ruby code even when there is a C extension, and
there are several automated tests for that. If it didn't, we wouldn't
be learning that this late in the game.

I don't think this is a bug in gem2deb, but you can convince me
otherwise.


signature.asc
Description: PGP signature

Bug#867599: gem2deb fails to install pure ruby part when a gem has both pure ruby and native libs

[Pirate Praveen]

package: gem2deb
version: 0.34

gem2deb is not installing the pure ruby libraries of grpc (ruby-grpc is
in alioth) and tests fail.

──┐
│ Run tests for ruby2.3 from debian/ruby-tests.rake
  │
└──┘

RUBYLIB=/<>/debian/ruby-grpc/usr/lib/x86_64-linux-gnu/ruby/vendor_ruby/2.3.0:.
GEM_PATH=debian/ruby-grpc/usr/share/rubygems-integration/2.3.0:/var/lib/gems/2.3.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.3.0:/usr/share/rubygems-integration/2.3.0:/usr/share/rubygems-integration/all
ruby2.3 -S rake -f debian/ruby-tests.rake
/usr/bin/ruby2.3 /usr/bin/rspec --pattern
./src/ruby/spec/\*\*/\*_spec.rb --format documentation
/<>/src/ruby/lib/grpc/errors.rb:48:in `':
uninitialized constant GRPC::Core (NameError)
from /<>/src/ruby/lib/grpc/errors.rb:45:in `'
from /<>/src/ruby/lib/grpc/errors.rb:33:in `'
from /<>/src/ruby/lib/grpc.rb:32:in `require_relative'
from /<>/src/ruby/lib/grpc.rb:32:in `'
from /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in
`require'
from /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in
`require'
from /<>/src/ruby/spec/call_credentials_spec.rb:30:in
`'
from /usr/lib/ruby/vendor_ruby/rspec/core/configuration.rb:1435:in 
`load'
from /usr/lib/ruby/vendor_ruby/rspec/core/configuration.rb:1435:in
`block in load_spec_files'
from /usr/lib/ruby/vendor_ruby/rspec/core/configuration.rb:1433:in 
`each'
from /usr/lib/ruby/vendor_ruby/rspec/core/configuration.rb:1433:in
`load_spec_files'
from /usr/lib/ruby/vendor_ruby/rspec/core/runner.rb:100:in `setup'
from /usr/lib/ruby/vendor_ruby/rspec/core/runner.rb:86:in `run'
from /usr/lib/ruby/vendor_ruby/rspec/core/runner.rb:71:in `run'
from /usr/lib/ruby/vendor_ruby/rspec/core/runner.rb:45:in `invoke'
from /usr/bin/rspec:4:in `'
/usr/bin/ruby2.3 /usr/bin/rspec --pattern
./src/ruby/spec/\*\*/\*_spec.rb --format documentation failed
ERROR: Test "ruby2.3" failed. Exiting.



signature.asc
Description: OpenPGP digital signature