Bug#1050340: [Pkg-puppet-devel] Bug#1050340: puppetserver: incompatibility with system hiera-eyaml

[Dominique Martinet]

Georg Faerber wrote on Tue, Sep 19, 2023 at 10:19:57AM +:
> So, symlinks to the rescue:
>
> [...]

This is horrible.
So I obviously did the same. Thanks a lot!

I'd be interested in a better solution that'll survive upgrades
though -- I agree wth Georg's analysis that this is "just" a problem of
loading paths, so either the puppetserver package could add hiera-eyaml
in its vendored-jruby-gems (which probably isn't too hard, but debian
frowns on vendored stuff in general so I don't see why it'd be better
for ruby deps..), or we'll need to look into why system gems cannot be
used with jruby...

-- 
Dominique Martinet | Asmadeus

Bug#1050340: [Pkg-puppet-devel] Bug#1050340: puppetserver: incompatibility with system hiera-eyaml

[Georg Faerber]

Hi all,

I ran into this issue as well and debugged a bit:

On 23-08-23 12:16:10, Antoine Beaupré wrote:
> Same here, I was just curious if you thought the manual gem update
> fixed this because it's a new upstream or just because "some gem
> thing".

I believe the issue is not about the new upstream, as according to the
changelog there were no relevant changes in between 3.3.0 and 3.4.0, but rather
it's about the directories, which are searched and code is loaded from.

Due to this, initially, I took the "easy route" and did this:

+++ a/etc/puppet/puppetserver/conf.d/puppetserver.conf
+++ b/etc/puppet/puppetserver/conf.d/puppetserver.conf
@@ -11,7 +11,7 @@ jruby-puppet: {
 # This setting defines the complete "GEM_PATH" for jruby.  If set, it 
should include
 # the gem-home directory as well as any other directories that gems can be 
loaded
 # from (including the vendored gems directory for gems that ship with 
puppetserver)
-gem-path: [${jruby-puppet.gem-home}, 
"/usr/lib/puppetserver/vendored-jruby-gems"]
+gem-path: [${jruby-puppet.gem-home}, 
"/usr/lib/puppetserver/vendored-jruby-gems", 
"/usr/share/rubygems-integration/all"]
 
 # PLEASE NOTE: Use caution when modifying the below settings. Modifying
 # these settings will change the value of the corresponding Puppet settings

However, this leads to the following:

Exception in thread "main" java.lang.IllegalStateException: Unable to borrow 
JRubyInstance from pool
[...]
Caused by: org.jruby.embed.EvalFailedException: (LoadError) Java extensions are 
required for JRuby.
no such file to load -- concurrent/concurrent_ruby.jar

concurrent_ruby.jar is part of the puppetserver package and installed to
/usr/lib/puppetserver/vendored-jruby-gems/gems/concurrent-ruby-1.1.5/lib/concurrent/concurrent_ruby.jar.

The ordering of the directory paths in puppetserver.conf doesn't make a
difference, the error stays the same.

So, symlinks to the rescue:

ln -s /usr/share/rubygems-integration/all/gems/hiera-eyaml-3.3.0/ 
/usr/lib/puppetserver/vendored-jruby-gems/gems/hiera-eyaml-3.3.0
ln -s /usr/share/rubygems-integration/all/gems/optimist-3.0.0/ 
/usr/lib/puppetserver/vendored-jruby-gems/gems/optimist-3.0.0
ln -s /usr/lib/ruby/vendor_ruby/highline 
/usr/lib/puppetserver/vendored-jruby-gems/gems/highline
ln -s 
/usr/share/rubygems-integration/all/specifications/hiera-eyaml-3.3.0.gemspec 
/usr/lib/puppetserver/vendored-jruby-gems/specifications/hiera-eyaml-3.3.0.gemspec
ln -s /usr/share/rubygems-integration/all/specifications/optimist-3.0.0.gemspec 
/usr/lib/puppetserver/vendored-jruby-gems/specifications/optimist-3.0.0.gemspec
ln -s /usr/share/rubygems-integration/all/specifications/highline-2.0.3.gemspec 
/usr/lib/puppetserver/vendored-jruby-gems/specifications/highline-2.0.3.gemspec

This made the error go away.

Maybe that's helpful,
all the best,
Georg

Bug#1050340: [Pkg-puppet-devel] Bug#1050340: puppetserver: incompatibility with system hiera-eyaml

[Cyril Brulebois]

Antoine Beaupré  (2023-08-23):
> Oh i meant the upstream version, the package in Debian is obviously out
> of date here. :)

Sure, I meant I would have tried (package relationship permitting) a
newer package if there was one available.

> Same here, I was just curious if you thought the manual gem update fixed
> this because it's a new upstream or just because "some gem thing".

“Yes.” ;p


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1050340: [Pkg-puppet-devel] Bug#1050340: puppetserver: incompatibility with system hiera-eyaml

[Antoine Beaupré]

On 2023-08-23 17:55:19, Cyril Brulebois wrote:
> Antoine Beaupré  (2023-08-23):
>> Do you think this is a version problem, in other words would it be
>> sufficient to upgrade the hiera-eyaml package to 3.4.0?
>
> That thought crossed my mind, but seeing the same version everywhere in
> rmadison's output, I considered I would stop there. Gut feelings:
>  - maybe there's a version mismatch;
>  - maybe there's some search path problem (e.g. only/mostly looking into
>some vendorized libraries, but then it really should have vendorized
>that particular one).

Oh i meant the upstream version, the package in Debian is obviously out
of date here. :)

> I'm happy to test a newer version once you have a tentative package. I'm
> not sure I can reasonably commit to trying and preparing the update
> myself.

Same here, I was just curious if you thought the manual gem update fixed
this because it's a new upstream or just because "some gem thing".

a.

-- 
We must learn to live together as brothers or perish together as fools.
- Martin Luther King, Jr.

Bug#1050340: [Pkg-puppet-devel] Bug#1050340: puppetserver: incompatibility with system hiera-eyaml

[Cyril Brulebois]

Antoine Beaupré  (2023-08-23):
> Do you think this is a version problem, in other words would it be
> sufficient to upgrade the hiera-eyaml package to 3.4.0?

That thought crossed my mind, but seeing the same version everywhere in
rmadison's output, I considered I would stop there. Gut feelings:
 - maybe there's a version mismatch;
 - maybe there's some search path problem (e.g. only/mostly looking into
   some vendorized libraries, but then it really should have vendorized
   that particular one).

I'm happy to test a newer version once you have a tentative package. I'm
not sure I can reasonably commit to trying and preparing the update
myself.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1050340: [Pkg-puppet-devel] Bug#1050340: puppetserver: incompatibility with system hiera-eyaml

[Antoine Beaupré]

On 2023-08-23 13:55:21, Cyril Brulebois wrote:
> Package: puppetserver
> Version: 7.9.5-2
> Severity: important
>
> Hi,
>
> I totally lost hiera eyaml support while upgrading from bullseye to
> bookworm. Neither the old hiera configuration file or the new one
> worked. Given the upstream upgrade path, I totally understand that
> there's little puppet packagers can do to ease the pain…
>
>   https://www.puppet.com/docs/puppet/7/hiera_migrate#hiera_migrate
>
> Just in case it helps others, here's what I ended up using, which
> lets me use nodes/*.eyaml files:
>

[...]

> But now I'm facing a bigger issue, which is that any use of hiera
> triggers this error, and dozens of log lines:
>
> Lookup using eyaml lookup_key function is only supported when the 
> hiera_eyaml library is present
>
> I'm attaching a log excerpt with a trace.
>
>
> Since I didn't want to keep a broken puppet {master,server} for too
> long, I bit the bullet and tried installing the gem, which worked around
> the immediate problem:
>
> puppetserver gem install hiera-eyaml
>
> But it'd be great if that problem could be debugged and a proper
> solution found, only using Debian packages… This is a rather small
> setup, changes don't happen very often, there's nothing really
> mission-critical, so I can happily assist debugging/running tests
> if instructed.

That's a great bug report, thanks!

Do you think this is a version problem, in other words would it be
sufficient to upgrade the hiera-eyaml package to 3.4.0?

-- 
feature, n: a documented bug | bug, n: an undocumented feature
- Mario S F Ferreira