Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Dennis Straffin 
Oops, I didn't see Ken's reply where he already figure this out (spotty cell 
service).

Where I work we were having this issue when bringing up new VMs. While  
provisioning, it would seem to just hang for 10 or 15 minutes when installing 
packages. No cpu usage, no disk io, no network io.  And every time you'd ssh in 
to the VM to run top, it'd start going again. Very frustrating.  Simply 
installing haveged first fixed the issue.

-Dennis

On August 8, 2017 5:47:11 PM EDT, Dennis Straffin  wrote:
>VMs lack hardware devices to fill up the pool of random numbers.
>Installing the haveged daemon will do expansion on the random numbers
>to keep the pool full.
>
>-Dennis
>
>On August 8, 2017 3:30:46 PM EDT, Ken D'Ambrosio  wrote:
>>On 2017-08-08 15:18, Joshua Judson Rosen wrote:
>>
>>>The  /dev/random  interface  is  considered  a  legacy  
>>> interface,  and
>>>/dev/urandom is preferred and sufficient in all  use  cases, 
>
>>> with  the
>>>exception  of  applications  which require randomness during 
>>> early boot
>>>time; for  these  applications,  getrandom(2)  must  be  used
>
>>
>>> instead,
>>>because it will block until the entropy pool is initialized.
>>> 
>>> So, there you go. "until the entropy pool is initialized" is
>>apparently
>>> about 3 minutes in your case ;)
>>
>>Yeah... getrandom() apparently pings /dev/urandom by default which, as
>
>>per the getrandom manpage, blocks until it has entropy.  Sounds like 
>>we've wound up at much the same place: I took some data off of 
>>/dev/random, stored it in a file, and am feeding that to /dev/urandom
>>at 
>>boot time (and re-seeding the file after five minutes' uptime).  Alas 
>>(because, you know, deadline), that doesn't seem to be working.  Which
>
>>is really, really annoying.  I'm *still* blocking for three-to-five on
>
>>getrandom().
>>
>>I guess it's time to cut my losses and start this in a different 
>>language.  I mean, most of the hard stuff was figuring out *how* to do
>
>>things, but I admit, my Perl and Python have grown rusty as I've
>>enjoyed 
>>my Ruby...
>>
>>-Ken
>>___
>>gnhlug-discuss mailing list
>>gnhlug-discuss@mail.gnhlug.org
>>http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Dennis Straffin 
VMs lack hardware devices to fill up the pool of random numbers. Installing the 
haveged daemon will do expansion on the random numbers to keep the pool full.

-Dennis

On August 8, 2017 3:30:46 PM EDT, Ken D'Ambrosio  wrote:
>On 2017-08-08 15:18, Joshua Judson Rosen wrote:
>
>>The  /dev/random  interface  is  considered  a  legacy  
>> interface,  and
>>/dev/urandom is preferred and sufficient in all  use  cases,  
>> with  the
>>exception  of  applications  which require randomness during 
>> early boot
>>time; for  these  applications,  getrandom(2)  must  be  used 
>
>> instead,
>>because it will block until the entropy pool is initialized.
>> 
>> So, there you go. "until the entropy pool is initialized" is
>apparently
>> about 3 minutes in your case ;)
>
>Yeah... getrandom() apparently pings /dev/urandom by default which, as 
>per the getrandom manpage, blocks until it has entropy.  Sounds like 
>we've wound up at much the same place: I took some data off of 
>/dev/random, stored it in a file, and am feeding that to /dev/urandom
>at 
>boot time (and re-seeding the file after five minutes' uptime).  Alas 
>(because, you know, deadline), that doesn't seem to be working.  Which 
>is really, really annoying.  I'm *still* blocking for three-to-five on 
>getrandom().
>
>I guess it's time to cut my losses and start this in a different 
>language.  I mean, most of the hard stuff was figuring out *how* to do 
>things, but I admit, my Perl and Python have grown rusty as I've
>enjoyed 
>my Ruby...
>
>-Ken
>___
>gnhlug-discuss mailing list
>gnhlug-discuss@mail.gnhlug.org
>http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Ken D'Ambrosio 
Well, I don't know what was wrong with catting random data to 
/dev/random and /dev/urandom, but it didn't to diddly.  "apt install 
haveged", howver, and I'm now booting in ~20 seconds instead of 3 - 5 
minutes.  (It adds entropy -- or, if you prefer, "entropy" -- by seeing 
how long certain things take to execute, and then doing it again, and 
again, and looking for deltas.)

#winning

Thanks, all...

-Ken


On 2017-08-08 15:18, Joshua Judson Rosen wrote:
> On 08/08/2017 02:52 PM, Ken D'Ambrosio wrote:
>> On 2017-08-08 14:43, Bill Freeman wrote:
>>> I don't know, but getrandom() may well be using /dev/urandom (or a
>>> related facility).  And that, in turn, might be waiting to "collect
>>> sufficient entropy".  So some network traffic, keystrokes, whatever,
>>> need to happen between boot time and the first random emission, or
>>> that first "random" number becomes predictable.  Since random numbers
>>> are often used cryptographically, predictability is a bad thing.
>> 
>> True, but there's debate about just *how* predictable, etc. Not a
>> subject for this particular thread, but I'd be perfectly happy with 
>> udev
>> almost-as-random.
>> 
>>> As to why ruby is designed to require a random number before being
>>> asked to do something dependent on such a random number is a question
>>> for the ruby developers.
>> 
>> Email already sent. :-)
>> 
>>> Re-linking /dev/urandom will probably break lots of things.  Maybe 
>>> run
>>> your script in a chroot jail that has a different /dev/urandom could
>>> work.
>> 
>> Alas, no -- I'm doing various admin chores, and a chroot won't be
>> helpful.
>> 
>>> Is your script too complex to rewrite in bash?  Not a general
>>> solution, but as a workaround it has its appeal.
>> 
>> *sigh* This is probably where I'm gonna wind up (or Perl, or Python).
>> Except I've now written a good handful of scripts that people are
>> waiting on, and it's gonna cause me physical pain to have to re-do 
>> them
>> at this point.
>> 
>> C'est la vie.  I guess that's the way the Ruby crumbles...
> 
> Instead of rewriting the whole thing, why not just seed the RNG 
> manually?
> 
> Slightly relevant-looking discussion BTW:
> 
> https://bugs.ruby-lang.org/issues/9569#note-56
> 
> ... mainly in that it points to the updated random(4) Linux man page,
> which says:
> 
>The  /dev/random  interface  is  considered  a  legacy  
> interface,  and
>/dev/urandom is preferred and sufficient in all  use  cases,  
> with  the
>exception  of  applications  which require randomness during 
> early boot
>time; for  these  applications,  getrandom(2)  must  be  used  
> instead,
>because it will block until the entropy pool is initialized.
> 
> So, there you go. "until the entropy pool is initialized" is apparently
> about 3 minutes in your case ;)
> 
> You should be able to explicitly seed Ruby's internal RNG,
> or explicitly seed the system RNG by writing bytes into
> /dev/random or /dev/urandom.
> 
> If you want `instant good entropy' at boot, you can even store
> some random data into a file at shutdown and then seed from that file
> at boot (be sure to invalidate that cache before seeding from it 
> though,
> to ensure that you don't use the same seed twice!). IIRC there are
> some preexisting packages for this, and some distributions even do it
> by default.
> 
> If you write a systemd service, it looks like you can depend on
> systemd-random-seed.service.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Tom Buskey 
On Tue, Aug 8, 2017 at 3:18 PM, Joshua Judson Rosen 
wrote:

> On 08/08/2017 02:52 PM, Ken D'Ambrosio wrote:
> > On 2017-08-08 14:43, Bill Freeman wrote:
> >> As to why ruby is designed to require a random number before being
> >> asked to do something dependent on such a random number is a question
> >> for the ruby developers.
>

I assume you meant not dependent.


> >
> > Email already sent. :-)
> >
> > C'est la vie.  I guess that's the way the Ruby crumbles...
>

That'd be a pretty bad design decision for some environments.


> Instead of rewriting the whole thing, why not just seed the RNG manually?
>
> Slightly relevant-looking discussion BTW:
>
> https://bugs.ruby-lang.org/issues/9569#note-56
>
>

> So, there you go. "until the entropy pool is initialized" is apparently
> about 3 minutes in your case ;)
>
> You should be able to explicitly seed Ruby's internal RNG,
> or explicitly seed the system RNG by writing bytes into
> /dev/random or /dev/urandom.
>
> If you want `instant good entropy' at boot, you can even store
> some random data into a file at shutdown and then seed from that file
> at boot (be sure to invalidate that cache before seeding from it though,
> to ensure that you don't use the same seed twice!). IIRC there are
> some preexisting packages for this, and some distributions even do it by
> default.
>
> If you write a systemd service, it looks like you can depend on
> systemd-random-seed.service.
>

The preseed might be a good 1st try at a fix that won't ruin the system RNG
and potentially compromising security.

This reminds me of the old days when Slackware was created because SLS
wouldn't fix permission issues correctly in the installer.
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Ken D'Ambrosio 
On 2017-08-08 15:18, Joshua Judson Rosen wrote:

>The  /dev/random  interface  is  considered  a  legacy  
> interface,  and
>/dev/urandom is preferred and sufficient in all  use  cases,  
> with  the
>exception  of  applications  which require randomness during 
> early boot
>time; for  these  applications,  getrandom(2)  must  be  used  
> instead,
>because it will block until the entropy pool is initialized.
> 
> So, there you go. "until the entropy pool is initialized" is apparently
> about 3 minutes in your case ;)

Yeah... getrandom() apparently pings /dev/urandom by default which, as 
per the getrandom manpage, blocks until it has entropy.  Sounds like 
we've wound up at much the same place: I took some data off of 
/dev/random, stored it in a file, and am feeding that to /dev/urandom at 
boot time (and re-seeding the file after five minutes' uptime).  Alas 
(because, you know, deadline), that doesn't seem to be working.  Which 
is really, really annoying.  I'm *still* blocking for three-to-five on 
getrandom().

I guess it's time to cut my losses and start this in a different 
language.  I mean, most of the hard stuff was figuring out *how* to do 
things, but I admit, my Perl and Python have grown rusty as I've enjoyed 
my Ruby...

-Ken
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Joshua Judson Rosen 
On 08/08/2017 02:52 PM, Ken D'Ambrosio wrote:
> On 2017-08-08 14:43, Bill Freeman wrote:
>> I don't know, but getrandom() may well be using /dev/urandom (or a
>> related facility).  And that, in turn, might be waiting to "collect
>> sufficient entropy".  So some network traffic, keystrokes, whatever,
>> need to happen between boot time and the first random emission, or
>> that first "random" number becomes predictable.  Since random numbers
>> are often used cryptographically, predictability is a bad thing.
> 
> True, but there's debate about just *how* predictable, etc. Not a 
> subject for this particular thread, but I'd be perfectly happy with udev 
> almost-as-random.
> 
>> As to why ruby is designed to require a random number before being
>> asked to do something dependent on such a random number is a question
>> for the ruby developers.
> 
> Email already sent. :-)
> 
>> Re-linking /dev/urandom will probably break lots of things.  Maybe run
>> your script in a chroot jail that has a different /dev/urandom could
>> work.
> 
> Alas, no -- I'm doing various admin chores, and a chroot won't be 
> helpful.
> 
>> Is your script too complex to rewrite in bash?  Not a general
>> solution, but as a workaround it has its appeal.
> 
> *sigh* This is probably where I'm gonna wind up (or Perl, or Python).  
> Except I've now written a good handful of scripts that people are 
> waiting on, and it's gonna cause me physical pain to have to re-do them 
> at this point.
> 
> C'est la vie.  I guess that's the way the Ruby crumbles...

Instead of rewriting the whole thing, why not just seed the RNG manually?

Slightly relevant-looking discussion BTW:

https://bugs.ruby-lang.org/issues/9569#note-56

... mainly in that it points to the updated random(4) Linux man page, which 
says:

   The  /dev/random  interface  is  considered  a  legacy  interface,  and
   /dev/urandom is preferred and sufficient in all  use  cases,  with  the
   exception  of  applications  which require randomness during early boot
   time; for  these  applications,  getrandom(2)  must  be  used  instead,
   because it will block until the entropy pool is initialized.

So, there you go. "until the entropy pool is initialized" is apparently
about 3 minutes in your case ;)

You should be able to explicitly seed Ruby's internal RNG,
or explicitly seed the system RNG by writing bytes into
/dev/random or /dev/urandom.

If you want `instant good entropy' at boot, you can even store
some random data into a file at shutdown and then seed from that file
at boot (be sure to invalidate that cache before seeding from it though,
to ensure that you don't use the same seed twice!). IIRC there are
some preexisting packages for this, and some distributions even do it by 
default.

If you write a systemd service, it looks like you can depend on
systemd-random-seed.service.

-- 
Connect with me on the GNU social network: 

Not on the network? Ask me for an invitation to the nhcrossing.com social hub!
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Ken D'Ambrosio 
On 2017-08-08 14:43, Bill Freeman wrote:
> I don't know, but getrandom() may well be using /dev/urandom (or a
> related facility).  And that, in turn, might be waiting to "collect
> sufficient entropy".  So some network traffic, keystrokes, whatever,
> need to happen between boot time and the first random emission, or
> that first "random" number becomes predictable.  Since random numbers
> are often used cryptographically, predictability is a bad thing.

True, but there's debate about just *how* predictable, etc. Not a 
subject for this particular thread, but I'd be perfectly happy with udev 
almost-as-random.

> As to why ruby is designed to require a random number before being
> asked to do something dependent on such a random number is a question
> for the ruby developers.

Email already sent. :-)

> Re-linking /dev/urandom will probably break lots of things.  Maybe run
> your script in a chroot jail that has a different /dev/urandom could
> work.

Alas, no -- I'm doing various admin chores, and a chroot won't be 
helpful.

> Is your script too complex to rewrite in bash?  Not a general
> solution, but as a workaround it has its appeal.

*sigh* This is probably where I'm gonna wind up (or Perl, or Python).  
Except I've now written a good handful of scripts that people are 
waiting on, and it's gonna cause me physical pain to have to re-do them 
at this point.

C'est la vie.  I guess that's the way the Ruby crumbles...

-Ken
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/

Re: Ruby slow to launch (was L-o-n-g delay for rc.local in systemd on Ubuntu.)

2017-08-08 Thread Bill Freeman 
I don't know, but getrandom() may well be using /dev/urandom (or a related
facility).  And that, in turn, might be waiting to "collect sufficient
entropy".  So some network traffic, keystrokes, whatever, need to happen
between boot time and the first random emission, or that first "random"
number becomes predictable.  Since random numbers are often used
cryptographically, predictability is a bad thing.

As to why ruby is designed to require a random number before being asked to
do something dependent on such a random number is a question for the ruby
developers.

Re-linking /dev/urandom will probably break lots of things.  Maybe run your
script in a chroot jail that has a different /dev/urandom could work.

Is your script too complex to rewrite in bash?  Not a general solution, but
as a workaround it has its appeal.

On Tue, Aug 8, 2017 at 2:24 PM, Ken D'Ambrosio  wrote:

> Well, I tried Tom's solution, and it made not a whit's worth of
> difference.  Because, assuming my ignorance about systemd equated with
> slow boot time, I hadn't troubleshot further than that.  Turns out that
> it's *Ruby's* fault.  A command like this:
> ruby -e 'puts 1'
> is blocking for *THREE MINUTES OR MORE* on getrandom() for the first
> post-boot execution.  (Subsequent ones run fine.)  Which leads to all
> sorts of questions:
> * Why in the Hell do we care about getrandom() when we're printing an
> integer??
> * Couldn't we just use /dev/urandom and be done with it?
> * So much etc.
>
> I love Ruby -- a lot -- but this is bash-my-head-against-the-monitor
> bad.
>
> Any suggestions?  Simply renaming /dev/random to something else and
> doing "ln -s /dev/urandom /dev/random" doesn't seem to be doing the job.
>
> -Ken
> ___
> gnhlug-discuss mailing list
> gnhlug-discuss@mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/