Re: Inconsistencies between juju debug-hooks and "normal" hook environment

[Cory Johns]

Matty,

I do think that a bug should be opened for it, and I will do so.  Is LP
still the better place to open the bug vs GitHub?

On Wed, Oct 21, 2015 at 5:48 AM, Matthew Williams <
matthew.willi...@canonical.com> wrote:

> Hey Folks,
>
> Is there some action we should take to deal with this - in which case can
> we raise an lp bug for it. Or do we think there's no action needed?
>
> Matty
>
> On Tue, Oct 6, 2015 at 3:55 PM, Matt Bruzek 
> wrote:
>
>> Merlijn,
>>
>> Now that I see more of what you are doing I have an alternate suggestion.
>>
>> When you are running 'juju init' use the su command to run as the
>> "ubuntu" user.
>>
>> su - ubuntu -c 'juju init'
>>
>> Using the su command in this way gives a "login" environment for the user
>> "ubuntu".  I wrote a charm (named merlijn) to test this and I saw the error
>> you are reporting:
>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>> unit.merlijn2/0.config-changed logger.go:40 + echo /root
>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>> unit.merlijn2/0.config-changed logger.go:40 /root
>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>> unit.merlijn2/0.config-changed logger.go:40 + juju init
>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>> unit.merlijn2/0.config-changed logger.go:40 error: cannot determine juju
>> home, required environment variables are not set
>>
>> What you were trying to do is create a juju environment for the root user
>> (which is not advised) rather you want to create a juju environment for the
>> normal (ubuntu) user.
>>
>> Using the su command I was able to create a juju configuration file:
>> 2015-10-06 14:47:10 INFO config-changed + echo /root
>> 2015-10-06 14:47:10 INFO config-changed /root
>> 2015-10-06 14:47:10 INFO config-changed + su - ubuntu -c 'juju init'
>> 2015-10-06 14:47:11 INFO config-changed A boilerplate environment
>> configuration file has been written to /home/ubuntu/.juju/environments.yaml.
>>
>> Likewise if you want to create artifacts for the ubuntu user, use the "su
>> - ubuntu" command.
>>
>> Hope that helps.
>>
>>
>>- Matt Bruzek 
>>
>> On Tue, Oct 6, 2015 at 9:30 AM, Cory Johns 
>> wrote:
>>
>>> Merlijn,
>>>
>>> That is an annoying inconsistency.  It's probably related to debug-hooks
>>> bootstrapping itself via `juju ssh` which logs in as the ubuntu user since
>>> root ssh is disabled, as well as it being unlikely that the actual hook
>>> context is a login shell.
>>>
>>> I imagine the generate-config error is due to the lack of either $HOME
>>> or $JUJU_HOME being set.  I would recommend trying explicitly setting
>>> JUJU_HOME=/home/ubuntu before calling generate-config and see if that works
>>> around it.
>>>
>>> I've also created a bug against juju-core for this:
>>> https://github.com/juju/juju/issues/3449
>>>
>>> On Tue, Oct 6, 2015 at 10:06 AM, Merlijn Sebrechts <
>>> merlijn.sebrec...@gmail.com> wrote:
>>>

 Hi Matt


 Thank you for your response. It was nice seeing you too!

 Let me explain my two, maybe related problems a bit more:


 1. Home directory is different between debug and non-debug hook context.

- Using the manual provider.
- When deploying a Charm, the install hook is run as the "root"
user.* "echo ~" prints "/root".*
- When the install hook crashes, I connect to the unit using
`debug-hooks` and restart the install hook as you described using  'juju
resolved --retry unit/#'. After a while, tmux shows "[unit/0] 0:bash-
1:install* " at the bottom and cwd is 
 '/var/lib/juju/agents/unit-0/charm'.
As I understand, I am now in the hook context, right? *However,
"echo ~" prints "/home/ubuntu".* Even though the current user is
root.



 2. Something causes "juju init" to fail in non-debug hook context and
 succeeds in debug hook context.


- Using manual provider
- Executing `juju init` or `juju generate-config` in a hook results
in the error message `error: cannot determine juju home, required
environment variables are not set`
- Executing `juju init` or `juju generate-config` in the hook
context using  `debug-hooks` and  'juju resolved --retry unit/#`
succeeds.


 My current workaround is to generate the initial juju config myself and
 not use "juju generate-config". However, I'm not sure if this behavior is
 intentional or a bug.


 Thanks in advance!


 2015-10-06 15:37 GMT+02:00 Matt Bruzek :

> Hell again Merlijn,
>
> It was nice to see you at the Charmer summit!  This is a very
> interesting use of Juju.
>
> What you are seeing is what we call a "hook context".  Hooks execute
> with root authority and have certain environment variables are set. This 
> is

Re: Inconsistencies between juju debug-hooks and "normal" hook environment

[Casey Marshall]

On Wed, Oct 21, 2015 at 1:49 PM, Cory Johns 
wrote:

> Matty,
>
> I do think that a bug should be opened for it, and I will do so.  Is LP
> still the better place to open the bug vs GitHub?
>

Yes, please open the bug in Launchpad.

Thanks,
Casey


>
> On Wed, Oct 21, 2015 at 5:48 AM, Matthew Williams <
> matthew.willi...@canonical.com> wrote:
>
>> Hey Folks,
>>
>> Is there some action we should take to deal with this - in which case can
>> we raise an lp bug for it. Or do we think there's no action needed?
>>
>> Matty
>>
>> On Tue, Oct 6, 2015 at 3:55 PM, Matt Bruzek > > wrote:
>>
>>> Merlijn,
>>>
>>> Now that I see more of what you are doing I have an alternate suggestion.
>>>
>>> When you are running 'juju init' use the su command to run as the
>>> "ubuntu" user.
>>>
>>> su - ubuntu -c 'juju init'
>>>
>>> Using the su command in this way gives a "login" environment for the
>>> user "ubuntu".  I wrote a charm (named merlijn) to test this and I saw the
>>> error you are reporting:
>>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>>> unit.merlijn2/0.config-changed logger.go:40 + echo /root
>>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>>> unit.merlijn2/0.config-changed logger.go:40 /root
>>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>>> unit.merlijn2/0.config-changed logger.go:40 + juju init
>>> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
>>> unit.merlijn2/0.config-changed logger.go:40 error: cannot determine juju
>>> home, required environment variables are not set
>>>
>>> What you were trying to do is create a juju environment for the root
>>> user (which is not advised) rather you want to create a juju environment
>>> for the normal (ubuntu) user.
>>>
>>> Using the su command I was able to create a juju configuration file:
>>> 2015-10-06 14:47:10 INFO config-changed + echo /root
>>> 2015-10-06 14:47:10 INFO config-changed /root
>>> 2015-10-06 14:47:10 INFO config-changed + su - ubuntu -c 'juju init'
>>> 2015-10-06 14:47:11 INFO config-changed A boilerplate environment
>>> configuration file has been written to /home/ubuntu/.juju/environments.yaml.
>>>
>>> Likewise if you want to create artifacts for the ubuntu user, use the
>>> "su - ubuntu" command.
>>>
>>> Hope that helps.
>>>
>>>
>>>- Matt Bruzek 
>>>
>>> On Tue, Oct 6, 2015 at 9:30 AM, Cory Johns 
>>> wrote:
>>>
 Merlijn,

 That is an annoying inconsistency.  It's probably related to
 debug-hooks bootstrapping itself via `juju ssh` which logs in as the ubuntu
 user since root ssh is disabled, as well as it being unlikely that the
 actual hook context is a login shell.

 I imagine the generate-config error is due to the lack of either $HOME
 or $JUJU_HOME being set.  I would recommend trying explicitly setting
 JUJU_HOME=/home/ubuntu before calling generate-config and see if that works
 around it.

 I've also created a bug against juju-core for this:
 https://github.com/juju/juju/issues/3449

 On Tue, Oct 6, 2015 at 10:06 AM, Merlijn Sebrechts <
 merlijn.sebrec...@gmail.com> wrote:

>
> Hi Matt
>
>
> Thank you for your response. It was nice seeing you too!
>
> Let me explain my two, maybe related problems a bit more:
>
>
> 1. Home directory is different between debug and non-debug hook
> context.
>
>- Using the manual provider.
>- When deploying a Charm, the install hook is run as the "root"
>user.* "echo ~" prints "/root".*
>- When the install hook crashes, I connect to the unit using
>`debug-hooks` and restart the install hook as you described using  
> 'juju
>resolved --retry unit/#'. After a while, tmux shows "[unit/0] 0:bash-
>1:install* " at the bottom and cwd is 
> '/var/lib/juju/agents/unit-0/charm'.
>As I understand, I am now in the hook context, right? *However,
>"echo ~" prints "/home/ubuntu".* Even though the current user is
>root.
>
>
>
> 2. Something causes "juju init" to fail in non-debug hook context and
> succeeds in debug hook context.
>
>
>- Using manual provider
>- Executing `juju init` or `juju generate-config` in a hook
>results in the error message `error: cannot determine juju home,
>required environment variables are not set`
>- Executing `juju init` or `juju generate-config` in the hook
>context using  `debug-hooks` and  'juju resolved --retry unit/#`
>succeeds.
>
>
> My current workaround is to generate the initial juju config myself
> and not use "juju generate-config". However, I'm not sure if this behavior
> is intentional or a bug.
>
>
> Thanks in advance!
>
>
> 2015-10-06 15:37 GMT+02:00 Matt Bruzek :
>
>> Hell again 

Re: Inconsistencies between juju debug-hooks and "normal" hook environment

[Matthew Williams]

Hey Folks,

Is there some action we should take to deal with this - in which case can
we raise an lp bug for it. Or do we think there's no action needed?

Matty

On Tue, Oct 6, 2015 at 3:55 PM, Matt Bruzek 
wrote:

> Merlijn,
>
> Now that I see more of what you are doing I have an alternate suggestion.
>
> When you are running 'juju init' use the su command to run as the "ubuntu"
> user.
>
> su - ubuntu -c 'juju init'
>
> Using the su command in this way gives a "login" environment for the user
> "ubuntu".  I wrote a charm (named merlijn) to test this and I saw the error
> you are reporting:
> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
> unit.merlijn2/0.config-changed logger.go:40 + echo /root
> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
> unit.merlijn2/0.config-changed logger.go:40 /root
> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
> unit.merlijn2/0.config-changed logger.go:40 + juju init
> unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
> unit.merlijn2/0.config-changed logger.go:40 error: cannot determine juju
> home, required environment variables are not set
>
> What you were trying to do is create a juju environment for the root user
> (which is not advised) rather you want to create a juju environment for the
> normal (ubuntu) user.
>
> Using the su command I was able to create a juju configuration file:
> 2015-10-06 14:47:10 INFO config-changed + echo /root
> 2015-10-06 14:47:10 INFO config-changed /root
> 2015-10-06 14:47:10 INFO config-changed + su - ubuntu -c 'juju init'
> 2015-10-06 14:47:11 INFO config-changed A boilerplate environment
> configuration file has been written to /home/ubuntu/.juju/environments.yaml.
>
> Likewise if you want to create artifacts for the ubuntu user, use the "su
> - ubuntu" command.
>
> Hope that helps.
>
>
>- Matt Bruzek 
>
> On Tue, Oct 6, 2015 at 9:30 AM, Cory Johns 
> wrote:
>
>> Merlijn,
>>
>> That is an annoying inconsistency.  It's probably related to debug-hooks
>> bootstrapping itself via `juju ssh` which logs in as the ubuntu user since
>> root ssh is disabled, as well as it being unlikely that the actual hook
>> context is a login shell.
>>
>> I imagine the generate-config error is due to the lack of either $HOME or
>> $JUJU_HOME being set.  I would recommend trying explicitly setting
>> JUJU_HOME=/home/ubuntu before calling generate-config and see if that works
>> around it.
>>
>> I've also created a bug against juju-core for this:
>> https://github.com/juju/juju/issues/3449
>>
>> On Tue, Oct 6, 2015 at 10:06 AM, Merlijn Sebrechts <
>> merlijn.sebrec...@gmail.com> wrote:
>>
>>>
>>> Hi Matt
>>>
>>>
>>> Thank you for your response. It was nice seeing you too!
>>>
>>> Let me explain my two, maybe related problems a bit more:
>>>
>>>
>>> 1. Home directory is different between debug and non-debug hook context.
>>>
>>>- Using the manual provider.
>>>- When deploying a Charm, the install hook is run as the "root" user.* 
>>> "echo
>>>~" prints "/root".*
>>>- When the install hook crashes, I connect to the unit using
>>>`debug-hooks` and restart the install hook as you described using  'juju
>>>resolved --retry unit/#'. After a while, tmux shows "[unit/0] 0:bash-
>>>1:install* " at the bottom and cwd is 
>>> '/var/lib/juju/agents/unit-0/charm'.
>>>As I understand, I am now in the hook context, right? *However,
>>>"echo ~" prints "/home/ubuntu".* Even though the current user is
>>>root.
>>>
>>>
>>>
>>> 2. Something causes "juju init" to fail in non-debug hook context and
>>> succeeds in debug hook context.
>>>
>>>
>>>- Using manual provider
>>>- Executing `juju init` or `juju generate-config` in a hook results
>>>in the error message `error: cannot determine juju home, required
>>>environment variables are not set`
>>>- Executing `juju init` or `juju generate-config` in the hook
>>>context using  `debug-hooks` and  'juju resolved --retry unit/#`
>>>succeeds.
>>>
>>>
>>> My current workaround is to generate the initial juju config myself and
>>> not use "juju generate-config". However, I'm not sure if this behavior is
>>> intentional or a bug.
>>>
>>>
>>> Thanks in advance!
>>>
>>>
>>> 2015-10-06 15:37 GMT+02:00 Matt Bruzek :
>>>
 Hell again Merlijn,

 It was nice to see you at the Charmer summit!  This is a very
 interesting use of Juju.

 What you are seeing is what we call a "hook context".  Hooks execute
 with root authority and have certain environment variables are set. This is
 so Juju commands and tools work correctly.  The debug-hooks command puts
 you into a "hook context" when you run  'juju resolved --retry unit/#'

 As I wrote all hooks are executed with these environment variables
 set.  If you are running the install script manually outside of a hook
 context these environment variables would not be 

Re: Inconsistencies between juju debug-hooks and "normal" hook environment

[Matt Bruzek]

Merlijn,

Now that I see more of what you are doing I have an alternate suggestion.

When you are running 'juju init' use the su command to run as the "ubuntu"
user.

su - ubuntu -c 'juju init'

Using the su command in this way gives a "login" environment for the user
"ubuntu".  I wrote a charm (named merlijn) to test this and I saw the error
you are reporting:
unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
unit.merlijn2/0.config-changed logger.go:40 + echo /root
unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
unit.merlijn2/0.config-changed logger.go:40 /root
unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
unit.merlijn2/0.config-changed logger.go:40 + juju init
unit-merlijn2-0[2994]: 2015-10-06 14:35:26 INFO
unit.merlijn2/0.config-changed logger.go:40 error: cannot determine juju
home, required environment variables are not set

What you were trying to do is create a juju environment for the root user
(which is not advised) rather you want to create a juju environment for the
normal (ubuntu) user.

Using the su command I was able to create a juju configuration file:
2015-10-06 14:47:10 INFO config-changed + echo /root
2015-10-06 14:47:10 INFO config-changed /root
2015-10-06 14:47:10 INFO config-changed + su - ubuntu -c 'juju init'
2015-10-06 14:47:11 INFO config-changed A boilerplate environment
configuration file has been written to /home/ubuntu/.juju/environments.yaml.

Likewise if you want to create artifacts for the ubuntu user, use the "su -
ubuntu" command.

Hope that helps.


   - Matt Bruzek 

On Tue, Oct 6, 2015 at 9:30 AM, Cory Johns  wrote:

> Merlijn,
>
> That is an annoying inconsistency.  It's probably related to debug-hooks
> bootstrapping itself via `juju ssh` which logs in as the ubuntu user since
> root ssh is disabled, as well as it being unlikely that the actual hook
> context is a login shell.
>
> I imagine the generate-config error is due to the lack of either $HOME or
> $JUJU_HOME being set.  I would recommend trying explicitly setting
> JUJU_HOME=/home/ubuntu before calling generate-config and see if that works
> around it.
>
> I've also created a bug against juju-core for this:
> https://github.com/juju/juju/issues/3449
>
> On Tue, Oct 6, 2015 at 10:06 AM, Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
>
>>
>> Hi Matt
>>
>>
>> Thank you for your response. It was nice seeing you too!
>>
>> Let me explain my two, maybe related problems a bit more:
>>
>>
>> 1. Home directory is different between debug and non-debug hook context.
>>
>>- Using the manual provider.
>>- When deploying a Charm, the install hook is run as the "root" user.* 
>> "echo
>>~" prints "/root".*
>>- When the install hook crashes, I connect to the unit using
>>`debug-hooks` and restart the install hook as you described using  'juju
>>resolved --retry unit/#'. After a while, tmux shows "[unit/0] 0:bash-
>>1:install* " at the bottom and cwd is '/var/lib/juju/agents/unit-0/charm'.
>>As I understand, I am now in the hook context, right? *However, "echo
>>~" prints "/home/ubuntu".* Even though the current user is root.
>>
>>
>>
>> 2. Something causes "juju init" to fail in non-debug hook context and
>> succeeds in debug hook context.
>>
>>
>>- Using manual provider
>>- Executing `juju init` or `juju generate-config` in a hook results
>>in the error message `error: cannot determine juju home, required
>>environment variables are not set`
>>- Executing `juju init` or `juju generate-config` in the hook context
>>using  `debug-hooks` and  'juju resolved --retry unit/#` succeeds.
>>
>>
>> My current workaround is to generate the initial juju config myself and
>> not use "juju generate-config". However, I'm not sure if this behavior is
>> intentional or a bug.
>>
>>
>> Thanks in advance!
>>
>>
>> 2015-10-06 15:37 GMT+02:00 Matt Bruzek :
>>
>>> Hell again Merlijn,
>>>
>>> It was nice to see you at the Charmer summit!  This is a very
>>> interesting use of Juju.
>>>
>>> What you are seeing is what we call a "hook context".  Hooks execute
>>> with root authority and have certain environment variables are set. This is
>>> so Juju commands and tools work correctly.  The debug-hooks command puts
>>> you into a "hook context" when you run  'juju resolved --retry unit/#'
>>>
>>> As I wrote all hooks are executed with these environment variables set.
>>> If you are running the install script manually outside of a hook context
>>> these environment variables would not be set.
>>>
>>> There is more information about the environment variables here:
>>>
>>> https://jujucharms.com/docs/devel/authors-hook-environment
>>>
>>> However, I don't see "JUJU_HOME" listed on this page. Perhaps you could
>>> give me more information on what variable is causing a problem and perhaps
>>> your hooks could set those values in the case where the script runs
>>> manually.
>>>
>>> Cheers!
>>>
>>>- 

Re: Inconsistencies between juju debug-hooks and "normal" hook environment

[Merlijn Sebrechts]

Hi


I'm developing a Charm that installs the Juju client, among other things.
I'm having some trouble getting this to work. When I use juju debug-hooks
and manually run the install hook, everything works fine. However, when the
hook is executed by Juju automatically, I get strange errors such as
"error: cannot determine juju home, required environment variables are not
set"

Both the debug-hooks and the normal hook environment execute as root.
However, a lot of env vars are different, like the home dir.

Debug-hooks:
echo ~ : /home/ubuntu
hook env:
echo ~: /root

Is this intentional? This causes a lot of bugs that cannot be reproduced
by debug-hooks. Is there a way to solve  "error: cannot determine juju
home, required environment variables are not set"?



Kind regards
Merlijn Sebrechts
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Inconsistencies between juju debug-hooks and "normal" hook environment

[Matt Bruzek]

Hell again Merlijn,

It was nice to see you at the Charmer summit!  This is a very interesting
use of Juju.

What you are seeing is what we call a "hook context".  Hooks execute with
root authority and have certain environment variables are set. This is so
Juju commands and tools work correctly.  The debug-hooks command puts you
into a "hook context" when you run  'juju resolved --retry unit/#'

As I wrote all hooks are executed with these environment variables set.  If
you are running the install script manually outside of a hook context these
environment variables would not be set.

There is more information about the environment variables here:

https://jujucharms.com/docs/devel/authors-hook-environment

However, I don't see "JUJU_HOME" listed on this page. Perhaps you could
give me more information on what variable is causing a problem and perhaps
your hooks could set those values in the case where the script runs
manually.

Cheers!

   - Matt Bruzek 

On Tue, Oct 6, 2015 at 5:07 AM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> Hi
>
>
> I'm developing a Charm that installs the Juju client, among other things.
> I'm having some trouble getting this to work. When I use juju debug-hooks
> and manually run the install hook, everything works fine. However, when the
> hook is executed by Juju automatically, I get strange errors such as
> "error: cannot determine juju home, required environment variables are not
> set"
>
> Both the debug-hooks and the normal hook environment execute as root.
> However, a lot of env vars are different, like the home dir.
>
> Debug-hooks:
> echo ~ : /home/ubuntu
> hook env:
> echo ~: /root
>
> Is this intentional? This causes a lot of bugs that cannot be reproduced
> by debug-hooks. Is there a way to solve  "error: cannot determine juju
> home, required environment variables are not set"?
>
>
>
> Kind regards
> Merlijn Sebrechts
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju