Re: [Openstack] heat-watch problem

2013-07-03 Thread Steven Hardy
On Wed, Jul 03, 2013 at 04:21:38PM +0200, Michaël Van de Borne wrote:
> Hello Steven,
> I'm mikemowgli from IRC. As requested, here are the logs.
> 
> 
> 1. First, here's a stack trace I*get in my shell periodically (once
> per minute approximately), but not in the logs: *
> http://pastebin.com/kPswnGNL
> (this might not be related to cloudwatch as I got this permanently)

This is due to https://bugs.launchpad.net/heat/+bug/1196618

Which is now fixed in heat master, and it is unrelated to the heat-watch
authentication failure

> 2. Then, here is the error I get when I perform a heat-watch
> command. The logs of engine and cloudwatch are in attachment. In
> order to minimize their size, I launched and killed the daemons for
> this single heat-watch command.
> 
> It seems that my AWS creds are accepted, but that the user does have
> enough permissions. However, in keystone, the heat user is admin of
> the service tenant. The config files of engine, cloudwatch and boto
> (2.9.0) are also in attachment.

As confirmed on IRC (repeated here for the benefit of others), the reason
is that your user associated with the ec2 keypair has the "heat_stack_user"
role, which is intended to be reserved for users created as part of a heat
stack (where the credentials are deployed inside an implicitly untrusted
instance).

The clues are (from api-cloudwatch.log)

2013-07-03 16:12:13.358 27398 INFO heat.api.aws.ec2token [-] AWS
authentication successful.

which proves the ec2 keypair and resulting signed request via boto is OK

Then we get "User is not authorized to perform action" response, but with
no corresponding log message in the engine.log - so it's the policy.json
rules which limit access to in-instance "heat_stack_user"s which is
preventing access, here's the rule in the /etc/heat/policy.json:

"deny_stack_user": "not role:heat_stack_user",

...

"cloudwatch:DeleteAlarms": "rule:deny_stack_user",
"cloudwatch:DescribeAlarmHistory": "rule:deny_stack_user",
"cloudwatch:DescribeAlarms": "rule:deny_stack_user",
"cloudwatch:DescribeAlarmsForMetric": "rule:deny_stack_user",
"cloudwatch:DisableAlarmActions": "rule:deny_stack_user",
"cloudwatch:EnableAlarmActions": "rule:deny_stack_user",
"cloudwatch:GetMetricStatistics": "rule:deny_stack_user",
"cloudwatch:ListMetrics": "rule:deny_stack_user",
"cloudwatch:PutMetricAlarm": "rule:deny_stack_user",
"cloudwatch:PutMetricData": "",
"cloudwatch:SetAlarmState": "rule:deny_stack_user"

As you can see, the policy only allows users with the heat_stack_user role
to access the PutMetricData action, which is the only action required by
the in-instance user (for the cfn-push-stats agent)

So no "real" users should ever have the "heat_stack_user" role, it should
be reserved for use inside heat.

Steve

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] heat-watch problem

2013-07-03 Thread Michaël Van de Borne
As Steven told me on IRC, the problem was that the user associated with 
my EC2 creds had the heat_stack_user role in keystone.
This role is intended to be used only for the in-instance users, created 
as part of the stack, not real human users. This is described in policy.json


thanks Steven,

btw: any idea about the first problem?

m.


Michaël Van de Borne
R&D Engineer, SOA team, CETIC
Phone: +32 (0)71 49 07 45 Mobile: +32 (0)472 69 57 16, Skype: mikemowgli
www.cetic.be, rue des Frères Wright, 29/3, B-6041 Charleroi

Le 03/07/2013 16:21, Michaël Van de Borne a écrit :

Hello Steven,
I'm mikemowgli from IRC. As requested, here are the logs.


1. First, here's a stack trace I*get in my shell periodically (once 
per minute approximately), but not in the logs: *

http://pastebin.com/kPswnGNL
(this might not be related to cloudwatch as I got this permanently)


2. Then, here is the error I get when I perform a heat-watch command. 
The logs of engine and cloudwatch are in attachment. In order to 
minimize their size, I launched and killed the daemons for this single 
heat-watch command.


It seems that my AWS creds are accepted, but that the user does have 
enough permissions. However, in keystone, the heat user is admin of 
the service tenant. The config files of engine, cloudwatch and boto 
(2.9.0) are also in attachment.


grizzly@leonard:~$ heat-watch -d describe
DEBUG:Debug level logging enabled
INFO:No AlarmName passed, getting results for ALL alarms
DEBUG:Using access key found in config file.
DEBUG:Using secret key found in config file.
DEBUG:Got CW connection object OK
DEBUG:Method: GET
DEBUG:Path: /v1/
DEBUG:Data:
DEBUG:Headers: {}
DEBUG:Host: 192.168.202.103:8003
DEBUG:Params: {'Action': 'DescribeAlarms', 'Version': '2010-08-01', 
'AlarmNames.member.1': None}

DEBUG:establishing HTTP connection: kwargs={'timeout': 70}
DEBUG:Token: None
DEBUG:using _calc_signature_2
DEBUG:query string: 
AWSAccessKeyId=88da7b10ddbe4f4cad198477352ef9fc&Action=DescribeAlarms&AlarmNames.member.1=None&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-07-03T14%3A08%3A54Z&Version=2010-08-01

DEBUG:string_to_sign: GET
192.168.202.103:8003
/v1/
AWSAccessKeyId=88da7b10ddbe4f4cad198477352ef9fc&Action=DescribeAlarms&AlarmNames.member.1=None&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-07-03T14%3A08%3A54Z&Version=2010-08-01
DEBUG:len(b64)=44
DEBUG:base64 encoded digest: UaFV/v+FEOEIStrQR7BAH2ci0uGjlWP+p1TwLO8FVM0=
DEBUG:query_string: 
AWSAccessKeyId=88da7b10ddbe4f4cad198477352ef9fc&Action=DescribeAlarms&AlarmNames.member.1=None&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-07-03T14%3A08%3A54Z&Version=2010-08-01 
Signature: UaFV/v+FEOEIStrQR7BAH2ci0uGjlWP+p1TwLO8FVM0=
DEBUG:User is not authorized to perform 
action:Action DescribeAlarms not allowed for 
userAccessDeniedSender

ERROR:403 AccessDenied
ERROR:User is not authorized to perform 
action:Action DescribeAlarms not allowed for 
userAccessDeniedSender

Traceback (most recent call last):
  File "/usr/local/bin/heat-watch", line 281, in 
main()
  File "/usr/local/bin/heat-watch", line 268, in main
result = cmd(opts, args)
  File 
"/usr/local/lib/python2.7/dist-packages/heat/cfn_client/utils.py", 
line 32, in wrapper

ret = func(*arguments, **kwargs)
  File "/usr/local/bin/heat-watch", line 65, in alarm_describe
result = c.describe_alarm(**parameters)
  File 
"/usr/local/lib/python2.7/dist-packages/heat/cfn_client/boto_client_cloudwatch.py", 
line 57, in describe_alarm

alarm_names=[name])
  File 
"/usr/local/lib/python2.7/dist-packages/boto/ec2/cloudwatch/__init__.py", 
line 393, in describe_alarms

[('MetricAlarms', MetricAlarms)])
  File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", 
line 1049, in get_list

raise self.ResponseError(response.status, response.reason, body)
boto.exception.BotoServerError: BotoServerError: 403 AccessDenied
User is not authorized to perform 
action:Action DescribeAlarms not allowed for 
userAccessDeniedSender



thank you for your help,

michaël


--
Michaël Van de Borne
R&D Engineer, SOA team, CETIC
Phone: +32 (0)71 49 07 45 Mobile: +32 (0)472 69 57 16, Skype: mikemowgli
www.cetic.be, rue des Frères Wright, 29/3, B-6041 Charleroi


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp