Re: TenantProfile proposal

2018-09-18 Thread Christie, Marcus Aaron 
Thanks Sudhakar.

In that case, I think it would be best to keep the gateway level 
ComputeResourcePreference models for the usageReportingGatewayId field and 
deprecate their other fields.

Thanks,

Marcus

On Sep 18, 2018, at 4:11 PM, Pamidighantam, Sudhakar 
mailto:pamid...@iu.edu>> wrote:

Marcus:
The reporting and ingestion mechanism could be resource dependent. For XSEDE 
resources the gateway id is important and is same across the resources. For 
others like campus gateways it may be different.
Even among XSEDE several current resources do not have this working. So this 
should be not invoked for those.

Thanks,
Sudhakar.


On Sep 18, 2018, at 3:52 PM, Christie, Marcus Aaron 
mailto:machr...@iu.edu>> wrote:

Thanks Sudhakar. One quick question, is the gateway id reported ever different 
for some compute resources than others? That is, would all of the resources for 
which SEAGrid reports usage use the same id (“seagrid”)?  I’m wondering if we 
really need a per compute host settings for the usageReportingGatewayId or if 
this can just be set in one place for the gateway.

Thanks,

Marcus

On Sep 12, 2018, at 6:16 PM, Pamidighantam, Sudhakar 
mailto:pamid...@iu.edu>> wrote:

Thanks Marcus:

Questions
• Where should usageReportingGatewayId go? Should this be set at the gateway 
level? Should this be overrideable by GroupResourceProfiles?

The usage reporting is currently an XSEDE requirement to tract gateway user 
specific usage. The XSEDE gateway usage attributes considers user level usage 
for any gateway for a given HPC system. This could stay at the gateway level. 
We need to provide ways for group heads/PIs to be able to look at usage from 
all users in any group they create/responsible for. But that can be handled by 
other functions.

Sudhakar.

On Sep 12, 2018, at 4:39 PM, Christie, Marcus Aaron 
mailto:machr...@iu.edu>> wrote:

Hi Dev,

A week or so ago, Suresh, Dimuthu and I met to discuss what to do with 
GatewayResourceProfile [0] now that we have the new GroupResourceProfile (on 
the develop branch).  We decided to basically deprecate GatewayResourceProfile 
and move anything in it that is still needed to a new “TenantProfile”. The 
reason for the name Tenant is to keep the terminology more general than 
“Gateway”.

Looking at the code, we already started down this path with the Profile 
Service. It has a TenantProfileService and there is actually a Tenant model 
[1], but actually the TenantProfileService uses the workspace’s Gateway model.

The main fields on the GatewayResourceProfile that aren’t part of the 
GroupResourceProfile are the following:
* default credentialStoreToken - I’m proposing adding a default 
credentialStoreToken field to GroupResourceProfile so we can fully migrate this 
field as well
* identityServerTenant and identityServerPwdCredToken - this is already part of 
the Gateway/Tenant model that TenantProfileService is using so these fields 
already have a home


Here’s a list of the proposed changes:

Profile Service changes
• Create TenantProfile model (see existing tenant_profile_model.thrift which is 
currently unused)
• Rename GatewayEntity to TenantProfileEntity
• Rename gateway->tenant in profile-tenant-cpi.thrift

Experiment Catalog changes
• Change GATEWAY table to just have GATEWAY_ID, only needed for referential 
integrity
• Change the Gateway model to just have gatewayId

App Catalog changes
• Same as Experiment Catalog, just need a Gateway/Tenant model with the ID for 
referential integrity

services-security changes
• Get password credential for identity server from TenantProfileService

GroupResourceProfile changes
• Add a default credential store token to GroupResourceProfile [2]
• Remove following from GroupComputeResourcePreference
• overrideByAiravata
• preferredJobSubmissionProtocol
• preferredDataMovementProtocol
• preferredBatchQueue
• usageReportingGatewayId?
• sshAccountProvisioner fields (move this somewhere else)

GFac/Helix changes
• Remove preferredJobSubmission/DataMovement protocols, preferred batch queue

PGA Changes
• Update names of fields and methods gateway->tenant. Functionality should be 
the same, just a name change.  Gateway request process will need testing

Related changes

  *   Move sshAccountProvisioner fields to its own tables

Questions
• Where should usageReportingGatewayId go? Should this be set at the gateway 
level? Should this be overrideable by GroupResourceProfiles?


Thanks,

Marcus


[0] 
https://issues.apache.org/jira/browse/AIRAVATA-2806
[1]

Re: TenantProfile proposal

2018-09-18 Thread Pamidighantam, Sudhakar 
Marcus:
The reporting and ingestion mechanism could be resource dependent. For XSEDE 
resources the gateway id is important and is same across the resources. For 
others like campus gateways it may be different.
Even among XSEDE several current resources do not have this working. So this 
should be not invoked for those.

Thanks,
Sudhakar.


On Sep 18, 2018, at 3:52 PM, Christie, Marcus Aaron 
mailto:machr...@iu.edu>> wrote:

Thanks Sudhakar. One quick question, is the gateway id reported ever different 
for some compute resources than others? That is, would all of the resources for 
which SEAGrid reports usage use the same id (“seagrid”)?  I’m wondering if we 
really need a per compute host settings for the usageReportingGatewayId or if 
this can just be set in one place for the gateway.

Thanks,

Marcus

On Sep 12, 2018, at 6:16 PM, Pamidighantam, Sudhakar 
mailto:pamid...@iu.edu>> wrote:

Thanks Marcus:

Questions
• Where should usageReportingGatewayId go? Should this be set at the gateway 
level? Should this be overrideable by GroupResourceProfiles?

The usage reporting is currently an XSEDE requirement to tract gateway user 
specific usage. The XSEDE gateway usage attributes considers user level usage 
for any gateway for a given HPC system. This could stay at the gateway level. 
We need to provide ways for group heads/PIs to be able to look at usage from 
all users in any group they create/responsible for. But that can be handled by 
other functions.

Sudhakar.

On Sep 12, 2018, at 4:39 PM, Christie, Marcus Aaron 
mailto:machr...@iu.edu>> wrote:

Hi Dev,

A week or so ago, Suresh, Dimuthu and I met to discuss what to do with 
GatewayResourceProfile [0] now that we have the new GroupResourceProfile (on 
the develop branch).  We decided to basically deprecate GatewayResourceProfile 
and move anything in it that is still needed to a new “TenantProfile”. The 
reason for the name Tenant is to keep the terminology more general than 
“Gateway”.

Looking at the code, we already started down this path with the Profile 
Service. It has a TenantProfileService and there is actually a Tenant model 
[1], but actually the TenantProfileService uses the workspace’s Gateway model.

The main fields on the GatewayResourceProfile that aren’t part of the 
GroupResourceProfile are the following:
* default credentialStoreToken - I’m proposing adding a default 
credentialStoreToken field to GroupResourceProfile so we can fully migrate this 
field as well
* identityServerTenant and identityServerPwdCredToken - this is already part of 
the Gateway/Tenant model that TenantProfileService is using so these fields 
already have a home


Here’s a list of the proposed changes:

Profile Service changes
• Create TenantProfile model (see existing tenant_profile_model.thrift which is 
currently unused)
• Rename GatewayEntity to TenantProfileEntity
• Rename gateway->tenant in profile-tenant-cpi.thrift

Experiment Catalog changes
• Change GATEWAY table to just have GATEWAY_ID, only needed for referential 
integrity
• Change the Gateway model to just have gatewayId

App Catalog changes
• Same as Experiment Catalog, just need a Gateway/Tenant model with the ID for 
referential integrity

services-security changes
• Get password credential for identity server from TenantProfileService

GroupResourceProfile changes
• Add a default credential store token to GroupResourceProfile [2]
• Remove following from GroupComputeResourcePreference
• overrideByAiravata
• preferredJobSubmissionProtocol
• preferredDataMovementProtocol
• preferredBatchQueue
• usageReportingGatewayId?
• sshAccountProvisioner fields (move this somewhere else)

GFac/Helix changes
• Remove preferredJobSubmission/DataMovement protocols, preferred batch queue

PGA Changes
• Update names of fields and methods gateway->tenant. Functionality should be 
the same, just a name change.  Gateway request process will need testing

Related changes

  *   Move sshAccountProvisioner fields to its own tables

Questions
• Where should usageReportingGatewayId go? Should this be set at the gateway 
level? Should this be overrideable by GroupResourceProfiles?


Thanks,

Marcus


[0] 
https://issues.apache.org/jira/browse/AIRAVATA-2806
[1]

Re: TenantProfile proposal

2018-09-18 Thread Christie, Marcus Aaron 
Thanks Sudhakar. One quick question, is the gateway id reported ever different 
for some compute resources than others? That is, would all of the resources for 
which SEAGrid reports usage use the same id (“seagrid”)?  I’m wondering if we 
really need a per compute host settings for the usageReportingGatewayId or if 
this can just be set in one place for the gateway.

Thanks,

Marcus

On Sep 12, 2018, at 6:16 PM, Pamidighantam, Sudhakar 
mailto:pamid...@iu.edu>> wrote:

Thanks Marcus:

Questions
• Where should usageReportingGatewayId go? Should this be set at the gateway 
level? Should this be overrideable by GroupResourceProfiles?

The usage reporting is currently an XSEDE requirement to tract gateway user 
specific usage. The XSEDE gateway usage attributes considers user level usage 
for any gateway for a given HPC system. This could stay at the gateway level. 
We need to provide ways for group heads/PIs to be able to look at usage from 
all users in any group they create/responsible for. But that can be handled by 
other functions.

Sudhakar.

On Sep 12, 2018, at 4:39 PM, Christie, Marcus Aaron 
mailto:machr...@iu.edu>> wrote:

Hi Dev,

A week or so ago, Suresh, Dimuthu and I met to discuss what to do with 
GatewayResourceProfile [0] now that we have the new GroupResourceProfile (on 
the develop branch).  We decided to basically deprecate GatewayResourceProfile 
and move anything in it that is still needed to a new “TenantProfile”. The 
reason for the name Tenant is to keep the terminology more general than 
“Gateway”.

Looking at the code, we already started down this path with the Profile 
Service. It has a TenantProfileService and there is actually a Tenant model 
[1], but actually the TenantProfileService uses the workspace’s Gateway model.

The main fields on the GatewayResourceProfile that aren’t part of the 
GroupResourceProfile are the following:
* default credentialStoreToken - I’m proposing adding a default 
credentialStoreToken field to GroupResourceProfile so we can fully migrate this 
field as well
* identityServerTenant and identityServerPwdCredToken - this is already part of 
the Gateway/Tenant model that TenantProfileService is using so these fields 
already have a home


Here’s a list of the proposed changes:

Profile Service changes
• Create TenantProfile model (see existing tenant_profile_model.thrift which is 
currently unused)
• Rename GatewayEntity to TenantProfileEntity
• Rename gateway->tenant in profile-tenant-cpi.thrift

Experiment Catalog changes
• Change GATEWAY table to just have GATEWAY_ID, only needed for referential 
integrity
• Change the Gateway model to just have gatewayId

App Catalog changes
• Same as Experiment Catalog, just need a Gateway/Tenant model with the ID for 
referential integrity

services-security changes
• Get password credential for identity server from TenantProfileService

GroupResourceProfile changes
• Add a default credential store token to GroupResourceProfile [2]
• Remove following from GroupComputeResourcePreference
• overrideByAiravata
• preferredJobSubmissionProtocol
• preferredDataMovementProtocol
• preferredBatchQueue
• usageReportingGatewayId?
• sshAccountProvisioner fields (move this somewhere else)

GFac/Helix changes
• Remove preferredJobSubmission/DataMovement protocols, preferred batch queue

PGA Changes
• Update names of fields and methods gateway->tenant. Functionality should be 
the same, just a name change.  Gateway request process will need testing

Related changes

  *   Move sshAccountProvisioner fields to its own tables

Questions
• Where should usageReportingGatewayId go? Should this be set at the gateway 
level? Should this be overrideable by GroupResourceProfiles?


Thanks,

Marcus


[0] 
https://issues.apache.org/jira/browse/AIRAVATA-2806
[1] 
https://github.com/apache/airavata/blob/3451a63dd21c96978226bc30754e94196a4519c3/thrift-interface-descriptions/data-models/user-tenant-group-models/tenant_profile_model.thrift#L52-L52
[2]