Re: Seeking Advice On Exporting Image Streams

2017-08-11 Thread Devan Goodwin 
Awesome info Maciej, thanks.

Our issue was somewhat postponed as I think we've just decided to
assume we're restoring users back to the same cluster and project name
they were archived from for our first phase, so we'll have to deal
with this at some point but it might be a little further out.

On Fri, Aug 11, 2017 at 4:05 AM, Maciej Szulik  wrote:
>
>
> On Wed, Aug 9, 2017 at 2:49 PM, Devan Goodwin  wrote:
>>
>> We are working on a more robust project export/import process (into a
>> new namespace, possibly a new cluster, etc) and have a question on how
>> to handle image streams.
>>
>> Our first test was with "oc new-app
>> https://github.com/openshift/ruby-hello-world.git";, this results in an
>> image stream like the following:
>>
>> $ oc get is ruby-hello-world -o yaml
>> apiVersion: v1
>> kind: ImageStream
>> metadata:
>>   annotations:
>> openshift.io/generated-by: OpenShiftNewApp
>>   creationTimestamp: 2017-08-08T12:01:22Z
>>   generation: 1
>>   labels:
>> app: ruby-hello-world
>>   name: ruby-hello-world
>>   namespace: project1
>>   resourceVersion: "183991"
>>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
>> spec:
>>   lookupPolicy:
>> local: false
>> status:
>>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>>   tags:
>>   - items:
>> - created: 2017-08-08T12:02:04Z
>>   dockerImageReference:
>>
>> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>>   generation: 1
>>   image:
>> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>> tag: latest
>>
>>
>> If we link up with the kubernetes resource exporting by adding --export:
>>
>> $ oc get is ruby-hello-world -o yaml --export
>> apiVersion: v1
>> kind: ImageStream
>> metadata:
>>   annotations:
>> openshift.io/generated-by: OpenShiftNewApp
>>   creationTimestamp: null
>>   generation: 1
>>   labels:
>> app: ruby-hello-world
>>   name: ruby-hello-world
>>   namespace: default
>>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
>> spec:
>>   lookupPolicy:
>> local: false
>> status:
>>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
>>
>>
>> This leads to an initial question, what stripped the status tags? I
>> would have expected this code to live in the image stream strategy:
>>
>> https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
>> but this does not satisfy RESTExportStrategy, I wasn't able to
>> determine where this is happening.
>
>
> Sorry for late reply. Wrt to image streams you're hitting this issue
> https://github.com/openshift/origin/issues/15492
> Basically this is caused by the fact that the push to internal registry
> updates (or creates) an appropriate image stream.
> But the stream has only status, since no actual spec was ever created and
> can't be created b/c you can't re-import
> that image. During export we can't return you a reliable registry endpoint
> that you can then take and import on a
> different cluster. That's why we're clearing the tags from status and only
> return those from spec. That's the current
> design. But there's some hope when we'll fully introduce the public pull
> spec I've started working in https://github.com/openshift/origin/pull/15643
> When this is all be in place we'll be able to return a spec that is
> importable on a different cluster. Assuming user
> provides appropriate auth secrets to be used during import.
>
> I hope that sheds some light on the topic of IS :)
> Maciej

___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: Seeking Advice On Exporting Image Streams

2017-08-11 Thread Maciej Szulik 
On Wed, Aug 9, 2017 at 2:49 PM, Devan Goodwin  wrote:

> We are working on a more robust project export/import process (into a
> new namespace, possibly a new cluster, etc) and have a question on how
> to handle image streams.
>
> Our first test was with "oc new-app
> https://github.com/openshift/ruby-hello-world.git";, this results in an
> image stream like the following:
>
> $ oc get is ruby-hello-world -o yaml
> apiVersion: v1
> kind: ImageStream
> metadata:
>   annotations:
> openshift.io/generated-by: OpenShiftNewApp
>   creationTimestamp: 2017-08-08T12:01:22Z
>   generation: 1
>   labels:
> app: ruby-hello-world
>   name: ruby-hello-world
>   namespace: project1
>   resourceVersion: "183991"
>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
> spec:
>   lookupPolicy:
> local: false
> status:
>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>   tags:
>   - items:
> - created: 2017-08-08T12:02:04Z
>   dockerImageReference:
> 172.30.1.1:5000/project1/ruby-hello-world@sha256:
> 8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>   generation: 1
>   image: sha256:8d0f81a13ec1b8f8fa4372d26075f0
> dd87578fba2ec120776133db71ce2c2074
> tag: latest
>
>
> If we link up with the kubernetes resource exporting by adding --export:
>
> $ oc get is ruby-hello-world -o yaml --export
> apiVersion: v1
> kind: ImageStream
> metadata:
>   annotations:
> openshift.io/generated-by: OpenShiftNewApp
>   creationTimestamp: null
>   generation: 1
>   labels:
> app: ruby-hello-world
>   name: ruby-hello-world
>   namespace: default
>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
> spec:
>   lookupPolicy:
> local: false
> status:
>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
>
>
> This leads to an initial question, what stripped the status tags? I
> would have expected this code to live in the image stream strategy:
> https://github.com/openshift/origin/blob/master/pkg/image/
> registry/imagestream/strategy.go
> but this does not satisfy RESTExportStrategy, I wasn't able to
> determine where this is happening.
>

Sorry for late reply. Wrt to image streams you're hitting this issue
https://github.com/openshift/origin/issues/15492
Basically this is caused by the fact that the push to internal registry
updates (or creates) an appropriate image stream.
But the stream has only status, since no actual spec was ever created and
can't be created b/c you can't re-import
that image. During export we can't return you a reliable registry endpoint
that you can then take and import on a
different cluster. That's why we're clearing the tags from status and only
return those from spec. That's the current
design. But there's some hope when we'll fully introduce the public pull
spec I've started working in https://github.com/openshift/origin/pull/15643
When this is all be in place we'll be able to return a spec that is
importable on a different cluster. Assuming user
provides appropriate auth secrets to be used during import.

I hope that sheds some light on the topic of IS :)
Maciej
___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Clayton Coleman 
I don't know what that us to do with templates though - what "parameter"
applies?  The registry external name?

On Aug 9, 2017, at 1:22 PM, Cesar Wong  wrote:

The purpose of the endpoint is to parameterize an aspect of the resources
passed to it. We are initially implementing image references, but the idea
is to allow parameterizing other things such as environment variables,
names, etc. This is orthogonal to transforming the resources for export
which ‘GET’ with the export parameter needs to handle.

On Aug 9, 2017, at 1:20 PM, Clayton Coleman  wrote:

Why do we need parameters? Which parameters are we adding?

On Aug 9, 2017, at 12:21 PM, Cesar Wong  wrote:

Hi Devan,

You can see my branch here:
https://github.com/csrwng/origin/tree/parameterize_template
(last 5 commits)

Hopefully should be a PR soon. The REST endpoint should be functional, the
CLI still needs work, but basically the idea is to have the reverse of the
‘oc process’ command, where the input is a list of resources and out comes
a template with parameters.

On Aug 9, 2017, at 11:40 AM, Devan Goodwin  wrote:

On Wed, Aug 9, 2017 at 11:44 AM, Cesar Wong  wrote:

Hi Devan,

This past iteration I started work on this same problem [1]

https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion

The problem is broad and the way I decided to break it up is to consider the
export and parameterize operations independently. The export should be
handled by the resource’s strategy as you mentioned in the Kube issue you
opened. The parameterization part can be a follow up to the export. Here’s
an initial document describing it:

https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing


Thanks that was a good read, will keep an eye on this document.

Does anything exist yet for your parameterization code? Curious what
it looks like and if it's something we could re-use yet, what the
inputs and outputs are, etc.


On the export side, I think we need to decide whether there is different
“types” of export that can happen which should affect the logic of the
resource strategy. For example, does a deployment config look different if
you’re exporting it for use in a different namespace vs a different cluster.
If this is the case, then right now is probably a good time to drive that
change to the upstream API as David suggested.


Is anyone working on a proposal for this export logic upstream? I am
wondering if I should try to put one together if I can find the time.
The general idea (as I understand it) would be to migrate the
currently quite broken export=true param to something strategy based,
and interpret "true" to mean a strategy that matches what we do today.
The references in code I've seen indicate that the current intention
is to strip anything the user cannot specify themselves.




On Aug 9, 2017, at 10:27 AM, Ben Parees  wrote:



On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  wrote:


On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:



On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin 
wrote:


We are working on a more robust project export/import process (into a
new namespace, possibly a new cluster, etc) and have a question on how
to handle image streams.

Our first test was with "oc new-app
https://github.com/openshift/ruby-hello-world.git";, this results in an
image stream like the following:

$ oc get is ruby-hello-world -o yaml
apiVersion: v1
kind: ImageStream
metadata:
 annotations:
   openshift.io/generated-by: OpenShiftNewApp
 creationTimestamp: 2017-08-08T12:01:22Z
 generation: 1
 labels:
   app: ruby-hello-world
 name: ruby-hello-world
 namespace: project1
 resourceVersion: "183991"
 selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
 uid: 4bd229be-7c31-11e7-badf-989096de63cb
spec:
 lookupPolicy:
   local: false
status:
 dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
 tags:
 - items:
   - created: 2017-08-08T12:02:04Z
 dockerImageReference:


172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
 generation: 1
 image:
sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
   tag: latest


If we link up with the kubernetes resource exporting by adding
--export:

$ oc get is ruby-hello-world -o yaml --export
apiVersion: v1
kind: ImageStream
metadata:
 annotations:
   openshift.io/generated-by: OpenShiftNewApp
 creationTimestamp: null
 generation: 1
 labels:
   app: ruby-hello-world
 name: ruby-hello-world
 namespace: default
 selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
spec:
 lookupPolicy:
   local: false
status:
 dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world


This leads to an initial question, what stripped the status tags? I
would have expected this code to live in the image stream strategy:


https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
bu

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Devan Goodwin 
On Wed, Aug 9, 2017 at 2:20 PM, Clayton Coleman  wrote:
> Why do we need parameters? Which parameters are we adding?

For our use case, the idea was detect image refs that point to the
internal registry and the old project name during archival/export,
parameterize, and provide the integrated registry IP and new project
name as params during unarchival/import.

>
> On Aug 9, 2017, at 12:21 PM, Cesar Wong  wrote:
>
> Hi Devan,
>
> You can see my branch here:
> https://github.com/csrwng/origin/tree/parameterize_template
> (last 5 commits)
>
> Hopefully should be a PR soon. The REST endpoint should be functional, the
> CLI still needs work, but basically the idea is to have the reverse of the
> ‘oc process’ command, where the input is a list of resources and out comes a
> template with parameters.
>
> On Aug 9, 2017, at 11:40 AM, Devan Goodwin  wrote:
>
> On Wed, Aug 9, 2017 at 11:44 AM, Cesar Wong  wrote:
>
> Hi Devan,
>
> This past iteration I started work on this same problem [1]
>
> https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion
>
> The problem is broad and the way I decided to break it up is to consider the
> export and parameterize operations independently. The export should be
> handled by the resource’s strategy as you mentioned in the Kube issue you
> opened. The parameterization part can be a follow up to the export. Here’s
> an initial document describing it:
>
> https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing
>
>
> Thanks that was a good read, will keep an eye on this document.
>
> Does anything exist yet for your parameterization code? Curious what
> it looks like and if it's something we could re-use yet, what the
> inputs and outputs are, etc.
>
>
> On the export side, I think we need to decide whether there is different
> “types” of export that can happen which should affect the logic of the
> resource strategy. For example, does a deployment config look different if
> you’re exporting it for use in a different namespace vs a different cluster.
> If this is the case, then right now is probably a good time to drive that
> change to the upstream API as David suggested.
>
>
> Is anyone working on a proposal for this export logic upstream? I am
> wondering if I should try to put one together if I can find the time.
> The general idea (as I understand it) would be to migrate the
> currently quite broken export=true param to something strategy based,
> and interpret "true" to mean a strategy that matches what we do today.
> The references in code I've seen indicate that the current intention
> is to strip anything the user cannot specify themselves.
>
>
>
>
> On Aug 9, 2017, at 10:27 AM, Ben Parees  wrote:
>
>
>
> On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  wrote:
>
>
> On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:
>
>
>
> On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin 
> wrote:
>
>
> We are working on a more robust project export/import process (into a
> new namespace, possibly a new cluster, etc) and have a question on how
> to handle image streams.
>
> Our first test was with "oc new-app
> https://github.com/openshift/ruby-hello-world.git";, this results in an
> image stream like the following:
>
> $ oc get is ruby-hello-world -o yaml
> apiVersion: v1
> kind: ImageStream
> metadata:
>  annotations:
>openshift.io/generated-by: OpenShiftNewApp
>  creationTimestamp: 2017-08-08T12:01:22Z
>  generation: 1
>  labels:
>app: ruby-hello-world
>  name: ruby-hello-world
>  namespace: project1
>  resourceVersion: "183991"
>  selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>  uid: 4bd229be-7c31-11e7-badf-989096de63cb
> spec:
>  lookupPolicy:
>local: false
> status:
>  dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>  tags:
>  - items:
>- created: 2017-08-08T12:02:04Z
>  dockerImageReference:
>
>
> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>  generation: 1
>  image:
> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>tag: latest
>
>
> If we link up with the kubernetes resource exporting by adding
> --export:
>
> $ oc get is ruby-hello-world -o yaml --export
> apiVersion: v1
> kind: ImageStream
> metadata:
>  annotations:
>openshift.io/generated-by: OpenShiftNewApp
>  creationTimestamp: null
>  generation: 1
>  labels:
>app: ruby-hello-world
>  name: ruby-hello-world
>  namespace: default
>  selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
> spec:
>  lookupPolicy:
>local: false
> status:
>  dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
>
>
> This leads to an initial question, what stripped the status tags? I
> would have expected this code to live in the image stream strategy:
>
>
> https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
> but this does

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Cesar Wong 
The purpose of the endpoint is to parameterize an aspect of the resources 
passed to it. We are initially implementing image references, but the idea is 
to allow parameterizing other things such as environment variables, names, etc. 
This is orthogonal to transforming the resources for export which ‘GET’ with 
the export parameter needs to handle.

> On Aug 9, 2017, at 1:20 PM, Clayton Coleman  wrote:
> 
> Why do we need parameters? Which parameters are we adding?
> 
> On Aug 9, 2017, at 12:21 PM, Cesar Wong  > wrote:
> 
>> Hi Devan,
>> 
>> You can see my branch here:
>> https://github.com/csrwng/origin/tree/parameterize_template 
>> 
>> (last 5 commits)
>> 
>> Hopefully should be a PR soon. The REST endpoint should be functional, the 
>> CLI still needs work, but basically the idea is to have the reverse of the 
>> ‘oc process’ command, where the input is a list of resources and out comes a 
>> template with parameters.
>> 
>>> On Aug 9, 2017, at 11:40 AM, Devan Goodwin >> > wrote:
>>> 
>>> On Wed, Aug 9, 2017 at 11:44 AM, Cesar Wong >> > wrote:
 Hi Devan,
 
 This past iteration I started work on this same problem [1]
 
 https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion
  
 
 
 The problem is broad and the way I decided to break it up is to consider 
 the
 export and parameterize operations independently. The export should be
 handled by the resource’s strategy as you mentioned in the Kube issue you
 opened. The parameterization part can be a follow up to the export. Here’s
 an initial document describing it:
 
 https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing
  
 
>>> 
>>> Thanks that was a good read, will keep an eye on this document.
>>> 
>>> Does anything exist yet for your parameterization code? Curious what
>>> it looks like and if it's something we could re-use yet, what the
>>> inputs and outputs are, etc.
>>> 
 
 On the export side, I think we need to decide whether there is different
 “types” of export that can happen which should affect the logic of the
 resource strategy. For example, does a deployment config look different if
 you’re exporting it for use in a different namespace vs a different 
 cluster.
 If this is the case, then right now is probably a good time to drive that
 change to the upstream API as David suggested.
>>> 
>>> Is anyone working on a proposal for this export logic upstream? I am
>>> wondering if I should try to put one together if I can find the time.
>>> The general idea (as I understand it) would be to migrate the
>>> currently quite broken export=true param to something strategy based,
>>> and interpret "true" to mean a strategy that matches what we do today.
>>> The references in code I've seen indicate that the current intention
>>> is to strip anything the user cannot specify themselves.
>>> 
>>> 
>>> 
 
 On Aug 9, 2017, at 10:27 AM, Ben Parees >>> > wrote:
 
 
 
 On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin >>> > wrote:
> 
> On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  > wrote:
>> 
>> 
>> On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin > >
>> wrote:
>>> 
>>> We are working on a more robust project export/import process (into a
>>> new namespace, possibly a new cluster, etc) and have a question on how
>>> to handle image streams.
>>> 
>>> Our first test was with "oc new-app
>>> https://github.com/openshift/ruby-hello-world.git 
>>> ", this results in an
>>> image stream like the following:
>>> 
>>> $ oc get is ruby-hello-world -o yaml
>>> apiVersion: v1
>>> kind: ImageStream
>>> metadata:
>>>  annotations:
>>>openshift.io/generated-by:  
>>> OpenShiftNewApp
>>>  creationTimestamp: 2017-08-08T12:01:22Z
>>>  generation: 1
>>>  labels:
>>>app: ruby-hello-world
>>>  name: ruby-hello-world
>>>  namespace: project1
>>>  resourceVersion: "183991"
>>>  selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>>>  uid: 4bd229be-7c31-11e7-badf-989096de63cb
>>> spec:
>>>  lookupPolicy:
>>>local: false
>>> status:
>>>  dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world 
>>>

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Clayton Coleman 
Why do we need parameters? Which parameters are we adding?

On Aug 9, 2017, at 12:21 PM, Cesar Wong  wrote:

Hi Devan,

You can see my branch here:
https://github.com/csrwng/origin/tree/parameterize_template
(last 5 commits)

Hopefully should be a PR soon. The REST endpoint should be functional, the
CLI still needs work, but basically the idea is to have the reverse of the
‘oc process’ command, where the input is a list of resources and out comes
a template with parameters.

On Aug 9, 2017, at 11:40 AM, Devan Goodwin  wrote:

On Wed, Aug 9, 2017 at 11:44 AM, Cesar Wong  wrote:

Hi Devan,

This past iteration I started work on this same problem [1]

https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion

The problem is broad and the way I decided to break it up is to consider the
export and parameterize operations independently. The export should be
handled by the resource’s strategy as you mentioned in the Kube issue you
opened. The parameterization part can be a follow up to the export. Here’s
an initial document describing it:

https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing


Thanks that was a good read, will keep an eye on this document.

Does anything exist yet for your parameterization code? Curious what
it looks like and if it's something we could re-use yet, what the
inputs and outputs are, etc.


On the export side, I think we need to decide whether there is different
“types” of export that can happen which should affect the logic of the
resource strategy. For example, does a deployment config look different if
you’re exporting it for use in a different namespace vs a different cluster.
If this is the case, then right now is probably a good time to drive that
change to the upstream API as David suggested.


Is anyone working on a proposal for this export logic upstream? I am
wondering if I should try to put one together if I can find the time.
The general idea (as I understand it) would be to migrate the
currently quite broken export=true param to something strategy based,
and interpret "true" to mean a strategy that matches what we do today.
The references in code I've seen indicate that the current intention
is to strip anything the user cannot specify themselves.




On Aug 9, 2017, at 10:27 AM, Ben Parees  wrote:



On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  wrote:


On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:



On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin 
wrote:


We are working on a more robust project export/import process (into a
new namespace, possibly a new cluster, etc) and have a question on how
to handle image streams.

Our first test was with "oc new-app
https://github.com/openshift/ruby-hello-world.git";, this results in an
image stream like the following:

$ oc get is ruby-hello-world -o yaml
apiVersion: v1
kind: ImageStream
metadata:
 annotations:
   openshift.io/generated-by: OpenShiftNewApp
 creationTimestamp: 2017-08-08T12:01:22Z
 generation: 1
 labels:
   app: ruby-hello-world
 name: ruby-hello-world
 namespace: project1
 resourceVersion: "183991"
 selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
 uid: 4bd229be-7c31-11e7-badf-989096de63cb
spec:
 lookupPolicy:
   local: false
status:
 dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
 tags:
 - items:
   - created: 2017-08-08T12:02:04Z
 dockerImageReference:


172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
 generation: 1
 image:
sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
   tag: latest


If we link up with the kubernetes resource exporting by adding
--export:

$ oc get is ruby-hello-world -o yaml --export
apiVersion: v1
kind: ImageStream
metadata:
 annotations:
   openshift.io/generated-by: OpenShiftNewApp
 creationTimestamp: null
 generation: 1
 labels:
   app: ruby-hello-world
 name: ruby-hello-world
 namespace: default
 selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
spec:
 lookupPolicy:
   local: false
status:
 dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world


This leads to an initial question, what stripped the status tags? I
would have expected this code to live in the image stream strategy:


https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
but this does not satisfy RESTExportStrategy, I wasn't able to
determine where this is happening.

The dockerImageRepository in status remains, but weirdly flips from
"project1" to "default" when doing an export. Should this remain in an
exported IS at all? And if so is there any reason why it would flip
from project1 to default?

Our real problem however picks up in the deployment config after
import, in here we end up with the following (partial) DC:

apiVersion: v1
kind: DeploymentConfig
metadata:
 annotations:
   openshift.io/generated-by: OpenShiftNewA

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Cesar Wong 
Hi Devan,

You can see my branch here:
https://github.com/csrwng/origin/tree/parameterize_template 

(last 5 commits)

Hopefully should be a PR soon. The REST endpoint should be functional, the CLI 
still needs work, but basically the idea is to have the reverse of the ‘oc 
process’ command, where the input is a list of resources and out comes a 
template with parameters.

> On Aug 9, 2017, at 11:40 AM, Devan Goodwin  wrote:
> 
> On Wed, Aug 9, 2017 at 11:44 AM, Cesar Wong  wrote:
>> Hi Devan,
>> 
>> This past iteration I started work on this same problem [1]
>> 
>> https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion
>> 
>> The problem is broad and the way I decided to break it up is to consider the
>> export and parameterize operations independently. The export should be
>> handled by the resource’s strategy as you mentioned in the Kube issue you
>> opened. The parameterization part can be a follow up to the export. Here’s
>> an initial document describing it:
>> 
>> https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing
> 
> Thanks that was a good read, will keep an eye on this document.
> 
> Does anything exist yet for your parameterization code? Curious what
> it looks like and if it's something we could re-use yet, what the
> inputs and outputs are, etc.
> 
>> 
>> On the export side, I think we need to decide whether there is different
>> “types” of export that can happen which should affect the logic of the
>> resource strategy. For example, does a deployment config look different if
>> you’re exporting it for use in a different namespace vs a different cluster.
>> If this is the case, then right now is probably a good time to drive that
>> change to the upstream API as David suggested.
> 
> Is anyone working on a proposal for this export logic upstream? I am
> wondering if I should try to put one together if I can find the time.
> The general idea (as I understand it) would be to migrate the
> currently quite broken export=true param to something strategy based,
> and interpret "true" to mean a strategy that matches what we do today.
> The references in code I've seen indicate that the current intention
> is to strip anything the user cannot specify themselves.
> 
> 
> 
>> 
>> On Aug 9, 2017, at 10:27 AM, Ben Parees  wrote:
>> 
>> 
>> 
>> On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  wrote:
>>> 
>>> On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:
 
 
 On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin 
 wrote:
> 
> We are working on a more robust project export/import process (into a
> new namespace, possibly a new cluster, etc) and have a question on how
> to handle image streams.
> 
> Our first test was with "oc new-app
> https://github.com/openshift/ruby-hello-world.git";, this results in an
> image stream like the following:
> 
> $ oc get is ruby-hello-world -o yaml
> apiVersion: v1
> kind: ImageStream
> metadata:
>  annotations:
>openshift.io/generated-by: OpenShiftNewApp
>  creationTimestamp: 2017-08-08T12:01:22Z
>  generation: 1
>  labels:
>app: ruby-hello-world
>  name: ruby-hello-world
>  namespace: project1
>  resourceVersion: "183991"
>  selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>  uid: 4bd229be-7c31-11e7-badf-989096de63cb
> spec:
>  lookupPolicy:
>local: false
> status:
>  dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>  tags:
>  - items:
>- created: 2017-08-08T12:02:04Z
>  dockerImageReference:
> 
> 
> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>  generation: 1
>  image:
> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>tag: latest
> 
> 
> If we link up with the kubernetes resource exporting by adding
> --export:
> 
> $ oc get is ruby-hello-world -o yaml --export
> apiVersion: v1
> kind: ImageStream
> metadata:
>  annotations:
>openshift.io/generated-by: OpenShiftNewApp
>  creationTimestamp: null
>  generation: 1
>  labels:
>app: ruby-hello-world
>  name: ruby-hello-world
>  namespace: default
>  selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
> spec:
>  lookupPolicy:
>local: false
> status:
>  dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
> 
> 
> This leads to an initial question, what stripped the status tags? I
> would have expected this code to live in the image stream strategy:
> 
> 
> https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
> but this does not satisfy RESTExportSt

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Devan Goodwin 
On Wed, Aug 9, 2017 at 11:44 AM, Cesar Wong  wrote:
> Hi Devan,
>
> This past iteration I started work on this same problem [1]
>
> https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion
>
> The problem is broad and the way I decided to break it up is to consider the
> export and parameterize operations independently. The export should be
> handled by the resource’s strategy as you mentioned in the Kube issue you
> opened. The parameterization part can be a follow up to the export. Here’s
> an initial document describing it:
>
> https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing

Thanks that was a good read, will keep an eye on this document.

Does anything exist yet for your parameterization code? Curious what
it looks like and if it's something we could re-use yet, what the
inputs and outputs are, etc.

>
> On the export side, I think we need to decide whether there is different
> “types” of export that can happen which should affect the logic of the
> resource strategy. For example, does a deployment config look different if
> you’re exporting it for use in a different namespace vs a different cluster.
> If this is the case, then right now is probably a good time to drive that
> change to the upstream API as David suggested.

Is anyone working on a proposal for this export logic upstream? I am
wondering if I should try to put one together if I can find the time.
The general idea (as I understand it) would be to migrate the
currently quite broken export=true param to something strategy based,
and interpret "true" to mean a strategy that matches what we do today.
The references in code I've seen indicate that the current intention
is to strip anything the user cannot specify themselves.



>
> On Aug 9, 2017, at 10:27 AM, Ben Parees  wrote:
>
>
>
> On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  wrote:
>>
>> On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:
>> >
>> >
>> > On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin 
>> > wrote:
>> >>
>> >> We are working on a more robust project export/import process (into a
>> >> new namespace, possibly a new cluster, etc) and have a question on how
>> >> to handle image streams.
>> >>
>> >> Our first test was with "oc new-app
>> >> https://github.com/openshift/ruby-hello-world.git";, this results in an
>> >> image stream like the following:
>> >>
>> >> $ oc get is ruby-hello-world -o yaml
>> >> apiVersion: v1
>> >> kind: ImageStream
>> >> metadata:
>> >>   annotations:
>> >> openshift.io/generated-by: OpenShiftNewApp
>> >>   creationTimestamp: 2017-08-08T12:01:22Z
>> >>   generation: 1
>> >>   labels:
>> >> app: ruby-hello-world
>> >>   name: ruby-hello-world
>> >>   namespace: project1
>> >>   resourceVersion: "183991"
>> >>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>> >>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
>> >> spec:
>> >>   lookupPolicy:
>> >> local: false
>> >> status:
>> >>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>> >>   tags:
>> >>   - items:
>> >> - created: 2017-08-08T12:02:04Z
>> >>   dockerImageReference:
>> >>
>> >>
>> >> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>> >>   generation: 1
>> >>   image:
>> >> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>> >> tag: latest
>> >>
>> >>
>> >> If we link up with the kubernetes resource exporting by adding
>> >> --export:
>> >>
>> >> $ oc get is ruby-hello-world -o yaml --export
>> >> apiVersion: v1
>> >> kind: ImageStream
>> >> metadata:
>> >>   annotations:
>> >> openshift.io/generated-by: OpenShiftNewApp
>> >>   creationTimestamp: null
>> >>   generation: 1
>> >>   labels:
>> >> app: ruby-hello-world
>> >>   name: ruby-hello-world
>> >>   namespace: default
>> >>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
>> >> spec:
>> >>   lookupPolicy:
>> >> local: false
>> >> status:
>> >>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
>> >>
>> >>
>> >> This leads to an initial question, what stripped the status tags? I
>> >> would have expected this code to live in the image stream strategy:
>> >>
>> >>
>> >> https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
>> >> but this does not satisfy RESTExportStrategy, I wasn't able to
>> >> determine where this is happening.
>> >>
>> >> The dockerImageRepository in status remains, but weirdly flips from
>> >> "project1" to "default" when doing an export. Should this remain in an
>> >> exported IS at all? And if so is there any reason why it would flip
>> >> from project1 to default?
>> >>
>> >> Our real problem however picks up in the deployment config after
>> >> import, in here we end up with the following (partial) DC:
>> >>
>> >> apiVersion: v1
>> >> kind: DeploymentConfig
>> >> metadata:
>> >>

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Cesar Wong 
Hi Devan,

This past iteration I started work on this same problem [1]

https://trello.com/c/I2ZJxS94/998-5-improve-oc-export-to-parameterize-containerapppromotion
 


The problem is broad and the way I decided to break it up is to consider the 
export and parameterize operations independently. The export should be handled 
by the resource’s strategy as you mentioned in the Kube issue you opened. The 
parameterization part can be a follow up to the export. Here’s an initial 
document describing it:

https://docs.google.com/a/redhat.com/document/d/15SLkhXRovY1dLbxpWFy_Wfq3I6xMznsOAnopTYrXw_A/edit?usp=sharing
 


On the export side, I think we need to decide whether there is different 
“types” of export that can happen which should affect the logic of the resource 
strategy. For example, does a deployment config look different if you’re 
exporting it for use in a different namespace vs a different cluster. If this 
is the case, then right now is probably a good time to drive that change to the 
upstream API as David suggested.

> On Aug 9, 2017, at 10:27 AM, Ben Parees  wrote:
> 
> 
> 
> On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  > wrote:
> On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  > wrote:
> >
> >
> > On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin  > > wrote:
> >>
> >> We are working on a more robust project export/import process (into a
> >> new namespace, possibly a new cluster, etc) and have a question on how
> >> to handle image streams.
> >>
> >> Our first test was with "oc new-app
> >> https://github.com/openshift/ruby-hello-world.git 
> >> ", this results in an
> >> image stream like the following:
> >>
> >> $ oc get is ruby-hello-world -o yaml
> >> apiVersion: v1
> >> kind: ImageStream
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by : 
> >> OpenShiftNewApp
> >>   creationTimestamp: 2017-08-08T12:01:22Z
> >>   generation: 1
> >>   labels:
> >> app: ruby-hello-world
> >>   name: ruby-hello-world
> >>   namespace: project1
> >>   resourceVersion: "183991"
> >>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
> >>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
> >> spec:
> >>   lookupPolicy:
> >> local: false
> >> status:
> >>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world 
> >> 
> >>   tags:
> >>   - items:
> >> - created: 2017-08-08T12:02:04Z
> >>   dockerImageReference:
> >>
> >> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
> >>  
> >> 
> >>   generation: 1
> >>   image:
> >> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
> >> tag: latest
> >>
> >>
> >> If we link up with the kubernetes resource exporting by adding --export:
> >>
> >> $ oc get is ruby-hello-world -o yaml --export
> >> apiVersion: v1
> >> kind: ImageStream
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by : 
> >> OpenShiftNewApp
> >>   creationTimestamp: null
> >>   generation: 1
> >>   labels:
> >> app: ruby-hello-world
> >>   name: ruby-hello-world
> >>   namespace: default
> >>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
> >> spec:
> >>   lookupPolicy:
> >> local: false
> >> status:
> >>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world 
> >> 
> >>
> >>
> >> This leads to an initial question, what stripped the status tags? I
> >> would have expected this code to live in the image stream strategy:
> >>
> >> https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
> >>  
> >> 
> >> but this does not satisfy RESTExportStrategy, I wasn't able to
> >> determine where this is happening.
> >>
> >> The dockerImageRepository in status remains, but weirdly flips from
> >> "project1" to "default" when doing an export. Should this remain in an
> >> exported IS at all? And if so is there any reason why it would flip
> >> from project1 to default?
> >>
> >> Our real problem however picks up in the deployment config after
> >> import, in here we end up with the following (partial) DC:
> >>
> >> apiVersion: v1
> >> kind: DeploymentConfig
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by : 
> >> OpenShi

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Ben Parees 
On Wed, Aug 9, 2017 at 10:00 AM, Devan Goodwin  wrote:

> On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:
> >
> >
> > On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin 
> wrote:
> >>
> >> We are working on a more robust project export/import process (into a
> >> new namespace, possibly a new cluster, etc) and have a question on how
> >> to handle image streams.
> >>
> >> Our first test was with "oc new-app
> >> https://github.com/openshift/ruby-hello-world.git";, this results in an
> >> image stream like the following:
> >>
> >> $ oc get is ruby-hello-world -o yaml
> >> apiVersion: v1
> >> kind: ImageStream
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by: OpenShiftNewApp
> >>   creationTimestamp: 2017-08-08T12:01:22Z
> >>   generation: 1
> >>   labels:
> >> app: ruby-hello-world
> >>   name: ruby-hello-world
> >>   namespace: project1
> >>   resourceVersion: "183991"
> >>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
> >>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
> >> spec:
> >>   lookupPolicy:
> >> local: false
> >> status:
> >>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
> >>   tags:
> >>   - items:
> >> - created: 2017-08-08T12:02:04Z
> >>   dockerImageReference:
> >>
> >> 172.30.1.1:5000/project1/ruby-hello-world@sha256:
> 8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
> >>   generation: 1
> >>   image:
> >> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
> >> tag: latest
> >>
> >>
> >> If we link up with the kubernetes resource exporting by adding --export:
> >>
> >> $ oc get is ruby-hello-world -o yaml --export
> >> apiVersion: v1
> >> kind: ImageStream
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by: OpenShiftNewApp
> >>   creationTimestamp: null
> >>   generation: 1
> >>   labels:
> >> app: ruby-hello-world
> >>   name: ruby-hello-world
> >>   namespace: default
> >>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
> >> spec:
> >>   lookupPolicy:
> >> local: false
> >> status:
> >>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
> >>
> >>
> >> This leads to an initial question, what stripped the status tags? I
> >> would have expected this code to live in the image stream strategy:
> >>
> >> https://github.com/openshift/origin/blob/master/pkg/image/
> registry/imagestream/strategy.go
> >> but this does not satisfy RESTExportStrategy, I wasn't able to
> >> determine where this is happening.
> >>
> >> The dockerImageRepository in status remains, but weirdly flips from
> >> "project1" to "default" when doing an export. Should this remain in an
> >> exported IS at all? And if so is there any reason why it would flip
> >> from project1 to default?
> >>
> >> Our real problem however picks up in the deployment config after
> >> import, in here we end up with the following (partial) DC:
> >>
> >> apiVersion: v1
> >> kind: DeploymentConfig
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by: OpenShiftNewApp
> >>   labels:
> >> app: ruby-hello-world
> >>   name: ruby-hello-world
> >>   namespace: project2
> >>   selfLink:
> >> /oapi/v1/namespaces/project2/deploymentconfigs/ruby-hello-world
> >> spec:
> >>   template:
> >> metadata:
> >>   annotations:
> >> openshift.io/generated-by: OpenShiftNewApp
> >>   labels:
> >> app: ruby-hello-world
> >> deploymentconfig: ruby-hello-world
> >> spec:
> >>   containers:
> >>   - image:
> >> 172.30.1.1:5000/project1/ruby-hello-world@sha256:
> 8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
> >> imagePullPolicy: Always
> >> name: ruby-hello-world
> >>
> >> So our deployment config still refers to a very specific image and
> >> points to the old project. Is there any logic we could apply safely to
> >> address this?
> >>
> >> It feels like this should boil down to something like
> >> "ruby-hello-world@sha256:HASH", could we watch for
> >> $REGISTRY_IP:PORT/projectname/ during export and strip that leading
> >> portion out? What would be the risks in doing so?
> >
> >
> > Adding Cesar since he was recently looking at some of the export logic
> you
> > have questions about and he's also very interested in this subject since
> > he's working on a related piece of functionality.  That said:
> >
> > if you've got an imagechangetrigger in the DC you should be able to strip
> > the entire image field (it should be repopulated from the ICT imagestream
> > reference during deployment).  However:
>
> Ok good, so during export we can iterate the image change triggers, if
> we see one we can match up on containerName and strip container.image
> for that name.
>
> >
> > 1) you still need to straighten out the ICT reference which is also
> going to
> > be pointing to an imagestreamtag in the old project/cluster/whatever
>
> Ok I think we can handle this explicitly. More below tho

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Devan Goodwin 
On Wed, Aug 9, 2017 at 9:58 AM, Ben Parees  wrote:
>
>
> On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin  wrote:
>>
>> We are working on a more robust project export/import process (into a
>> new namespace, possibly a new cluster, etc) and have a question on how
>> to handle image streams.
>>
>> Our first test was with "oc new-app
>> https://github.com/openshift/ruby-hello-world.git";, this results in an
>> image stream like the following:
>>
>> $ oc get is ruby-hello-world -o yaml
>> apiVersion: v1
>> kind: ImageStream
>> metadata:
>>   annotations:
>> openshift.io/generated-by: OpenShiftNewApp
>>   creationTimestamp: 2017-08-08T12:01:22Z
>>   generation: 1
>>   labels:
>> app: ruby-hello-world
>>   name: ruby-hello-world
>>   namespace: project1
>>   resourceVersion: "183991"
>>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
>> spec:
>>   lookupPolicy:
>> local: false
>> status:
>>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>>   tags:
>>   - items:
>> - created: 2017-08-08T12:02:04Z
>>   dockerImageReference:
>>
>> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>>   generation: 1
>>   image:
>> sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>> tag: latest
>>
>>
>> If we link up with the kubernetes resource exporting by adding --export:
>>
>> $ oc get is ruby-hello-world -o yaml --export
>> apiVersion: v1
>> kind: ImageStream
>> metadata:
>>   annotations:
>> openshift.io/generated-by: OpenShiftNewApp
>>   creationTimestamp: null
>>   generation: 1
>>   labels:
>> app: ruby-hello-world
>>   name: ruby-hello-world
>>   namespace: default
>>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
>> spec:
>>   lookupPolicy:
>> local: false
>> status:
>>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
>>
>>
>> This leads to an initial question, what stripped the status tags? I
>> would have expected this code to live in the image stream strategy:
>>
>> https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
>> but this does not satisfy RESTExportStrategy, I wasn't able to
>> determine where this is happening.
>>
>> The dockerImageRepository in status remains, but weirdly flips from
>> "project1" to "default" when doing an export. Should this remain in an
>> exported IS at all? And if so is there any reason why it would flip
>> from project1 to default?
>>
>> Our real problem however picks up in the deployment config after
>> import, in here we end up with the following (partial) DC:
>>
>> apiVersion: v1
>> kind: DeploymentConfig
>> metadata:
>>   annotations:
>> openshift.io/generated-by: OpenShiftNewApp
>>   labels:
>> app: ruby-hello-world
>>   name: ruby-hello-world
>>   namespace: project2
>>   selfLink:
>> /oapi/v1/namespaces/project2/deploymentconfigs/ruby-hello-world
>> spec:
>>   template:
>> metadata:
>>   annotations:
>> openshift.io/generated-by: OpenShiftNewApp
>>   labels:
>> app: ruby-hello-world
>> deploymentconfig: ruby-hello-world
>> spec:
>>   containers:
>>   - image:
>> 172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>> imagePullPolicy: Always
>> name: ruby-hello-world
>>
>> So our deployment config still refers to a very specific image and
>> points to the old project. Is there any logic we could apply safely to
>> address this?
>>
>> It feels like this should boil down to something like
>> "ruby-hello-world@sha256:HASH", could we watch for
>> $REGISTRY_IP:PORT/projectname/ during export and strip that leading
>> portion out? What would be the risks in doing so?
>
>
> Adding Cesar since he was recently looking at some of the export logic you
> have questions about and he's also very interested in this subject since
> he's working on a related piece of functionality.  That said:
>
> if you've got an imagechangetrigger in the DC you should be able to strip
> the entire image field (it should be repopulated from the ICT imagestream
> reference during deployment).  However:

Ok good, so during export we can iterate the image change triggers, if
we see one we can match up on containerName and strip container.image
for that name.

>
> 1) you still need to straighten out the ICT reference which is also going to
> be pointing to an imagestreamtag in the old project/cluster/whatever

Ok I think we can handle this explicitly. More below though.

> 2) if you don't have an ICT reference you do need to sort this out and
> stripping it the way you propose is definitely not a good idea...what's
> going to repopulate that w/ the right prefix/project in the new cluster?
> What if the image field was pointing to docker.io or some other external
> registry?

I definitely wouldn't

Re: Seeking Advice On Exporting Image Streams

2017-08-09 Thread Ben Parees 
On Wed, Aug 9, 2017 at 8:49 AM, Devan Goodwin  wrote:

> We are working on a more robust project export/import process (into a
> new namespace, possibly a new cluster, etc) and have a question on how
> to handle image streams.
>
> Our first test was with "oc new-app
> https://github.com/openshift/ruby-hello-world.git";, this results in an
> image stream like the following:
>
> $ oc get is ruby-hello-world -o yaml
> apiVersion: v1
> kind: ImageStream
> metadata:
>   annotations:
> openshift.io/generated-by: OpenShiftNewApp
>   creationTimestamp: 2017-08-08T12:01:22Z
>   generation: 1
>   labels:
> app: ruby-hello-world
>   name: ruby-hello-world
>   namespace: project1
>   resourceVersion: "183991"
>   selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
>   uid: 4bd229be-7c31-11e7-badf-989096de63cb
> spec:
>   lookupPolicy:
> local: false
> status:
>   dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
>   tags:
>   - items:
> - created: 2017-08-08T12:02:04Z
>   dockerImageReference:
> 172.30.1.1:5000/project1/ruby-hello-world@sha256:
> 8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
>   generation: 1
>   image: sha256:8d0f81a13ec1b8f8fa4372d26075f0
> dd87578fba2ec120776133db71ce2c2074
> tag: latest
>
>
> If we link up with the kubernetes resource exporting by adding --export:
>
> $ oc get is ruby-hello-world -o yaml --export
> apiVersion: v1
> kind: ImageStream
> metadata:
>   annotations:
> openshift.io/generated-by: OpenShiftNewApp
>   creationTimestamp: null
>   generation: 1
>   labels:
> app: ruby-hello-world
>   name: ruby-hello-world
>   namespace: default
>   selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
> spec:
>   lookupPolicy:
> local: false
> status:
>   dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world
>
>
> This leads to an initial question, what stripped the status tags? I
> would have expected this code to live in the image stream strategy:
> https://github.com/openshift/origin/blob/master/pkg/image/
> registry/imagestream/strategy.go
> but this does not satisfy RESTExportStrategy, I wasn't able to
> determine where this is happening.
>
> The dockerImageRepository in status remains, but weirdly flips from
> "project1" to "default" when doing an export. Should this remain in an
> exported IS at all? And if so is there any reason why it would flip
> from project1 to default?
>
> Our real problem however picks up in the deployment config after
> import, in here we end up with the following (partial) DC:
>
> apiVersion: v1
> kind: DeploymentConfig
> metadata:
>   annotations:
> openshift.io/generated-by: OpenShiftNewApp
>   labels:
> app: ruby-hello-world
>   name: ruby-hello-world
>   namespace: project2
>   selfLink: /oapi/v1/namespaces/project2/deploymentconfigs/ruby-hello-
> world
> spec:
>   template:
> metadata:
>   annotations:
> openshift.io/generated-by: OpenShiftNewApp
>   labels:
> app: ruby-hello-world
> deploymentconfig: ruby-hello-world
> spec:
>   containers:
>   - image: 172.30.1.1:5000/project1/ruby-hello-world@sha256:
> 8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
> imagePullPolicy: Always
> name: ruby-hello-world
>
> So our deployment config still refers to a very specific image and
> points to the old project. Is there any logic we could apply safely to
> address this?
>
> It feels like this should boil down to something like
> "ruby-hello-world@sha256:HASH", could we watch for
> $REGISTRY_IP:PORT/projectname/ during export and strip that leading
> portion out? What would be the risks in doing so?
>

Adding Cesar since he was recently looking at some of the export logic you
have questions about and he's also very interested in this subject since
he's working on a related piece of functionality.  That said:

if you've got an imagechangetrigger in the DC you should be able to strip
the entire image field (it should be repopulated from the ICT imagestream
reference during deployment).  However:

1) you still need to straighten out the ICT reference which is also going
to be pointing to an imagestreamtag in the old project/cluster/whatever
2) if you don't have an ICT reference you do need to sort this out and
stripping it the way you propose is definitely not a good idea...what's
going to repopulate that w/ the right prefix/project in the new cluster?
What if the image field was pointing to docker.io or some other external
registry?

In short, you're attempting to tackle a very complex problem where the
answer is frequently "it depends".  We wrote some documentation discussing
some of the considerations when exporting/importing resources between
clusters/projects:

https://docs.openshift.org/latest/dev_guide/application_lifecycle/promoting_applications.html



>
> All help appreciated, thanks.
>
> Devan
>



-- 
Ben Parees | OpenShift

Seeking Advice On Exporting Image Streams

2017-08-09 Thread Devan Goodwin 
We are working on a more robust project export/import process (into a
new namespace, possibly a new cluster, etc) and have a question on how
to handle image streams.

Our first test was with "oc new-app
https://github.com/openshift/ruby-hello-world.git";, this results in an
image stream like the following:

$ oc get is ruby-hello-world -o yaml
apiVersion: v1
kind: ImageStream
metadata:
  annotations:
openshift.io/generated-by: OpenShiftNewApp
  creationTimestamp: 2017-08-08T12:01:22Z
  generation: 1
  labels:
app: ruby-hello-world
  name: ruby-hello-world
  namespace: project1
  resourceVersion: "183991"
  selfLink: /oapi/v1/namespaces/project1/imagestreams/ruby-hello-world
  uid: 4bd229be-7c31-11e7-badf-989096de63cb
spec:
  lookupPolicy:
local: false
status:
  dockerImageRepository: 172.30.1.1:5000/project1/ruby-hello-world
  tags:
  - items:
- created: 2017-08-08T12:02:04Z
  dockerImageReference:
172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
  generation: 1
  image: 
sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
tag: latest


If we link up with the kubernetes resource exporting by adding --export:

$ oc get is ruby-hello-world -o yaml --export
apiVersion: v1
kind: ImageStream
metadata:
  annotations:
openshift.io/generated-by: OpenShiftNewApp
  creationTimestamp: null
  generation: 1
  labels:
app: ruby-hello-world
  name: ruby-hello-world
  namespace: default
  selfLink: /oapi/v1/namespaces/default/imagestreams/ruby-hello-world
spec:
  lookupPolicy:
local: false
status:
  dockerImageRepository: 172.30.1.1:5000/default/ruby-hello-world


This leads to an initial question, what stripped the status tags? I
would have expected this code to live in the image stream strategy:
https://github.com/openshift/origin/blob/master/pkg/image/registry/imagestream/strategy.go
but this does not satisfy RESTExportStrategy, I wasn't able to
determine where this is happening.

The dockerImageRepository in status remains, but weirdly flips from
"project1" to "default" when doing an export. Should this remain in an
exported IS at all? And if so is there any reason why it would flip
from project1 to default?

Our real problem however picks up in the deployment config after
import, in here we end up with the following (partial) DC:

apiVersion: v1
kind: DeploymentConfig
metadata:
  annotations:
openshift.io/generated-by: OpenShiftNewApp
  labels:
app: ruby-hello-world
  name: ruby-hello-world
  namespace: project2
  selfLink: /oapi/v1/namespaces/project2/deploymentconfigs/ruby-hello-world
spec:
  template:
metadata:
  annotations:
openshift.io/generated-by: OpenShiftNewApp
  labels:
app: ruby-hello-world
deploymentconfig: ruby-hello-world
spec:
  containers:
  - image: 
172.30.1.1:5000/project1/ruby-hello-world@sha256:8d0f81a13ec1b8f8fa4372d26075f0dd87578fba2ec120776133db71ce2c2074
imagePullPolicy: Always
name: ruby-hello-world

So our deployment config still refers to a very specific image and
points to the old project. Is there any logic we could apply safely to
address this?

It feels like this should boil down to something like
"ruby-hello-world@sha256:HASH", could we watch for
$REGISTRY_IP:PORT/projectname/ during export and strip that leading
portion out? What would be the risks in doing so?

All help appreciated, thanks.

Devan

___
dev mailing list
dev@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev