[Pulp-dev] possible to upload container image blobs?

[Tom McKay]

I know I can upload a skopeo copy tar file but is it possible to upload
individual blobs directly? Or upload the components of the tar file
individually? I am on pulp-2.15.2.
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Migrating Sprint to Custom Field [happening tomorrow]

[Tatiana Tereshchenko]

Thank you! My late but big +1.

Tanya

On Tue, Mar 6, 2018 at 11:04 PM, Brian Bouterse  wrote:

> Thanks for all the feedback! With many +1s and no -1s, I'm going to make
> this change tomorrow. I'll send out links to the result when it is done.
>
> -Brian
>
>
> On Mon, Mar 5, 2018 at 12:43 PM, Jeff Ortel  wrote:
>
>> +1
>>
>>
>> On 03/02/2018 04:17 PM, Brian Bouterse wrote:
>>
>> Redmine's milestone feature allows for roadmap pages to be published for
>> each project on pulp.plan.io like this one [0]. Currently all projects
>> use a single set of milestones from the main 'Pulp' project on Redmine
>> which is what defines the Sprints, e.g. 'Sprint 22'. This creates a few
>> problems:
>>
>> 1. Redmine projects can't use the milestone feature of Redmine for
>> release planning. This is unfortunate since the milestone feature is a good
>> release planning and roadmapping tool.
>>
>> 2. Any project that does use milestones can't have issues associated with
>> milestones also on a sprint. Like pulp_rpm pulp3 issues [0].
>>
>> I'm interested in hearing any solution on resolving these issues, but I
>> also have one to share:
>>
>> 1. We could create a custom field called 'Sprint' and make that available
>> to all projects.
>> 2. Populate the custom field with all existing Sprint values
>> 3. We then port the existing historic sprint issues to the correct custom
>> field which preserves all past sprints.
>> 4. Clear the milestone for all isues on plan.io
>> 5. Delete the old, unused milestones
>> 6. enjoy
>>
>> At least one issue at the upcoming sprint planning meeting won't be able
>> to be added because of this so I'm hoping we can resolve in the next few
>> days.
>>
>> [0]: https://pulp.plan.io/versions/50
>>
>> Thanks!
>> Brian
>>
>>
>> ___
>> Pulp-dev mailing 
>> listPulp-dev@redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] possible to upload container image blobs?

[Ina Panova]

Hello Tom,

No it is not possible to upload individual blobs, the only way how to get
image manifest schema1 or schema2 for now is to use skopeo copy and tar the
output.
For more info please checks the docs and recipes [0].

In addition it is not safe to upload the components of the tar file
individually, because if you upload just a single json file without blobs,
it ends up to be a corrupted image manifest. In case you'd upload blob by
blob, you also risk to end up with corrupted image manifest in case you'd
forget some blob to upload.

Let me know in case you have more questions.

[0]
https://github.com/pulp/pulp_docker/blob/master/docs/user-guide/recipes.rst#upload-v2-schema-2-and-schema-1-images-to-pulp




Regards,

Ina Panova
Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."

On Wed, Mar 7, 2018 at 10:06 PM, Tom McKay  wrote:

> I know I can upload a skopeo copy tar file but is it possible to upload
> individual blobs directly? Or upload the components of the tar file
> individually? I am on pulp-2.15.2.
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] possible to upload container image blobs?

[Tom McKay]

resending since mobile email triggered approval...

i have created an api endpoint in foreman for docker push and from there
will be calling pulp. don’t worry about “safe” since there is no manual
aspect. if pulp doesn’t support it, i will work around. if it us possible,
please tell me how. thanks!!

In my proof-of-concept foreman work, I have docker login working with RBAC
for limiting scope of images for docker pull. I also added the /v2/_catalog
and /v2/$repo/tags/list since those are very useful for integrating with
tools like openshift. If I can get docker push / skopeo working, that would
be a huge win.

I understand that these topics are not a priority but any suggestions on
how to make this work with existing pulp-2 code would be very welcome.


On Wed, Mar 7, 2018 at 9:31 PM, Mihai Ibanescu 
wrote:

> See https://pulp.plan.io/issues/3231 as well.
>
> On Wed, Mar 7, 2018 at 5:25 PM, Ina Panova  wrote:
>
>> Hello Tom,
>>
>> No it is not possible to upload individual blobs, the only way how to get
>> image manifest schema1 or schema2 for now is to use skopeo copy and tar the
>> output.
>> For more info please checks the docs and recipes [0].
>>
>> In addition it is not safe to upload the components of the tar file
>> individually, because if you upload just a single json file without blobs,
>> it ends up to be a corrupted image manifest. In case you'd upload blob by
>> blob, you also risk to end up with corrupted image manifest in case you'd
>> forget some blob to upload.
>>
>> Let me know in case you have more questions.
>>
>> [0] https://github.com/pulp/pulp_docker/blob/master/docs/user-gu
>> ide/recipes.rst#upload-v2-schema-2-and-schema-1-images-to-pulp
>>
>>
>>
>> 
>> Regards,
>>
>> Ina Panova
>> Software Engineer| Pulp| Red Hat Inc.
>>
>> "Do not go where the path may lead,
>>  go instead where there is no path and leave a trail."
>>
>> On Wed, Mar 7, 2018 at 10:06 PM, Tom McKay 
>> wrote:
>>
>>> I know I can upload a skopeo copy tar file but is it possible to upload
>>> individual blobs directly? Or upload the components of the tar file
>>> individually? I am on pulp-2.15.2.
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Next Community Demo March 28

[Brian Bouterse]

FYI, I scheduled the next demo for March 28. We're pushing it out a little
bit so the core team can focus on core a bit more to hopefully wrap it up
into a beta at some point.

Feel free to record demos ahead of time as you do work. The demos are a
great way to communicate to users how things should work. It can be hard to
remember all the good work that has been accomplished when so much time has
passed.

FYI, see these docs on submitted recorded demos:
https://pulp.plan.io/projects/pulp/wiki/Demo_Presenter_Notes

Thanks!
Brian
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] 2.15.3 dev freeze -- 22:00 UTC Tuesday, March 6th

[Milan Kovacik]

On Tue, Mar 6, 2018 at 11:30 PM, Brian Bouterse  wrote:

> Per the release schedule, the code for 2.15.3 is now frozen. There are a
> total of 8 issues spread across core, rpm, and docker. This is ready for
> release engineering. The current beta date for 2.15.3 is March 13th.
>
> https://pulp.plan.io/issues?query_id=59
>
> @milan, these two issues [0][1] are shown as stories so they are not
> included in 2.15.3. They should be included in 2.16 though since they are
> already merged. Will that work?
>

Both stories turned out to be documentation issues only (lack of current
state of affairs description) and in both the cases the patchs are just
straightforward (small additions) docs updates so I'd say should be cool to
cherry-pick into 2.15.3 anyway.  Unsure though how to make that formally
appealing; a comment in the issue/story?

Cheers,
milan


>
> [0]: https://pulp.plan.io/issues/3353
> [1]: https://pulp.plan.io/issues/3339
>
> Thanks,
> Brian
>
>
> On Mon, Mar 5, 2018 at 11:18 AM, Brian Bouterse 
> wrote:
>
>> Any Pulp2 core or plugin code that you want included in the 2.15.3
>> release must be:
>>
>> a) merged to master by 22:00 UTC, March 6th
>> b) be associated with a bugfix issue. stories, refactors, and tasks are
>> not included in z-stream releases.
>>
>> If you want/need to adjust this date, please reply reply on-list.
>>
>> Thanks!
>> Brian
>>
>
>
> On Mon, Mar 5, 2018 at 11:18 AM, Brian Bouterse 
> wrote:
>
>> Any Pulp2 core or plugin code that you want included in the 2.15.3
>> release must be:
>>
>> a) merged to master by 22:00 UTC, March 6th
>> b) be associated with a bugfix issue. stories, refactors, and tasks are
>> not included in z-stream releases.
>>
>> If you want/need to adjust this date, please reply reply on-list.
>>
>> Thanks!
>> Brian
>>
>
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] 2.15.3 dev freeze -- 22:00 UTC Tuesday, March 6th

[Brian Bouterse]

Yes they are docs only. In that case, I think it's ok to include them in
the Z-stream. I switched them to issues since only issues can be in the
z-stream.

The 2.15.3 release now contains 10 issues.
https://pulp.plan.io/issues?query_id=59

Thanks!
Brian

On Wed, Mar 7, 2018 at 4:43 AM, Milan Kovacik  wrote:

>
>
> On Tue, Mar 6, 2018 at 11:30 PM, Brian Bouterse 
> wrote:
>
>> Per the release schedule, the code for 2.15.3 is now frozen. There are a
>> total of 8 issues spread across core, rpm, and docker. This is ready for
>> release engineering. The current beta date for 2.15.3 is March 13th.
>>
>> https://pulp.plan.io/issues?query_id=59
>>
>> @milan, these two issues [0][1] are shown as stories so they are not
>> included in 2.15.3. They should be included in 2.16 though since they are
>> already merged. Will that work?
>>
>
> Both stories turned out to be documentation issues only (lack of current
> state of affairs description) and in both the cases the patchs are just
> straightforward (small additions) docs updates so I'd say should be cool to
> cherry-pick into 2.15.3 anyway.  Unsure though how to make that formally
> appealing; a comment in the issue/story?
>
> Cheers,
> milan
>
>
>>
>> [0]: https://pulp.plan.io/issues/3353
>> [1]: https://pulp.plan.io/issues/3339
>>
>> Thanks,
>> Brian
>>
>>
>> On Mon, Mar 5, 2018 at 11:18 AM, Brian Bouterse 
>> wrote:
>>
>>> Any Pulp2 core or plugin code that you want included in the 2.15.3
>>> release must be:
>>>
>>> a) merged to master by 22:00 UTC, March 6th
>>> b) be associated with a bugfix issue. stories, refactors, and tasks are
>>> not included in z-stream releases.
>>>
>>> If you want/need to adjust this date, please reply reply on-list.
>>>
>>> Thanks!
>>> Brian
>>>
>>
>>
>> On Mon, Mar 5, 2018 at 11:18 AM, Brian Bouterse 
>> wrote:
>>
>>> Any Pulp2 core or plugin code that you want included in the 2.15.3
>>> release must be:
>>>
>>> a) merged to master by 22:00 UTC, March 6th
>>> b) be associated with a bugfix issue. stories, refactors, and tasks are
>>> not included in z-stream releases.
>>>
>>> If you want/need to adjust this date, please reply reply on-list.
>>>
>>> Thanks!
>>> Brian
>>>
>>
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Plugin triage

[Robin Chan]

I would like to hear from QE since I know they had some concerns here.
Since the Pulp QE team is much smaller, having to manage attendance to
multiple traige's per week is more of a strain on them to have the
interruptions and their expertise may not be as clearly delineated as a
larger team which has had some time to align with plugin teams. I hope they
can chime in on their specific concerns here and suggestions on what needs
they have or suggestions.

+1 to Daniel's suggestion on perhaps ordering the items sounds promising.
If we wanted to make a small tweak, maybe we can have the core (and is
packaging meaning build packaging and not a plug in) could go first and try
to order 1-2 plugins (maybe the more active ones that we are triaging more
issues on that others), and then letting the rest go in whatever order
would allow us to evaluate and try out the predictions/concerns Daniel
listed.
I think some of the issue is that some of the items are not always tagged
appropriately? I think if plugin teams could at least ensure those are
tagged appropriately that might cut down on some of the discussion on
those? Or perhaps we can set a rule that if 2 members of plugin team aren't
present, the item is automatically skipped?

Some of my concerns regarding individual plugin triages:
Ina brings up a point that if triage continues to maintain the power to add
things to the current sprint, then lack of whole team feedback could be
problematic. I think we may not want to grant that power until we are
handling plugin staffing truly independently, which I don't believe we are
able to do yet.
If plugin items are triaged separately, will they still have the pulpbot
minutes recorded? Or would that be a downside to doing them independently?

Robin

On Tue, Mar 6, 2018 at 12:30 PM, Ina Panova  wrote:

> It makes sense to let to mini-teams to triage the issues, but the decision
> whether to put or not on the sprint still should be addressed by whole
> team, or at least acknowledged.
>
>
>
> 
> Regards,
>
> Ina Panova
> Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
> On Tue, Mar 6, 2018 at 3:29 AM, Daniel Alley  wrote:
>
>> I'm fine with this.  I dislike the idea of multiple meetings but I think
>> that what will end up happening is that the issue load for each project
>> individually will be low enough that they will can and will all end up
>> being handled asynchronously as they come in.  I also think that letting
>> each plugin decide what is best for them.
>>
>> But just to throw this out there, there are a few other things we could
>> do to help address the problem.
>>
>> We could modify the triage bot to group the issues by type instead of
>> listing them chronologically by number.  All core issues would be handled
>> first, followed by the plugin with the largest number of issues, followed
>> by the plugin with the next largest, etc.  The triage bot could know the
>> composition of the plugin teams and ping the relevant members when a group
>> of issues that concerns them comes up.
>>
>> Pros:
>>
>> - No concerns about lack of cross-pollination, everything is still
>> completely transparent
>> - Community members could still be involved and/or observe the
>> process, which they can't do if every plugin meeting is separate and done
>> in email or IRC
>>
>> Cons:
>>
>> - If you're involved in triaging issues a couple minutes apart, what can
>> you _really_ do in that time?
>> - Multiple interruptions, not *necessarily* gaining much efficiency
>> - Triage lead still would still have to be involved the entire time,
>> whereas ideally someone directly involved with that plugin would be in
>> control
>> - Triage would still take a long time, and would hold up #pulp-dev for
>> that duration
>>
>>
>> On Mon, Mar 5, 2018 at 6:05 PM, Brian Bouterse 
>> wrote:
>>
>>> Currently the biweekly triage query includes a large number of unrelated
>>> topics: Pulp, RPM, Puppet, Python, Ansible (the pulp3 role plugin),
>>> Packaging, OS Tree, Crane, Docker, External, and File Support. These are
>>> all different top-level pulp.plan.io projects in Redmine. These are so
>>> many specializations I think it makes sense to have issues triaged by just
>>> the people who focus on them. Also once per week may or may not be the
>>> right frequency for all of these things which could bring people into
>>> meetings they may not contribute to or benefit from. +1 to having plugin
>>> teams triage issues how they want.
>>>
>>> For Ansible for example, @daviddavis and I can just talk about issues as
>>> they come it. I have it set to email me when they are filed, so we don't
>>> need a meeting at all.
>>>
>>> What about a gradual transition? If/when plugin/project committers
>>> decide to do it differently, then can email pulp-dev asking to be removed
>>> and someone can update the 

Re: [Pulp-dev] Plugin triage

[Preethi Thomas]

On Wed, Mar 7, 2018 at 8:58 AM, Robin Chan  wrote:

> I would like to hear from QE since I know they had some concerns here.
> Since the Pulp QE team is much smaller, having to manage attendance to
> multiple traige's per week is more of a strain on them to have the
> interruptions and their expertise may not be as clearly delineated as a
> larger team which has had some time to align with plugin teams. I hope they
> can chime in on their specific concerns here and suggestions on what needs
> they have or suggestions.
>

This is definitely a concern. Being able to watch triage and having the
triage log available does benefit us in adding issues to the automation
backlog and and in priorotizing them. And their may be specific issues that
QE has filed that would be of particular interest to them. So keeping track
of which triages to attend and the possibility of having overlapping
triages is concerning.


>
> +1 to Daniel's suggestion on perhaps ordering the items sounds promising.
> If we wanted to make a small tweak, maybe we can have the core (and is
> packaging meaning build packaging and not a plug in) could go first and try
> to order 1-2 plugins (maybe the more active ones that we are triaging more
> issues on that others), and then letting the rest go in whatever order
> would allow us to evaluate and try out the predictions/concerns Daniel
> listed.
> I think some of the issue is that some of the items are not always tagged
> appropriately? I think if plugin teams could at least ensure those are
> tagged appropriately that might cut down on some of the discussion on
> those? Or perhaps we can set a rule that if 2 members of plugin team aren't
> present, the item is automatically skipped?
>

+1 to this suggestion. I think this will keep the triage in a somewhat
orderly manner. And allows everyone to participate

>
> Some of my concerns regarding individual plugin triages:
> Ina brings up a point that if triage continues to maintain the power to
> add things to the current sprint, then lack of whole team feedback could be
> problematic. I think we may not want to grant that power until we are
> handling plugin staffing truly independently, which I don't believe we are
> able to do yet.
> If plugin items are triaged separately, will they still have the pulpbot
> minutes recorded? Or would that be a downside to doing them independently?
>

+1 on this as well.

>
>
> Robin
>
> On Tue, Mar 6, 2018 at 12:30 PM, Ina Panova  wrote:
>
>> It makes sense to let to mini-teams to triage the issues, but the
>> decision whether to put or not on the sprint still should be addressed by
>> whole team, or at least acknowledged.
>>
>>
>>
>> 
>> Regards,
>>
>> Ina Panova
>> Software Engineer| Pulp| Red Hat Inc.
>>
>> "Do not go where the path may lead,
>>  go instead where there is no path and leave a trail."
>>
>> On Tue, Mar 6, 2018 at 3:29 AM, Daniel Alley  wrote:
>>
>>> I'm fine with this.  I dislike the idea of multiple meetings but I think
>>> that what will end up happening is that the issue load for each project
>>> individually will be low enough that they will can and will all end up
>>> being handled asynchronously as they come in.  I also think that letting
>>> each plugin decide what is best for them.
>>>
>>> But just to throw this out there, there are a few other things we could
>>> do to help address the problem.
>>>
>>> We could modify the triage bot to group the issues by type instead of
>>> listing them chronologically by number.  All core issues would be handled
>>> first, followed by the plugin with the largest number of issues, followed
>>> by the plugin with the next largest, etc.  The triage bot could know the
>>> composition of the plugin teams and ping the relevant members when a group
>>> of issues that concerns them comes up.
>>>
>>> Pros:
>>>
>>> - No concerns about lack of cross-pollination, everything is still
>>> completely transparent
>>> - Community members could still be involved and/or observe the
>>> process, which they can't do if every plugin meeting is separate and done
>>> in email or IRC
>>>
>>> Cons:
>>>
>>> - If you're involved in triaging issues a couple minutes apart, what can
>>> you _really_ do in that time?
>>> - Multiple interruptions, not *necessarily* gaining much efficiency
>>> - Triage lead still would still have to be involved the entire time,
>>> whereas ideally someone directly involved with that plugin would be in
>>> control
>>> - Triage would still take a long time, and would hold up #pulp-dev for
>>> that duration
>>>
>>>
>>> On Mon, Mar 5, 2018 at 6:05 PM, Brian Bouterse 
>>> wrote:
>>>
 Currently the biweekly triage query includes a large number of
 unrelated topics: Pulp, RPM, Puppet, Python, Ansible (the pulp3 role
 plugin), Packaging, OS Tree, Crane, Docker, External, and File Support.
 These are all different top-level pulp.plan.io projects in