Re: [Pulp-dev] HEADS UP: Black formatting in pulpcore

2020-04-29 Thread David Davis 
FYI, this has been merged. Thank you mdellweg!

David


On Fri, Apr 24, 2020 at 11:58 AM Matthias Dellweg 
wrote:

> tl;dr black formatting will be enforced in pulpcore end of april.
>
> We decided to finally switch enforcing blacks formatting roles [0] in the
> pulpcore repository. As this will give conflicts with (probably) all
> overlapping contributions, we have set a date for this action.
>
> The switch will be done on the end of 29th of April 2020.
>
> After that, all open PR's will have merge conflicts and need to be
> rebased, and also conform to the coding style set by black. Simply run
> "black ." in the repository root.
>
> [0] https://github.com/pulp/pups/blob/master/pup-0008.md
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Pulp Installer meeting minutes -- April 29, 2020

2020-04-29 Thread Brian Bouterse 
Topics

   -

   Do we still need the pulp_webserver_static_dir role var?
   -

  No, issue filed https://pulp.plan.io/issues/6601
  -

   Do we need to install prereleases? https://pulp.plan.io/issues/6551
   (able to install galaxy_ng without --pre)
   -

  Not for galaxy_ng, but needed as part of CI epic mikedep333 is
  working on
  -

  Next step: move back to NEW until it’s needed as part of mikedep333
  epic
  -

   Change to nginx config snippets dir
   -

  https://github.com/pulp/pulp_installer/pull/278
  -

 https://pulp.plan.io/issues/6594
 -

  https://github.com/pulp/pulp_installer/pull/277#discussion_r416894965
  -

   Adding Changelog for installer
   -

  https://pulp.plan.io/issues/6602
  -

   Time to have docs page?
   -

  Want to have docs that work both on Galaxy and for users off galaxy
  -

  Next step: mikedep333 to develop a plan see action items
  -

   Bin Li’s installer issue
   -

  Dkliban to email asking which version of pulp_installer is being used


Previous Action Items

   -


Action Items

   -

   [fao89] PR 278: Add a separate ansible tasks to enumerate our symlinks,
   and delete them. Tag (per ansible task/block) with “cleanup”.
   -

  Testing will involve installing with ansible-pulp 3.2.x (pulpcore
  3.2.x) on pulplift, and plugins specified via “version”. Then running
  latest ansible-pulp (and “upgrade: true” for plugins.
  -

   [x9c4] https://pulp.plan.io/issues/6601
   -

   [mikedep333] to implement 6602 to review next meeting
   -

   [mikedep333] to come to May 6th installer meeting with an actionable
   plan for how to improve our documentation, maybe a docs site, or maybe
   README.md on galaxy or both
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] fedorapeople.org fixtures

2020-04-29 Thread Tatiana Tereshchenko 
I personally prefer to keep fixtures published somewhere, fedorapeople or
not, doesn't matter.
It is convenient to refer to in situations which are not related to feature
development or functional testing:
 - when one files a redmine issue and provides steps to reproduce
 - when one works with, say, Katello, or any other related project and
needs to try/test something quickly
 - when one tries to help some user remotely and ask to sync this or that.

It's not a strong reason, it's just a matter of convenience, in my opinion.

Tanya


On Wed, Apr 29, 2020 at 8:31 AM Quirin Pamp  wrote:

> I have grown used to always running the fixtures container locally in my
> pulplift boxes using the pfixtures command (essential when working on new
> fixtures).
>
> This command could be made a bit more flexible (right now it always runs
> in the foreground and always uses the latest container image from quay.io),
> but those would be trivial changes.
>
> As a result, I personally have no problems with retiring the fixtures on
> fedorapeople.org completely.
>
> The disadvantage of the approach is that it requires either downloading
> the (pretty large) fixtures container from quay.io, or building it
> locally.
>
>
> Quirin (quba42)
> --
> *From:* pulp-dev-boun...@redhat.com  on
> behalf of David Davis 
> *Sent:* 28 April 2020 22:19:23
> *To:* Pulp-dev 
> *Subject:* [Pulp-dev] fedorapeople.org fixtures
>
> Our Jenkins jobs for Pulp 2 are disabled and that also includes the job
> that builds the fixtures and publishes them to fedorapeople.org[0]. With
> the new pulp-fixtures container[1], it's less essential that we have
> fixtures published somewhere. I think the two options we have are to either
> retire the fedorapeople.org fixtures and remove them, or to move where
> the job runs and possibly the place where they are hosted.
>
> Thoughts?
>
> [0] https://repos.fedorapeople.org/pulp/pulp/fixtures/
> [1] https://quay.io/repository/pulp/pulp-fixtures
>
>
> David
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] CI/CD meeting notes

2020-04-29 Thread David Davis 
April 29, 2020


   -

   Check CI statuses
   -

  pulpcore - failing for 20 days
  -

 Merge single container PR
 -

  pulp - failing for 24 days
  -

 Dkliban to file a bug
 -

 https://pulp.plan.io/issues/6597
 -

  pulp_rpm - failing intermittently?
  -

 Perf tests failing. Will raise with pulp_rpm team. (Failing due
 repos hosted on http://mirror.linux.duke.edu/)
 -

  pulp_installer - failing?
  -

 Mikedep333 is working on it
 -

 Molecule pinned the dependency - Mike will cleanup our own pinning
 -

   Which approach for https://pulp.plan.io/issues/6586
   -

  Do #4, based on whether running in container or not
  -

   What we need to automate the release?
   -

  https://pulp.plan.io/projects/pulp/wiki/Pulp3_Release_Guide
  -

  Automate steps 1-9. Have it output the git tag + push step and the
  query for step 12.
  -

  Goal: have at least one automation task per sprint
  -

  AI: daviddavis to file an issue
  -

   Is there a label or GHA for “merge on CI success and approving review”?
   -


  https://github.com/marketplace/actions/merge-pull-requests#configuration
  -

  AI: mikedep333 to test out action on pulp_installer and
  pulp_rpm_prereqs
  -

  https://github.com/marketplace/actions/merge-pull-requests
  -

 Mike to add on pulp_instaler & rpm_prereqs
 -

 Write access is required to add label. So for now, educate
 reviewers to not add label, so that no drive-by contributor totally
 rewrites the commit after review (before CI finishes). In
long-term, look
 into requiring re-review when a change is made to a PR.
 -

   Can any of these MODIFIED issues be closed out?
   -

  https://pulp.plan.io/issues/5763
  -

  https://pulp.plan.io/issues/5764
  -

  https://pulp.plan.io/issues/5766
  -

  https://pulp.plan.io/issues/5775
  -

  https://pulp.plan.io/issues/6043
  -

  https://pulp.plan.io/issues/6071
  -

  https://pulp.plan.io/issues/6094
  -

  https://pulp.plan.io/issues/6095
  -

  https://pulp.plan.io/issues/6339
  -

  https://pulp.plan.io/issues/6409
  -

  AI: dkliban to close out these issues
  -

   Plugins’ 3.3.0 release process had (easily overcome) errors due to devs
   not updating plugin_template.yml branches variables.
   -

  Mike to add to release process.
  -

   https://github.com/pulp/plugin_template/pull/211
   -

  Concerns about “--all” option
  -

  AI: mdellweg to reach out to mcorr for advice
  -

   Moving repos to travis-ci.com: follow up on pulp-dev list
   -

   AI: mdellweg to file issue to test against python 3.6


David
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] serializers in pulp 3 can't use write_only fields

2020-04-29 Thread Dennis Kliban 
This discussion grew out of concern for the users of bindings. These users
don't usually instantiate models when performing GET requests. Instead,
they pass in a pulp_href to an api client object and receive an
instantiated model. On the other hand, when performing POST/PATCH
operations, the user does instantiate a model. So another solution to the
problem is to have the openapi generator automatically split the
serializers into two when write_only fields are present. However, only
rename the serializer used for GET operations. In this case, users will be
guaranteed that models that they do instantiate themselves, will never
change in name. If there is code that performs some kind of type checking
on the model returned by a GET operation, the user would need to update
that code.

I prefer this solution.

On Mon, Apr 27, 2020 at 9:11 AM Daniel Alley  wrote:

> I just want to point out that we have a somewhat similar kind of feature
> implemented, "minimal serializers", which uses the 2-serializer approach.
> It is a very tiny amount of extra code/work.
>
> e.g.
>
> https://github.com/pulp/pulpcore/blob/master/pulpcore/app/serializers/task.py#L119
>
> https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/serializers.py#L290
>
> the implementation
>
> https://github.com/pulp/pulpcore/blob/master/pulpcore/app/viewsets/base.py#L114-L139
>
> I lean towards option 2 as well for consistency.
>
> On Mon, Apr 27, 2020 at 5:20 AM Ina Panova  wrote:
>
>> I like option number 2 more, even if there is more work behind the scene
>> it is more explicit and does not leave opened questions like ' why certain
>> fields are null'
>> 
>> Regards,
>>
>> Ina Panova
>> Senior Software Engineer| Pulp| Red Hat Inc.
>>
>> "Do not go where the path may lead,
>>  go instead where there is no path and leave a trail."
>>
>>
>> On Fri, Apr 24, 2020 at 5:24 PM Dennis Kliban  wrote:
>>
>>> Today during open floor we concluded that write_only fields cannot be
>>> used in serializers because any automated handling of them during OpenAPI
>>> schema generation will cause mutations to the schema which will result in
>>> backwards incompatible changes to our bindings users.
>>>
>>> This change should only affect the Content serializers that have a
>>> 'file' and 'repository' write_only fields that can be used for uploading
>>> content into a repository in a single request. Two solutions were proposed:
>>>
>>> 1) Use a single serializer for GET and POST and leave 'file' and
>>> 'repository' null for GET requests. This solution is the simplest for
>>> plugin writers because they don't have to do anything extra. However, it
>>> does cause users to wonder why those fields are present and null on GET
>>> requests.
>>>
>>> 2) Require plugin writers to provide 2 serializers for Content - one for
>>> GET and another for POST. This is more work for plugin writers, but will
>>> make it clear to users what fields are expected in each case.
>>>
>>> Even though I initially proposed 1, I am in favor of adopting 2. What
>>> does everyone else think?
>>>
>>> We will need to provide validation at startup that will check
>>> serializers for presence of write_only fields. The application should fail
>>> to start if any are present. This way plugin writers will know not to use
>>> the fields. We will also need to provide plugin writer docs that clearly
>>> state the guidelines around write_only fields and suggestions for getting
>>> by without them.
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Moving to travis-ci.com

2020-04-29 Thread David Davis 
Out of curiosity this morning I tried migrating pulp_file over to travis-ci
this morning and it worked. So then I went back and tried pulp-certguard
and that worked as well. So now both projects are on travis-ci.com.

I think these are the main projects still on travis-ci.org:
- pulp_ansible
- pulp_rpm
- pulp
- pulp_python
- pulp_gem
- pulp-smash
- pulp-ci
- Pulp-2-Tests
- pulp_ostree
- crane

Should we move these projects over as well? Maybe just the Pulp 3 ones? I'd
also like confirmation from plugin teams though before we move their repos
over.

David


On Thu, Apr 23, 2020 at 11:01 AM Daniel Alley  wrote:

> I kid, but it is a funny situation
>
> On Thu, Apr 23, 2020 at 10:01 AM David Davis 
> wrote:
>
>> LOL.
>>
>> To answer your question though, we're kind of in a holding pattern right
>> now. There are some missing features like sharing jobs between workflows
>> and restarting jobs within workflows that we're hoping Github Actions adds.
>> Maybe we can revisit though at our next CI meeting.
>>
>> David
>>
>>
>> On Thu, Apr 23, 2020 at 9:46 AM Daniel Alley  wrote:
>>
>>> travis-ci.org is broken

>>> migrating away from travis-ci.org is also broken

>>>
>>> So, uh, when will github actions be ready xD
>>>
>>> On Thu, Apr 23, 2020 at 8:40 AM David Davis 
>>> wrote:
>>>
 I wrote to Travis support yesterday and they responded (below). I tried
 again this morning and it's still broken. I'll wait a few more days unless
 anyone objects.









 *Thank you for reaching out! We are sorry about the issue that you are
 having.We are currently aware of a bug related to migrating repositories
 from travis-ci.org  to travis-ci.com
  and our Engineering Team is working on this issue to
 be solved as soon as possible.Until the fix is deployed, we would kindly
 suggest you build your projects on travis-ci.org
 .Thank you for your patience and once again we are
 extremely sorry about the issue.*

 David


 On Thu, Apr 23, 2020 at 8:34 AM Brian Bouterse 
 wrote:

>
>
> On Wed, Apr 22, 2020 at 3:48 PM David Davis 
> wrote:
>
>> Ok, I agree. I tried to migrate certguard to travis-ci.com and hit a
>> 500 error. Looks like we're not alone both in status updates not being
>> reported from travis-ci.org and having problems trying to migrate:
>>
>>
>> https://travis-ci.community/t/github-status-not-posted-on-commits-on-repositories-using-legacy-service-integration/7798/16
>>
>> I'll wait a day for Travis support to respond but I think we could
>> also just enable the repos and start from scratch in travis-ci.com.
>> I think we'd lose past build information and settings though if we don't
>> migrate.
>>
> This also sounds good to me. If you want to tag-team via irc or video
> we can do that.
>
>
>> David
>>
>>
>> On Wed, Apr 22, 2020 at 3:21 PM Brian Bouterse 
>> wrote:
>>
>>> OK now more than an annoyance, travis-ci.org not notifying github
>>> is preventing certguard from merging without me disabling the CI check  
>>> :(
>>>
>>> I think we need to do this asap.
>>>
>>> On Wed, Apr 22, 2020 at 9:21 AM Brian Bouterse 
>>> wrote:
>>>
 I don't know the answer to your question about capacity, but I've
 been hitting this "not-updating-github" issue a lot recently. +1 to 
 moving
 everything to travis-ci.com.

 On Wed, Apr 15, 2020 at 1:55 PM David Davis 
 wrote:

> A week or two ago, build statuses for repos whose Ci is hosted on
> travis-ci.org failed to be reported back to Github. This thread
> in the Travis support forum about the issue[0] seems to recommend 
> that we
> migrate off travis-ci.org. My one question is whether we'd
> experience more congestion if we had all our repos on
> travis-ci.com I know this has been an issue in the past.
>
> [0]
> https://travis-ci.community/t/github-status-not-posted-on-commits-on-repositories-using-legacy-service-integration/7798/14
>
> David
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
 ___
 Pulp-dev mailing list
 Pulp-dev@redhat.com
 https://www.redhat.com/mailman/listinfo/pulp-dev

>>>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] fedorapeople.org fixtures

2020-04-29 Thread Quirin Pamp 
I have grown used to always running the fixtures container locally in my 
pulplift boxes using the pfixtures command (essential when working on new 
fixtures).

This command could be made a bit more flexible (right now it always runs in the 
foreground and always uses the latest container image from quay.io), but those 
would be trivial changes.

As a result, I personally have no problems with retiring the fixtures on 
fedorapeople.org completely.

The disadvantage of the approach is that it requires either downloading the 
(pretty large) fixtures container from quay.io, or building it locally.


Quirin (quba42)


From: pulp-dev-boun...@redhat.com  on behalf of 
David Davis 
Sent: 28 April 2020 22:19:23
To: Pulp-dev 
Subject: [Pulp-dev] fedorapeople.org fixtures

Our Jenkins jobs for Pulp 2 are disabled and that also includes the job that 
builds the fixtures and publishes them to 
fedorapeople.org[0]. With the new pulp-fixtures 
container[1], it's less essential that we have fixtures published somewhere. I 
think the two options we have are to either retire the 
fedorapeople.org fixtures and remove them, or to move 
where the job runs and possibly the place where they are hosted.

Thoughts?

[0] https://repos.fedorapeople.org/pulp/pulp/fixtures/
[1] https://quay.io/repository/pulp/pulp-fixtures


David
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev