[Pulp-dev] Benchmarking Tasking System Results

[Brian Bouterse]

Ahead of the tasking system changes @mdellweg and I are collaborating on,
we wanted to benchmark the current tasking system throughput which I've
measured as a maximum 2.6 tasks / sec.

Check out the test plan, results, and analysis here:
https://hackmd.io/DV633ocwTnShsI8LdajshA

We'll be using this same benchmark on the new-style workers we're putting
together. Look for more info soon on that.

All the best,
Brian
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] Pulp 3 Roadmap for Django Upgrades - 3.2 LTS?

[David Davis]

Great question. We haven't had a compelling reason to upgrade to django 3.2
so there are no current plans or roadmap that I'm aware of.

It looks like the EOL for 2.2 security fixes is April 2022 so we definitely
need to upgrade by then. Our two main stakeholders (Katello and Galaxy) are
both about to release soon so maybe after that (summer/fall) might be a
good opportunity to upgrade to 3.2.

I think it'd be worthwhile for us to see how much of an impact upgrading
3.2 will have on our codebase. To that end, I've filed this issue:

https://pulp.plan.io/issues/8488

Will see if we can look at this soon.

David


On Wed, Mar 31, 2021 at 1:52 PM Chris Hambridge  wrote:

> Hi pulp-dev,
>
> We were discussing upcoming plans for Ansible Automation Platform and one
> of the items we touched on was Django upgrade to 3.2 which is the next LTS
> release. I wanted to reach out and understand if Pulp had any current
> plans/roadmap for a similar upgrade. Given the multiple consumers of Pulp
> and plugins for Pulp, I wasn't sure how much additional complexity that may
> add.
>
> Thanks for the info.
>
> Best,
> Chris
>
> --
>
> Chris Hambridge
>
> Senior Principal Software Engineer
>
> Red Hat 
>
> chamb...@redhat.com
> M: 770.365.6343
> @RedHat    Red Hat
>   Red Hat
> 
> 
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Pulp 3 Roadmap for Django Upgrades - 3.2 LTS?

[Chris Hambridge]

Hi pulp-dev,

We were discussing upcoming plans for Ansible Automation Platform and one
of the items we touched on was Django upgrade to 3.2 which is the next LTS
release. I wanted to reach out and understand if Pulp had any current
plans/roadmap for a similar upgrade. Given the multiple consumers of Pulp
and plugins for Pulp, I wasn't sure how much additional complexity that may
add.

Thanks for the info.

Best,
Chris

-- 

Chris Hambridge

Senior Principal Software Engineer

Red Hat 

chamb...@redhat.com
M: 770.365.6343
@RedHat    Red Hat
  Red Hat


___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] Katello/Pulp3 Integration meeting

[Grant Gainey]

March 31, 2021

Overview

   -

   Katello Schedule
   -

  3.18 November 2020
  -

 pulpcore 3.7
 -

  4.0 branching ~February 2021 (dry-run needed by end-of-Dec)
  -

 pulpcore 3.9
 -

  4.1 branching ~May 2021
  -

 pulpcore 3.10 (or newer)
 -

  4.2 branching ~August 2021
  -

  4.3 branching ~Nov 2021

Pulp

   -

   Pulpcore
   -

  3.12 still on track for 4/6 (or possibly 4/8)
  -

 go/no-go meeting tomorrow
 -

 Migration fixes
 -

 importer/exporter fixes
 -

  Katello to look at using 3.12 for 4.1
  -

  Evgeni applied/tested FIPS downstream 3.11 and Things Worked
  -

 Jsherrill to coordinate releases that need it
 -

   RPM
   -

  3.10 released w/ errata fixes
  -

  Mirroring-repodata discussion - can it be used in 4.1?
  -

 Ttereshc to investigate whether pulpcore-changes needed
 -

 Aiming for pulp_rpm mid-April might work
 -

  Pulp3-depsolve-bug - under investigation
  -

   Migration
   -

  No declared artifact error https://pulp.plan.io/issues/8377
  -

 Issue in the pulpcore, should be solved
 -

Might be causing pulp_container some pain :(
-

  Worked on user reported issues
  -

  ggainey testing migration and FIPS
  -

 There is an existing issue raised https://pulp.plan.io/issues/8453
 -

   Ansible
   -

  Token-auth work postponed
  -

   Pulp Container
   -

  Docker-push issues being investigated/resolved
  -

   Pulp CLI
   -

  Shell-mode added
  -

  0.8.0 release ‘soon’
  -

   PulpCon 2021
   -

  Week of November 8-12


Katello

   -

   Dogfood testing
   -

  Reset, and rerun complete, tanya is going to look into missing rpms
  -

  Issue filed to make mongo-batch-size configurable
  -

  Maybe do a file-repo migration to reproduce rel-path issues that
  blocked us?
  -

  work on 3.11 upgrade looking good
  -

   Ansible collection enhancements ongoing
   -

   Auth’d podman pull support on container gateway
   -

   Pipeline for live vcr testing with latest bindings


QE

   -

   Retesting all test scenarios for final snap
   -

   Continuing to verify ON_QAs
   -

  Reset - https://bugzilla.redhat.com/show_bug.cgi?id=1916759.  Seems
  to work on small scale, but larger db such as customer seems to have
  issues.  Created this bz:
  https://bugzilla.redhat.com/show_bug.cgi?id=1944375
  -

   Waiting on dogfood once devs are done with their testing
   -

   Waiting on fix on customer db from dev so accurate testing on customer
   db can begin
   -

  “Fix” is “get actual RPMs on disk for imported customer DB”
  -

   Do we have enough docker testing?
   -

  There are docker test-cases, so codepaths are being tested
  -

   Running a migration after ‘switchover’ causes some grief
   -

  QE, please file katello BZ to handle this more gracefully


G
-- 
Grant Gainey
Principal Software Engineer, Red Hat System Management Engineering
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] [Pulp-list] PulpCon 2021

[Ina Panova]

+1 for November date.

Another argument to prefer November dates  to September is the preparation
for the AnsibleFest. I think every year we whether get involved in the
booth/talks preparation or some feature delivery to show off.



Regards,

Ina Panova
Senior Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."


On Wed, Mar 31, 2021 at 5:25 PM Tanya Tereshchenko 
wrote:

>
>
> On Wed, Mar 31, 2021 at 2:39 PM Melanie Corr  wrote:
>
>>
>>
>> Ar Céad 31 Márta 2021 ag 13:36, scríobh Grant Gainey > >:
>>
>>> On Wed, Mar 31, 2021 at 8:00 AM David Davis 
>>> wrote:
>>>
 After talking to some people this morning, it sounds like November 8-12
 would also work. I think people would rather meet in the fall when the
 weather is more conducive to being indoors.

 Any objections/thoughts on us having PulpCon the week of Nov 8-12?

>>>
>>> works4me!
>>>
>> Can't think of a better way to spend Nov 8-12
>>
>
> Exactly! Thanks, David.
>
>
>>
>>
>>> G
>>>
>>> --
>>> Grant Gainey
>>> Principal Software Engineer, Red Hat System Management Engineering
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
>>
>> --
>>
>> Melanie Corr, RHCE
>>
>> Community Manager
>>
>> Red Hat 
>>
>> Remote, Ireland
>>
>> mc...@redhat.com
>> M: +353857774436 IM: mcorr
>> 
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] [Pulp-list] PulpCon 2021

[Tanya Tereshchenko]

On Wed, Mar 31, 2021 at 2:39 PM Melanie Corr  wrote:

>
>
> Ar Céad 31 Márta 2021 ag 13:36, scríobh Grant Gainey :
>
>> On Wed, Mar 31, 2021 at 8:00 AM David Davis 
>> wrote:
>>
>>> After talking to some people this morning, it sounds like November 8-12
>>> would also work. I think people would rather meet in the fall when the
>>> weather is more conducive to being indoors.
>>>
>>> Any objections/thoughts on us having PulpCon the week of Nov 8-12?
>>>
>>
>> works4me!
>>
> Can't think of a better way to spend Nov 8-12
>

Exactly! Thanks, David.


>
>
>> G
>>
>> --
>> Grant Gainey
>> Principal Software Engineer, Red Hat System Management Engineering
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>
>
>
> --
>
> Melanie Corr, RHCE
>
> Community Manager
>
> Red Hat 
>
> Remote, Ireland
>
> mc...@redhat.com
> M: +353857774436 IM: mcorr
> 
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] CI/CD meeting minutes

[Brian Bouterse]

On Wed, Mar 31, 2021 at 10:50 AM Matthias Dellweg 
wrote:

>
>
> On Wed, Mar 31, 2021 at 4:42 PM Brian Bouterse 
> wrote:
>
>>
>>
>> On Wed, Mar 31, 2021 at 10:34 AM Fabricio Aguiar 
>> wrote:
>>
>>> ## March 31, 2021
>>>
>>> * Github Actions
>>> * good overall
>>> * faster and more reliable so far than Travis
>>> * easier to debug with tmate action
>>> * downsides
>>> * virtualization
>>> * not possible to re-run individual jobs
>>> * https://github.com/actions/runner/issues/432
>>> * people need write permission to re-run CI
>>> * https://github.com/actions/runner/issues/841
>>> * workaround: `git commit --amend`, no changes, force push
>>> * Release automation https://pulp.plan.io/issues/8093
>>> * Availability
>>> * Mike after operator?
>>> * David has ~1 FTE
>>> * Fabricio? Not sure. Maybe 0.5 FTE.
>>> * Dkliban?
>>> * First goal? https://pulp.plan.io/issues/7868
>>> * [david] to write up design
>>> * Regular CI/CD meeting?
>>> * Every other Wednesday 9:30-9:55am (15:30 EST)
>>> * [david] to schedule
>>> * Convert CI/CD code to ansible playbook?
>>> * Benefits: idempotence, easier to run locally
>>> * *Big* rewrite
>>>
>> I'm a little worried about this even besides its effort. One of my
>> worries if we switch to Ansible is that not everyone knows it. If folks
>> have to learn a new technology that's another barrier to usage. I've seen
>> this happen with the installer already to some extent.
>>
>
> Def valid concern. OTOH this was (and it is missing from the benefits)
> also driven by the idea that our current CI is understood only by a handful
> of people. And that would vastly improve if you could run that locally and
> see where it breaks.
>
That's fair. When I think about why our CI isn't well understood (to me)
it's because of a lack of working with it versus a lack of understanding
the technology that builds it. I'd be more concerned about the latter.
Running locally would be nice tho.

>
>
>>> https://hackmd.io/@pulp/cicd
>>>
>>> Best regards,
>>> Fabricio Aguiar
>>> Software Engineer, Pulp Project
>>> Red Hat Brazil - Latam 
>>> +55 22 999000595
>>>
>>> ___
>>> Pulp-dev mailing list
>>> Pulp-dev@redhat.com
>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] CI/CD meeting minutes

[Matthias Dellweg]

On Wed, Mar 31, 2021 at 4:42 PM Brian Bouterse  wrote:

>
>
> On Wed, Mar 31, 2021 at 10:34 AM Fabricio Aguiar 
> wrote:
>
>> ## March 31, 2021
>>
>> * Github Actions
>> * good overall
>> * faster and more reliable so far than Travis
>> * easier to debug with tmate action
>> * downsides
>> * virtualization
>> * not possible to re-run individual jobs
>> * https://github.com/actions/runner/issues/432
>> * people need write permission to re-run CI
>> * https://github.com/actions/runner/issues/841
>> * workaround: `git commit --amend`, no changes, force push
>> * Release automation https://pulp.plan.io/issues/8093
>> * Availability
>> * Mike after operator?
>> * David has ~1 FTE
>> * Fabricio? Not sure. Maybe 0.5 FTE.
>> * Dkliban?
>> * First goal? https://pulp.plan.io/issues/7868
>> * [david] to write up design
>> * Regular CI/CD meeting?
>> * Every other Wednesday 9:30-9:55am (15:30 EST)
>> * [david] to schedule
>> * Convert CI/CD code to ansible playbook?
>> * Benefits: idempotence, easier to run locally
>> * *Big* rewrite
>>
> I'm a little worried about this even besides its effort. One of my worries
> if we switch to Ansible is that not everyone knows it. If folks have to
> learn a new technology that's another barrier to usage. I've seen this
> happen with the installer already to some extent.
>

Def valid concern. OTOH this was (and it is missing from the benefits) also
driven by the idea that our current CI is understood only by a handful of
people. And that would vastly improve if you could run that locally and see
where it breaks.


>> https://hackmd.io/@pulp/cicd
>>
>> Best regards,
>> Fabricio Aguiar
>> Software Engineer, Pulp Project
>> Red Hat Brazil - Latam 
>> +55 22 999000595
>>
>> ___
>> Pulp-dev mailing list
>> Pulp-dev@redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] CI/CD meeting minutes

[Brian Bouterse]

On Wed, Mar 31, 2021 at 10:34 AM Fabricio Aguiar 
wrote:

> ## March 31, 2021
>
> * Github Actions
> * good overall
> * faster and more reliable so far than Travis
> * easier to debug with tmate action
> * downsides
> * virtualization
> * not possible to re-run individual jobs
> * https://github.com/actions/runner/issues/432
> * people need write permission to re-run CI
> * https://github.com/actions/runner/issues/841
> * workaround: `git commit --amend`, no changes, force push
> * Release automation https://pulp.plan.io/issues/8093
> * Availability
> * Mike after operator?
> * David has ~1 FTE
> * Fabricio? Not sure. Maybe 0.5 FTE.
> * Dkliban?
> * First goal? https://pulp.plan.io/issues/7868
> * [david] to write up design
> * Regular CI/CD meeting?
> * Every other Wednesday 9:30-9:55am (15:30 EST)
> * [david] to schedule
> * Convert CI/CD code to ansible playbook?
> * Benefits: idempotence, easier to run locally
> * *Big* rewrite
>
I'm a little worried about this even besides its effort. One of my worries
if we switch to Ansible is that not everyone knows it. If folks have to
learn a new technology that's another barrier to usage. I've seen this
happen with the installer already to some extent.

>
> https://hackmd.io/@pulp/cicd
>
> Best regards,
> Fabricio Aguiar
> Software Engineer, Pulp Project
> Red Hat Brazil - Latam 
> +55 22 999000595
>
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] CI/CD meeting minutes

[Fabricio Aguiar]

## March 31, 2021

* Github Actions
* good overall
* faster and more reliable so far than Travis
* easier to debug with tmate action
* downsides
* virtualization
* not possible to re-run individual jobs
* https://github.com/actions/runner/issues/432
* people need write permission to re-run CI
* https://github.com/actions/runner/issues/841
* workaround: `git commit --amend`, no changes, force push
* Release automation https://pulp.plan.io/issues/8093
* Availability
* Mike after operator?
* David has ~1 FTE
* Fabricio? Not sure. Maybe 0.5 FTE.
* Dkliban?
* First goal? https://pulp.plan.io/issues/7868
* [david] to write up design
* Regular CI/CD meeting?
* Every other Wednesday 9:30-9:55am (15:30 EST)
* [david] to schedule
* Convert CI/CD code to ansible playbook?
* Benefits: idempotence, easier to run locally
* *Big* rewrite

https://hackmd.io/@pulp/cicd

Best regards,
Fabricio Aguiar
Software Engineer, Pulp Project
Red Hat Brazil - Latam 
+55 22 999000595
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

[Pulp-dev] pulp-cli Meeting Minutes

[Matthias Dellweg]

## March 31, 2021

* We plan to create a feature gap matrix
* Feature, endpoint, plugin, implemented?, implementable?, priority
* Discuss (asynchronously) a release date for 0.8.0 next wednesday
* [mdellweg] start the thread
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] [Pulp-list] PulpCon 2021

[Melanie Corr]

Ar Céad 31 Márta 2021 ag 13:36, scríobh Grant Gainey :

> On Wed, Mar 31, 2021 at 8:00 AM David Davis  wrote:
>
>> After talking to some people this morning, it sounds like November 8-12
>> would also work. I think people would rather meet in the fall when the
>> weather is more conducive to being indoors.
>>
>> Any objections/thoughts on us having PulpCon the week of Nov 8-12?
>>
>
> works4me!
>
Can't think of a better way to spend Nov 8-12

>
> G
>
> --
> Grant Gainey
> Principal Software Engineer, Red Hat System Management Engineering
> ___
> Pulp-dev mailing list
> Pulp-dev@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>


-- 

Melanie Corr, RHCE

Community Manager

Red Hat 

Remote, Ireland

mc...@redhat.com
M: +353857774436 IM: mcorr

___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] [Pulp-list] PulpCon 2021

[Grant Gainey]

On Wed, Mar 31, 2021 at 8:00 AM David Davis  wrote:

> After talking to some people this morning, it sounds like November 8-12
> would also work. I think people would rather meet in the fall when the
> weather is more conducive to being indoors.
>
> Any objections/thoughts on us having PulpCon the week of Nov 8-12?
>

works4me!

G

-- 
Grant Gainey
Principal Software Engineer, Red Hat System Management Engineering
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev

Re: [Pulp-dev] [Pulp-list] PulpCon 2021

[David Davis]

After talking to some people this morning, it sounds like November 8-12
would also work. I think people would rather meet in the fall when the
weather is more conducive to being indoors.

Any objections/thoughts on us having PulpCon the week of Nov 8-12?

David


On Tue, Mar 30, 2021 at 3:01 PM David Davis  wrote:

> It looks like everyone on the team has voted. The week of September 13-17
> seems to be the most popular week. Let's try to aim for some time that
> week. September 20-25 can be a possible backup week if the week of
> September 13 doesn't work out for some reason.
>
> I'll try to get the ball rolling in the next month or so in terms of the
> actual days/times.
>
> Thanks again everyone.
>
> David
>
>
> On Wed, Mar 24, 2021 at 11:15 AM David Davis 
> wrote:
>
>> I haven't heard any objections to having PulpCon virtual this year so
>> under that assumption, I wanted to gather feedback on when to have PulpCon.
>> I've put together a doodle:
>>
>> https://doodle.com/poll/mnnv48t9m35cfzuf
>>
>> I've removed a few weeks that were nonstarters: the week of September 6th
>> has holidays in the US and Brazil; October 25th has holidays in Ireland and
>> CZ; November 1st has holidays in Brazil and Germany; Nov 15th has Brazil
>> and CZ holiday; and Nov 22 is the week of Thanksgiving in the US.
>>
>> Please double check though before voting as I probably missed some
>> holidays.
>>
>> David
>>
>>
>> On Tue, Mar 16, 2021 at 11:08 AM Brian Bouterse 
>> wrote:
>>
>>>
>>>
>>> On Mon, Mar 15, 2021 at 11:12 AM David Davis 
>>> wrote:
>>>
 The topic of PulpCon came up today as spring is usually the time we
 begin to plan PulpCon. The main question I think is whether we should hold
 PulpCon again virtually this year or not.

 Optimally, we'd like to meet in person but given the uncertainty of our
 current situation, I think we should consider going virtual again this
 year. I noticed that other conferences such as DevConf.us (September 2021)
 are already planning to be virtual this year. And also, if we meet
 virtually this year, we could do an in-person PulpCon in early 2022 
 perhaps.

>>> +1 to planning on virtual again for fall 2021. I think it allowed for
>>> more attendance. +1 to also having an in-person event in early 2022.
>>>
>>>
 As for time frame, we'd need at least 3 months to prepare if it's
 virtual. So we'd be looking at sometime between July and December.

 Thoughts?

 David
 ___
 Pulp-list mailing list
 pulp-l...@redhat.com
 https://listman.redhat.com/mailman/listinfo/pulp-list
>>>
>>>
___
Pulp-dev mailing list
Pulp-dev@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-dev