Re: [Pulp-dev] [Breaking] Stopping the installer from auto-creating migrations - Sept 3rd

2019-08-27 Thread Brian Bouterse
d close 5321 if you prefer. > > On Tue, Aug 27, 2019 at 2:16 PM Brian Bouterse > wrote: > >> tl;dr if we make this change on Sept 3rd, the installer won't auto-create >> migrations anymore. For every change needing a migration, please commit one. >> >> # Backgr

Re: [Pulp-dev] [Breaking] Stopping the installer from auto-creating migrations - Sept 3rd

2019-08-27 Thread Brian Bouterse
ot; but not "makemigrations" along with this change in the installer? My goal is to keep the containers and installer consistent. > On Tue, Aug 27, 2019 at 2:41 PM Brian Bouterse > wrote: > > > > Excellent and good catch on the duplicate already on sprint. Yes let's

Re: [Pulp-dev] [BREAKING] the Pagination Default in pulpcore is changing, bindings may break

2019-08-28 Thread Brian Bouterse
could also trigger a manual push in Travis cron if you need it sooner. [0]: https://travis-ci.com/pulp/pulpcore/builds/124902105 On Fri, Aug 23, 2019 at 5:25 PM Brian Bouterse wrote: > I made a PR for making the pagination default switch: > https://github.com/pulp/pulpcore/pull/275 >

Re: [Pulp-dev] Call for Presenters: Hiatus does not stop demos!

2019-08-28 Thread Brian Bouterse
On Wed, Aug 28, 2019 at 9:17 AM Austin Macdonald wrote: > I suggest posting demos individually. This will be more searchable, better > to link to, and easier to consume. The community update portion could also > be uploaded separately. The downside of fully asynchronous demos is that we > lose th

Re: [Pulp-dev] artifact stages adjustment or any other solution?

2019-08-29 Thread Brian Bouterse
I agree Austin's suggestion of a mechanism like that sounds good. Can a more detailed description be written in Redmine and sent to the list? This change is in Stages API which has such broad impact that I'd feel most comfortable if we could get to that level of detail together somehow. The Stages

Re: [Pulp-dev] artifact stages adjustment or any other solution?

2019-08-29 Thread Brian Bouterse
that adds a declarative artifact in > the middle of the pipeline. > > [0] > > https://github.com/pulp/pulp_deb/blob/master/pulp_deb/app/tasks/synchronizing.py#L204 > > On Thu, 29 Aug 2019 10:42:33 -0400 > Brian Bouterse wrote: > > > I agree Austin's suggestion of

[Pulp-dev] Pulpcore RC5 Release Planning

2019-09-02 Thread Brian Bouterse
tl;dr -- Let's release pulpcore and pulpcore-plugin on Tuesday Sept 10th. Please identify any backwards incompatible changes or high-risk changes as blockers here [2]. It's been ~ 5 weeks since pulpcore and pulpcore-plugin have released. There are several changes ready for each [0][1] . I propo

Re: [Pulp-dev] status of the ansible postgres role

2019-09-03 Thread Brian Bouterse
I'm also in favor of doing option 1 until option 2 is possible. I had an existing email thread with the upstream maintainer going. To work towards option (2) I've bumped that thread with him just now, identifying the PRs below and asking him to review+feedback or review+merge. https://github.com/

Re: [Pulp-dev] Namespacing one shot upload and copy endpoints

2019-09-04 Thread Brian Bouterse
I want to bring back a variation on upload needs that has come out of various discussions w/ several plugin developers. I wrote it up here in more detail and I'll sumarize below: https://pulp.plan.io/issues/5403 1. Make all uploads of a specific content type live at POST /pulp/api/v3/content/// 2

Re: [Pulp-dev] renaming user facing fields on content serializers

2019-09-10 Thread Brian Bouterse
I also agree with this. As a user, it would feel random to have to specify a field as _artifact or _relative_path when the underscore isn't a well-known convention. +1 to filing an issue and proceeding to fix. It'll be a breaking change so it needs to be declared on the mailing list with the [BREA

[Pulp-dev] State of Pulp3 Settings

2019-09-12 Thread Brian Bouterse
I want to share an update on the state of settings in Pulp3. Pulp uses dynaconf for settings. Dynaconf provides lots of ways to do this, and Pulp recently enabled Pulp plugins to provide settings directly too. # settings plugins set for their users Plugins can now provide plugins programatically.

[Pulp-dev] Pulp3 Installer CI Blocked -- Need Help

2019-09-18 Thread Brian Bouterse
The Installer's molecule tests started failing yesterday-ish on master. We think it's related to the release of centos7.7 maybe, but that isn't confirmed. I'm trying to get help with two goals: 1) getting the tests passing <-- this is the immediate goal. 2) Getting more info on how to run the tes

Re: [Pulp-dev] pulp3 architecture / HA

2019-10-01 Thread Brian Bouterse
Pulp3's 3 primary software components are the rest-api, content-app, and the tasking system (its workers). Each is scalable by deploying more processes, which can provide both increased availability and capacity. The docs for the architecture and deployment are here: https://docs.pulpproject.org/en

Re: [Pulp-dev] [pulp-dev] modularity upload

2019-10-02 Thread Brian Bouterse
The additional endpoint sounds fine, especially since uploading the modules.yaml file will produce two types of content units ModuleMD andModuleMDDefaults (is my understanding). What's important to me is that our users have similar functionality anytime they upload. It's not as important to me tha

Re: [Pulp-dev] [pulp-dev] modularity upload

2019-10-02 Thread Brian Bouterse
On Wed, Oct 2, 2019 at 8:38 AM Tatiana Tereshchenko wrote: > +1 for custom endpoint for modules.yaml upload. > As a reminder, modules are special, they often come in a batch which > contains 2 content types. > To upload modules.yaml file means to upload/add X modulemd content units > and Y module

Re: [Pulp-dev] Google's Code Reviewer Guide

2019-10-02 Thread Brian Bouterse
Yes thank you! I read it, and I will strive towards its practices. On Wed, Oct 2, 2019 at 8:40 AM Tatiana Tereshchenko wrote: > Thanks for sharing! > > Tanya > > On Tue, Oct 1, 2019 at 1:52 PM David Davis wrote: > >> I've been reading through Google's Code Reviewer Guide and I've found it >> r

Re: [Pulp-dev] ContentViewset assumes that POST is needed for any content

2019-10-02 Thread Brian Bouterse
On Wed, Oct 2, 2019 at 11:03 AM Tatiana Tereshchenko wrote: > Thank you! Good idea to have an additional one. > I agree I'm on the fence between ReadOnly and just Generic one without any mixins. > I see having the ReadOnlyContentViewset as overall being slightly better than a Generic option for

Re: [Pulp-dev] ContentViewset assumes that POST is needed for any content

2019-10-02 Thread Brian Bouterse
On Wed, Oct 2, 2019 at 12:44 PM David Davis wrote: > > On Wed, Oct 2, 2019 at 12:37 PM Brian Bouterse > wrote: > >> >> >> On Wed, Oct 2, 2019 at 11:03 AM Tatiana Tereshchenko >> wrote: >> >>> Thank you! Good idea to have an additional one. >

Re: [Pulp-dev] [pulp-dev] modularity upload

2019-10-03 Thread Brian Bouterse
On Wed, Oct 2, 2019 at 9:55 AM Pavel Picka wrote: > > > On Wed, Oct 2, 2019 at 2:42 PM Brian Bouterse wrote: > >> The additional endpoint sounds fine, especially since uploading the >> modules.yaml file will produce two types of content units ModuleMD >> andModuleM

Re: [Pulp-dev] SingleArtifactContentUploadSerializer and task's created_resources

2019-10-03 Thread Brian Bouterse
On Tue, Oct 1, 2019 at 5:37 PM Simon Baatz wrote: > Using the new SingleArtifactContentUploadSerializer I noticed two > things that I would like to get feedback on: > > 1. If a repo version is created (using the "repository" parameter), >"created_resources" of the respective task contains the

[Pulp-dev] SELinux for Pulp3

2019-10-04 Thread Brian Bouterse
tl;dr Pulp3 will have an SELinux policy at GA, and Katello will use it. In the future SELinux failures will require a functional test to demonstrate the failure before the SELinux error can be fixed. For Pulp3's use in Katello, SELinux is a requirement, so we need to provide SELinux policies (for

[Pulp-dev] Issue exporting Publications to POSIX Filesystems

2019-10-08 Thread Brian Bouterse
At open floor today, we talked about an issue with the exporting Publications with some relative_path data to POSIX filesystems. I'm wondering what you think should/could be done about this. https://pulp.plan.io/issues/5559 Thanks! Brian ___ Pulp-dev ma

[Pulp-dev] Installer Team Changes

2019-10-08 Thread Brian Bouterse
Recently the installer has had a lot of changes, and Mike DePaulo (@mikedep333) has been the most involved with it over the past few months. With all favorable support, the existing committers accept @mikedep333 as the lead for the Installer mini-team on Github. Additionally, we wanted to find out

Re: [Pulp-dev] Repo version validation

2019-10-08 Thread Brian Bouterse
On Tue, Oct 8, 2019 at 2:55 PM David Davis wrote: > I think @bmbouter's solution could handle the first example of checking > RPMs against a specific key. > > The second example is trickier though because the validation would have to > know which module is being removed in order to know which pac

Re: [Pulp-dev] Repo version validation

2019-10-09 Thread Brian Bouterse
on in some way is important > here - sync, copy, upload, removal of content. > It's just happened that for sync we already have a mechanism for plugins > to influence the result, however it can likely be simplified and reuse what > will be implemented for the story [0] under discussion. >

Re: [Pulp-dev] Repo version validation

2019-10-09 Thread Brian Bouterse
t 5:52 PM Brian Bouterse wrote: > >> >> >> On Wed, Oct 9, 2019 at 5:23 AM Tatiana Tereshchenko >> wrote: >> >>> I think the main confusion I have is that we call it validation. >>> Semantically, I'd expect the validation operation to compla

[Pulp-dev] Why do we have 'pluginname.app' and what if we only had 'pluginname'?

2019-10-10 Thread Brian Bouterse
I recently have been wondering why we have an 'app' folder. Not for 3.0, but maybe 3.1 we could adjust the plugin_template and pulpcore's plugin loader to load the plugin as 'pluginname' instead of 'pluginname.app'. Here are some strange things created by having the 'app' folder. * pulpcore tries

[Pulp-dev] 3.0 Blockers Moving Forward

2019-10-10 Thread Brian Bouterse
Here's a list of the issues @daviddavis and I have groomed along with some input from others. If anything on here does not look right, please let us know! I put these onto the Sprint 60. Core should not add/remove content to a repository or create a repository_version without plugin input - http

[Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-16 Thread Brian Bouterse
Having just released RC7, there are a variety of problems we are dealing with as a result of having pulpcore and pulpcore-plugin being in separate repos. @daviddavis and I were talking, and we want to ask for feedback on merging the code from pulpcore-plugin into pulpcore. I wrote this up as an is

Re: [Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-17 Thread Brian Bouterse
gt;> > >> >She / Her / Hers >> > >> >Software Engineer, Pulp Project >> > >> >[3]Red Hat >> > >> >[4]dawal...@redhat.com >> >[5][Logo-RedHat-Email.png] >> > >> > On Wed, Oct

Re: [Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-17 Thread Brian Bouterse
s change in RC8? > > David > > > On Thu, Oct 17, 2019 at 10:49 AM Mike DePaulo > wrote: > >> On Thu, Oct 17, 2019 at 10:25 AM Brian Bouterse >> wrote: >> >>> I put some responses inline. I'm interested in what you think. >>> >>&g

Re: [Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-17 Thread Brian Bouterse
ome to think it over at least, call for more feedback. I was hoping to keep discussion on the list though, even with us meeting in person. The list I think gives inclusivity and record to those who cannot join. We can remove from sprint now? > Tanya > > > On Thu, Oct 17, 2019

[Pulp-dev] [breaking] Redeploy your Environment

2019-10-17 Thread Brian Bouterse
With 51395 pulpcore no longer has a hard-coded settings file, but the installer maintains this functionality by keeping it's settings at /etc/pulp/settings.py. This was part of https://pulp.plan.io/issues/5560 You'l

Re: [Pulp-dev] [breaking] Redeploy your Environment

2019-10-18 Thread Brian Bouterse
t;> Got this solution from here: >> https://www.techchorus.net/microblog/vagrant-libvirt-issue-after-upgrading-to-fedora-30/ >> >> Best regards, >> Fabricio Aguiar >> Software Engineer, Pulp Project >> Red Hat Brazil - Latam <https://www.redhat.com/> >>

Re: [Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-23 Thread Brian Bouterse
there is no path and leave a trail." >> >> >> On Thu, Oct 17, 2019 at 7:25 PM David Davis >> wrote: >> >>> I agree that it makes sense to talk about it next week. I have a few >>> concerns both in favor and against merging the repos that are not

Re: [Pulp-dev] Merging pulpcore.plugin into pulp/pulpcore repo?

2019-10-28 Thread Brian Bouterse
on it in their source trees. On Mon, Oct 28, 2019 at 4:03 PM Daniel Alley wrote: > This will hopefully be merged today across all plugins: > https://pulp.plan.io/issues/5580#note-8 > > On Wed, Oct 23, 2019 at 5:19 AM Brian Bouterse > wrote: > >> Thanks @dalley! >>

[Pulp-dev] CONTENT_HOST woes -- need input

2019-10-29 Thread Brian Bouterse
A Distribution, e.g. FileDistribution has a base_url (not base_path) which defaults to returning data with "relative" urls, e.g. "/pulp/content/foo/..." If you set the CONTENT_HOST

Re: [Pulp-dev] CONTENT_HOST woes -- need input

2019-10-29 Thread Brian Bouterse
On Tue, Oct 29, 2019 at 11:19 AM Brian Bouterse wrote: > A Distribution, e.g. FileDistribution > <https://docs.pulpproject.org/en/3.0/nightly/restapi.html#operation/distributions_file_file_read> > has a base_url (not base_path) which defaults to returning data with > "rela

Re: [Pulp-dev] CONTENT_HOST woes -- need input

2019-10-29 Thread Brian Bouterse
On Tue, Oct 29, 2019 at 11:37 AM Brian Bouterse wrote: > > > On Tue, Oct 29, 2019 at 11:19 AM Brian Bouterse > wrote: > >> A Distribution, e.g. FileDistribution >> <https://docs.pulpproject.org/en/3.0/nightly/restapi.html#operation/distributions_file_file_read>

[Pulp-dev] Role Based Access Control for Pulp 3.y -- Use Cases

2019-10-29 Thread Brian Bouterse
I'd like to host a chat meeting to gather use cases for Role Based Access Control needs in Pulp 3.y. The meeting details and agenda link are below. Both users and plugin developers are invited. Please join if you're interested! When: Wednesday, November 20⋅9:00 – 10:00am EST or in your time zone <

Re: [Pulp-dev] Repo versions with no changes

2019-11-04 Thread Brian Bouterse
tl;dr I'm +1 to making this switch. On Mon, Nov 4, 2019 at 3:51 PM David Davis wrote: > Currently in pulp, syncs always create repository versions regardless of > whether or not any content changed. One of the tasks[0] for 3.0 GA is to > document this behavior. However, I've heard several compla

[Pulp-dev] Solving the "callback problem" ... aka how pulpcore will stop finalizing RepositoryVersion

2019-11-05 Thread Brian Bouterse
As a followup to the chat discussion from triage/open-floor today, here is the POC on top of typed repositories. It's actually a very small change, the *only* significant difference is that the stages API no longer uses the RepositoryVersion context manager. Thus, the plugin writer must finalize it

Re: [Pulp-dev] Solving the "callback problem" ... aka how pulpcore will stop finalizing RepositoryVersion

2019-11-05 Thread Brian Bouterse
On Tue, Nov 5, 2019 at 3:53 PM Simon Baatz wrote: > Hi Brian, > > On Tue, Nov 05, 2019 at 02:46:18PM -0500, Brian Bouterse wrote: > >... > >Note that the context manager is only syntactic sugar. The pulp_file > >sync code could also just as easily

Re: [Pulp-dev] Solving the "callback problem" ... aka how pulpcore will stop finalizing RepositoryVersion

2019-11-05 Thread Brian Bouterse
ndling, let's plan that the actual implementation all over would be using the style like this gist had: https://github.com/dralley/pulp_file/compare/typed-repositories...bmbouter:plugin-finalize-no-context-manager > On Tue, 5 Nov 2019 14:46:18 -0500 > Brian Bouterse wrote: > > &

Re: [Pulp-dev] [Breaking Change] Typed Repositories

2019-11-06 Thread Brian Bouterse
After much, great work from @dalley on the typed repository prototype, unfortunately, we believe it cannot be adopted at this time. Here's a writeup of why: https://pulp.plan.io/issues/5625#note-8 and another writeup of alternatives considered: https://pulp.plan.io/issues/5625#note-9 This writeup

Re: [Pulp-dev] [Breaking Change] Typed Repositories

2019-11-09 Thread Brian Bouterse
55f25a8c4fab0aebR103 # Next Steps Another reply to this thread will be sent once this is fully merged. Please share any concerns, corrections, questions, or ideas! Thank you to everyone for the excellent collaboration and contribution. Pulp's users and its API is significantly better f

[Pulp-dev] pulpcore branching expected tomorrow Nov 13th

2019-11-12 Thread Brian Bouterse
Today was our target branch date for pulpcore. We are delaying branching pulpcore for one day, waiting on the following PRs to merge. Once branched I'll send an update tomorrow and then prepare the RC8 release. #pulpcore PRs: https://github.com/pulp/pulpcore/pull/365 https://github.com/pulp/pulpco

[Pulp-dev] Removing `spawned_tasks` and `parent` fields from Task Status API?

2019-11-13 Thread Brian Bouterse
Very early on in Pulp3 we made the Task API have this "tasks can dispatch tasks" machinery. Is anyone spawning tasks from inside tasks currently? I wrote up https://pulp.plan.io/issues/5710 to remove it before 3.0 GA. ___ Pulp-dev mailing list Pulp-dev@r

Re: [Pulp-dev] pulpcore branching expected tomorrow Nov 13th

2019-11-13 Thread Brian Bouterse
h could arguably be a bugfix so this seems ok. Thanks to everyone working so hard to make this possible. Next up plugin branching! On Tue, Nov 12, 2019 at 4:47 PM Brian Bouterse wrote: > Today was our target branch date for pulpcore. We are delaying branching > pulpcore for one day, waiting

[Pulp-dev] 3.0.0rc8 is available

2019-11-14 Thread Brian Bouterse
I'm announcing only on pulp-dev since plugins will need to release compatibility updates. pulp_file hopefully will release today, if I can get it done before my PTO. When it does release that announcement should go to pulp-list. See the new combined user-facing and plugin-writer-facing changelog h

Re: [Pulp-dev] pulpcore branching expected tomorrow Nov 13th

2019-11-14 Thread Brian Bouterse
I've renamed the pulp_file branch to 0.1 to reflect its actual X.Y number. https://github.com/pulp/pulp_file/tree/0.1 On Wed, Nov 13, 2019 at 4:21 PM Brian Bouterse wrote: > I've branched pulpcore and pulp_file creating the 3.0 branches below. I've > modified the branc

[Pulp-dev] Plugin Release Issues -- needs assistance

2019-11-15 Thread Brian Bouterse
I've been making PRs to release pulp_file, and what I realized (after @dkliban told me) was that we have a circular release dependency. We can't release pulp-certguard (see the failed build [0]) without pulp_file being released, and we can't release pulp_file without pulp-cerguard being released [1

Re: [Pulp-dev] Cherry pick process

2019-11-21 Thread Brian Bouterse
After cherry picking a whole bunch today, I am very in favor of automation to do it. I had hoped we could avoid writing/maintaining a service, but I looked around, and I don't see a hosted "cherry-pick bot" like dependabot for example. Can we set it up to run next to pulpbot? Or maybe we run it as

Re: [Pulp-dev] Cherry pick process

2019-11-22 Thread Brian Bouterse
ry-picking. I like that depending on the label set, > commit will be cherry-picked or not. > > > > Regards, > > Ina Panova > Senior Software Engineer| Pulp| Red Hat Inc. > > "Do not go where the path may lead, > go instead where there is no path and

Re: [Pulp-dev] Cherry pick process

2019-11-25 Thread Brian Bouterse
y merge any > failed cherry picks if the job fails. > +1 to Travis failing the job, leaving the labels unchanged, and having a human look at it > > David > > > On Fri, Nov 22, 2019 at 10:35 AM Brian Bouterse > wrote: > >> I was thinking we could make this happen

Re: [Pulp-dev] Cherry pick process

2019-11-26 Thread Brian Bouterse
t rely >> on human checking it every time? >> >> Tanya >> >> On Mon, Nov 25, 2019 at 6:14 PM David Davis >> wrote: >> >>> I opened an issue to outline the design we're discussing: >>> >>> https://pulp.plan.io/issues/5795 >>

[Pulp-dev] Pulp 3.0.0 GA Date -- Dec 13

2019-11-26 Thread Brian Bouterse
We're planning one final release candidate 3.0.0rc9 for Dec 3rd, and a generally available release with improved documentation on Dec 12th. This will include pulp_rpm, pulp_container, and pulp_file as well. This is a delay from the initial GA release date of Dec 3rd. Read the blog post for more

Re: [Pulp-dev] Pulp 3.0.0 GA Date -- Dec 12

2019-11-26 Thread Brian Bouterse
title correction: Dec 12th On Tue, Nov 26, 2019 at 11:08 AM Brian Bouterse wrote: > > > We're planning one final release candidate 3.0.0rc9 for Dec 3rd, and a > generally available release with improved documentation on Dec 12th. This > will include pulp_rpm, pulp_contain

Re: [Pulp-dev] Required fields for models at the DB level

2019-12-02 Thread Brian Bouterse
Thank you for bumping this thread. The problem you illustrate with UpdateRecord makes sense and is problematic. Django doesn't have Model.save() call full_clean() by default; they document that here ( https://docs.djangoproject.com/en/dev/ref/models/instances/?from=olddocs#django.db.models.Model.f

[Pulp-dev] [breaking] resource-manager slight naming adjustment

2019-12-02 Thread Brian Bouterse
The resource manager now requires the worker to be launched with the `-n resource-manager` name. Previously it accepted `-n resource-manager*`. This makes Pulp more HA because now resource managers can be started safely. You can edit the systemd file on an existing system, which lives at: /lib/sys

Re: [Pulp-dev] Required fields for models at the DB level

2019-12-02 Thread Brian Bouterse
> to go stricter and later loosen validation if needed, than the other way > around. > Agreed > > What do you think? > > 1. Should we add validation for all the models or just selectively? > Let's try to add on all 2. Before or after GA? > Let's try for before. >

Re: [Pulp-dev] Required fields for models at the DB level

2019-12-02 Thread Brian Bouterse
rking under the assumption that it's easier from the DB perspective >> to go stricter and later loosen validation if needed, than the other way >> around. >> >> What do you think? >> >> 1. Should we add validation for all the models or just selectively? >&

[Pulp-dev] [breaking] Model renamed to BaseModel

2019-12-03 Thread Brian Bouterse
As part of the Plugin API final audit, we realized having Pulp provide an object called `Model` was confusing versus the Django object `Model`. We've renamed pulpcore.plugin.models.Model to pulpcore.plugin.models.BaseModel. The following PRs have been merged: pulpcore: https://github.com/pulp/pulp

Re: [Pulp-dev] Required fields for models at the DB level

2019-12-03 Thread Brian Bouterse
6:52 PM Simon Baatz wrote: > On Mon, Dec 02, 2019 at 03:53:06PM -0500, Brian Bouterse wrote: > >If anyone has concerns with us enabling Model validation by default on > >all models please let us know soon. > > I don't know (yet) if I have concerns, but DRF see

[Pulp-dev] pulpcore release freeze and release timeline details

2019-12-05 Thread Brian Bouterse
Releasing takes time, and we need pulpcore to be available before the releasing of plugins can start. Here's a proposed timeline to allow that to happen. Dec 10 - 22:00 all code merged to 'master' Dec 11 - pushing pulpcore 3.0.0 GA to pypi, then pulp_file 0.1.0 to pypi. Announce only to pulp-dev (

Re: [Pulp-dev] pulpcore release freeze and release timeline details

2019-12-06 Thread Brian Bouterse
t; goal for pulp_rpm. > > Tanya > > On Fri, Dec 6, 2019 at 1:54 AM Ina Panova wrote: > >> Brian, >> Thank you for the provided details. >> >> Ack, on releasing pulp_container on Dec 12. >> >> On Thu, 5 Dec 2019, 18:52 Brian Bouterse, wrote: >

[Pulp-dev] Removing ProgressReport auto-throttling?

2019-12-09 Thread Brian Bouterse
With the recent performance push, it's become clear to me that plugin writers probably need to think in batches as a first class concept. If they do then the auto-throttling in pulpcore seems out of place. From a high level, I'm concerned we're accidentally ending up with multiple ways of solving t

[Pulp-dev] 3.1 Milestone for pulpcore on pulp.plan.io

2019-12-09 Thread Brian Bouterse
A pulpcore 3.1 milestone is now available, which created this (mostly empty) roadmap: https://pulp.plan.io/versions/73 @jsherrill I added the items you identified earlier for Katello. If there is something you want to do or have done for pulpcore 3.1 please add to the roadmap or bring it up via

Re: [Pulp-dev] pulpcore release freeze and release timeline details

2019-12-11 Thread Brian Bouterse
[1]: https://docs.pulpproject.org/en/3.0.0/ [2]: https://rubygems.org/gems/pulpcore_client/versions/3.0.0 [3]: https://pypi.org/project/pulpcore-client/3.0.0/ On Thu, Dec 5, 2019 at 12:51 PM Brian Bouterse wrote: > Releasing takes time, and we need pulpcore to be available before the &

Re: [Pulp-dev] pulpcore release freeze and release timeline details

2019-12-12 Thread Brian Bouterse
pulp_file is released. https://pypi.org/project/pulp-file/ https://pulp-file.readthedocs.io/en/0.1.0/ https://pypi.org/project/pulp-file-client/0.1.0/ https://rubygems.org/gems/pulp_file_client/versions/0.1.0 On Wed, Dec 11, 2019 at 6:45 PM Brian Bouterse wrote: > The 3.0.0 release

Re: [Pulp-dev] pulpcore release freeze and release timeline details

2019-12-12 Thread Brian Bouterse
ient/versions/1.0.0/ >> >> >> Regards, >> >> Ina Panova >> Senior Software Engineer| Pulp| Red Hat Inc. >> >> "Do not go where the path may lead, >> go instead where there is no path and leave a trail." >> &

Re: [Pulp-dev] pulpcore release freeze and release timeline details

2019-12-13 Thread Brian Bouterse
in redmine, should we > also set a version which features/bugfixes were released in? > Target release? Platform release? > > Tanya > > On Thu, Dec 12, 2019 at 12:46 AM Brian Bouterse > wrote: > >> The 3.0.0 release is available on PyPI [0], and docs are available [1] >

[Pulp-dev] Bindings Testing Epic

2019-12-16 Thread Brian Bouterse
I believe we have an opportunity to improve Pulp's testing through the use of bindings tests. I wrote up an epic to gather ideas and discussion. https://pulp.plan.io/issues/5888 Initially this would be for pulpcore and pulp_file, but the idea is we would enable this automatically via the plugin_te

[Pulp-dev] GA plugins, update your setup.py classifiers

2019-12-18 Thread Brian Bouterse
The pulpcore and pulp_file packages on PyPI are showing as pre-release because of the classifiers we are using. If your plugin is GA, you probably want a change like this one: https://github.com/pulp/pulpcore/pull/459 It uses classifiers from here: https://pypi.org/classifiers/ Thanks to @Zhene

Re: [Pulp-dev] RPM mirror repository

2019-12-20 Thread Brian Bouterse
I haven't looked very closely yet, but I see that the PR [1] has a conditional check in `finalize_new_version` here: https://github.com/pulp/pulp_rpm/pull/1568/files#diff-886596b2b9cc3257ce61648f8aab8ebfR124 When finalize_new_version was added, an issue came up similar to yours: "where should we p

[Pulp-dev] pulpcore 3.0.1 -- dev freeze Jan 13

2020-01-06 Thread Brian Bouterse
I propose we release pulpcore 3.0.1 on Jan 14th. To do that, please have any bugfixes for inclusion merged to master by end of business Jan 13th. There will be no backwards incompatible changes for users or plugin writers, so plugin releases won't be necessary. Plugins should release z-stream fixe

[Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-07 Thread Brian Bouterse
We had two bugs filed recently [0][1] which suggest that when using the default backend for Pulp, i.e. pulpcore.app.models.storage.FileSystem Pulp should not be "moving" files. This is the default behavior Django gives us, and it destroys data when sync'ed from file:/// for example [1]. I propose

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-08 Thread Brian Bouterse
On Wed, Jan 8, 2020 at 12:02 PM Simon Baatz wrote: > On Tue, Jan 07, 2020 at 04:45:54PM -0500, Brian Bouterse wrote: > >We had two bugs filed recently [0][1] which suggest that when using > the > >default backend for Pulp, i.e. pulpcore.app.models.storage.FileSystem

Re: [Pulp-dev] CI/CD meeting notes

2020-01-08 Thread Brian Bouterse
On Wed, Jan 8, 2020 at 4:05 PM David Davis wrote: > *January 8, 2020* > > >- > >https://github.com/pulp/plugin_template/pull/165 >- > > Looks good, mikedep333 to approve > - > >https://github.com/pulp/plugin_template/pull/164 >- > > Speed improves or worsens?

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-08 Thread Brian Bouterse
a bug where some cases may not cleanup the worker directory which would now leave those files around forever. I will resume on those things tomorrow. On Wed, Jan 8, 2020 at 12:32 PM Brian Bouterse wrote: > > > On Wed, Jan 8, 2020 at 12:02 PM Simon Baatz wrote: > >> On Tue, Jan

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-10 Thread Brian Bouterse
way django says to customize our storage system. On Wed, Jan 8, 2020 at 6:10 PM Brian Bouterse wrote: > It's not finished, but I have a POC with tests passing locally for me > here: https://github.com/pulp/pulpcore/pull/486 It does use the reflink > library and falls back to copy.

[Pulp-dev] 3.1 GA plan -- Jan 30th

2020-01-10 Thread Brian Bouterse
David and I are coordinating the 3.1 pulpcore release. We are proposing we release 3.1 on Jan 30th, and have it be a time-based release. Tentatively, we hope to release about a pulpcore y-release every month for the foreseeable feature. It's also worth noting that 3.1 could bring breaking changes t

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-10 Thread Brian Bouterse
I was able to get this working, and with a double ack I merged it. Further refinements are welcome. https://github.com/pulp/pulpcore/pull/486 On Fri, Jan 10, 2020 at 9:56 AM Brian Bouterse wrote: > Here's a summary of what I've learned about our options to resolve the two > iss

Re: [Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

2020-01-13 Thread Brian Bouterse
On Sun, Jan 12, 2020 at 3:47 PM Simon Baatz wrote: > On Tue, Jan 07, 2020 at 04:45:54PM -0500, Brian Bouterse wrote: > >We had two bugs filed recently [0][1] which suggest that when using > the > >default backend for Pulp, i.e. pulpcore.app.models.storage.FileSystem

Re: [Pulp-dev] 3.1 GA plan -- Jan 30th

2020-01-14 Thread Brian Bouterse
> > David > > > On Sun, Jan 12, 2020 at 3:16 PM Simon Baatz wrote: > >> On Fri, Jan 10, 2020 at 12:10:34PM -0500, Brian Bouterse wrote: >> >David and I are coordinating the 3.1 pulpcore release. We are >> proposing >> >we release 3.1 on Ja

Re: [Pulp-dev] pulpcore 3.0.1 -- dev freeze Jan 13

2020-01-14 Thread Brian Bouterse
We had some issues with the final round of cherry picks today for 3.0.1. @daviddavis and I are resolving it. We should be able to release and announce tomorrow after we resolve this issue. On Mon, Jan 6, 2020 at 11:32 AM Brian Bouterse wrote: > I propose we release pulpcore 3.0.1 on Jan 1

Re: [Pulp-dev] pulpcore 3.0.1 -- dev freeze Jan 13

2020-01-15 Thread Brian Bouterse
daviddavis, @mikedep333, @dkliban, and others (who I may have missed thanking) for helping resolve the various Travis/build challenges that were worked through today. [0]: https://pypi.org/project/pulpcore/ [1]: https://travis-ci.com/pulp/pulpcore/builds/144642873 On Tue, Jan 14, 2020 at 5:03 PM Bria

[Pulp-dev] Re-downloading Corrupted Content (solving bit rot)

2020-01-16 Thread Brian Bouterse
A feature is being discussed on re-downloading corrupted content. There are some questions about how to position the API, the locking, and how it interacts with repo versions that allow 404 errors to still complete a repo version. Questions posted here: https://pulp.plan.io/issues/5613#note-3 Plea

Re: [Pulp-dev] Pulp3 Applicability Design thoughts (and Katello)

2020-01-17 Thread Brian Bouterse
tl;dr: at least for the initial Katello and Pulp3 integration, moving applicability to Katello I think is the best outcome for both communities. What I'm reading (correct me if not accurate) is that Katello's calls to Pulp for applicability calculation need to flow so much data that Pulp's value t

Re: [Pulp-dev] Hanging tasks (reloaded)

2020-01-20 Thread Brian Bouterse
On Fri, Jan 17, 2020 at 4:15 AM Matthias Dellweg wrote: > Hello all, > I believe i have found a new incarnation of hanging tasks (tm). This > time it is pulp3 and as hard to nail down as ever. > I think, it is introduced by e36e7b5f0eccc176a6e6298df29293b014f4710c. > Can you link me to this comm

[Pulp-dev] SELinux repository renamed to pulp/pulpcore-selinux

2020-01-21 Thread Brian Bouterse
A suggestion was given [0] to have the SELinux repo renamed from pulp/pulp-selinux to pulp/pulpcore-selinux. That was done just now. The SELinux policy for Pulp3 now lives here: https://github.com/pulp/pulpcore-selinux Suggestions or concerns are welcome. [0]: https://pulp.plan.io/issues/5994 T

Re: [Pulp-dev] API for repairing bitrot -- input requested

2020-01-22 Thread Brian Bouterse
FYI, this is for issue: https://pulp.plan.io/issues/5613 On Wed, Jan 22, 2020 at 10:24 AM Brian Bouterse wrote: > A "repair" feature is being planned that will check-and-re-download all > Artifacts associated with a RepositoryVersion. The last thing for us to > work out is

[Pulp-dev] API for repairing bitrot -- input requested

2020-01-22 Thread Brian Bouterse
A "repair" feature is being planned that will check-and-re-download all Artifacts associated with a RepositoryVersion. The last thing for us to work out is how the API will be offered to users. Initially, the feature will launch to fix Artifacts for the 'latest' repository version for a given Repo

[Pulp-dev] Introducing Plugin Snippets for custom Plugin URLs

2020-01-27 Thread Brian Bouterse
We're having a few challenges with the custom webserver routes. These are the nginx and apache configured routes in addition to /pulp/api/v3/ or /pulp/content/. I wrote up the problem descriptions on this epic: https://pulp.plan.io/issues/6057 This would involve plugins that want additional route

Re: [Pulp-dev] Bindings - Installer and functional tests

2020-01-28 Thread Brian Bouterse
I agree! Thank you! On Tue, Jan 28, 2020 at 10:19 AM Daniel Alley wrote: > Excellent work, Fabricio! This is a big deal :) > > On Tue, Jan 28, 2020 at 10:06 AM Fabricio Aguiar < > fabricio.agu...@redhat.com> wrote: > >> >> With #6020 *plugin_template* >> funct

[Pulp-dev] Markdown conversion on pulp.plan.io

2020-01-29 Thread Brian Bouterse
plan.io (who hosts pulp.plan.io) is switching to markdown. See their blogpost here [0]. They are offering a "conversion" service to convert existing data to markdown. I propose we schedule a time to have pulp.plan.io switch to using markdown on pulp.plan.io. What do you think? [0]: https://suppor

Re: [Pulp-dev] Not equal filters

2020-01-29 Thread Brian Bouterse
On Wed, Jan 29, 2020 at 10:03 AM David Davis wrote: > A few weeks ago, Katello opened an issue[0] requesting a set of "not > equal" filters (ie filters where a field is not equal to a certain value). > I created a pulpcore issue[0] to investigate whether pulpcore could provide > this functionalit

Re: [Pulp-dev] Markdown conversion on pulp.plan.io

2020-01-30 Thread Brian Bouterse
gt;> > David >> > >> > >> > On Wed, Jan 29, 2020 at 9:46 AM Daniel Alley wrote: >> >> >> >> I vote "as soon as possible" :) Definitely a welcome change! >> >> >> >> On Wed, Jan 29, 2020 at 9:40 AM

[Pulp-dev] 3.1 Release Update

2020-01-30 Thread Brian Bouterse
We're waiting on Travis to finalize the release [0], and the once tagged to retest and publish the build. I'll resume in the morning, and I'll email this thread in addition to the pulp-list announcement. https://github.com/pulp/pulpcore/pull/525 -Brian

Re: [Pulp-dev] Markdown conversion on pulp.plan.io

2020-01-31 Thread Brian Bouterse
They scheduled us for tomorrow, Saturday, between 1 AM and 5 AM UTC. The actual downtime will be a few minutes, but that's the window. I'll relay any confirmation I receive to the list. On Thu, Jan 30, 2020 at 5:38 AM Brian Bouterse wrote: > I sent them a request for pulp

<    3   4   5   6   7   8   9   10   >