Re: [Pulp-list] Fwd: [pulp-internal] Pulp sync with Fedora34 Update repo

[Robin Chan]

Hello Chung,
I've made sure you are added to the mailing list and I've included the
reply below as well.

You can also find us at: https://app.element.io/#/room/#pulp:matrix.org
and  https://discourse.pulpproject.org/

-Robin

Robin Chan

She/Her/Hers

Senior Software Engineering Manager - Pulp

Asian Network Co-Chair

Red Hat <https://www.redhat.com>

IRC: rchan

Red Hat respects your work life balance. Therefore there is no need to
answer this email out of your office hours.
<https://www.redhat.com> [image:
https://source.redhat.com/communitiesatredhat/diversity_and_inclusion/asian_network]
<https://source.redhat.com/communitiesatredhat/diversity_and_inclusion/asian_network>



On Thu, Mar 10, 2022 at 8:13 PM Daniel Alley  wrote:

> Yeah this is strange.  That's an escape character, which is an ascii
> control character, and as far as I can tell ascii control characters aren't
> allowed in XML 1.0.  Which is what the header of the file declares.
>
> Thanks for the report, I will try to deal with it upstream.  Potentially
> multiple upstreams (Bodhi, createrepo_c)
>
> -- Forwarded message -
> From: Chung Chung 
> Date: Thu, Mar 10, 2022 at 6:58 PM
> Subject: Fwd: [pulp-internal] Pulp sync with Fedora34 Update repo
> To: Daniel Alley 
>
>
>
> I will subscribe to the list, but do you mind forwarding it?  Thanks!
>
> Chung
>
> -- Forwarded message -
> From: 
> Date: Thu, Mar 10, 2022 at 6:56 PM
> Subject: Re: [pulp-internal] Pulp sync with Fedora34 Update repo
> To: 
>
>
> Your message has been rejected, probably because you are not
> subscribed to the mailing list and the list's policy is to prohibit
> non-members from posting to it.  If you think that your messages are
> being rejected in error, contact the mailing list owner at
> pulp-list-ow...@redhat.com.
>
>
>
>
> -- Forwarded message --
> From: Chung Chung 
> To: Daniel Alley 
> Cc: pulp-list 
> Bcc:
> Date: Thu, 10 Mar 2022 18:55:47 -0500
> Subject: Re: [pulp-internal] Pulp sync with Fedora34 Update repo
> Here is the file, looks like the ESC is the problem?  Thanks for getting
> back to me so quickly.
>
> None
> ghc-http-directory-0.1.9-1.fc34 enhancement update
> ESC[A
> 
> 
>
> On Thu, Mar 10, 2022 at 6:42 PM Daniel Alley  wrote:
>
>> Switching to the correct list.
>>
>> Could you send me that file?  Thanks!
>>
>> On Thu, Mar 10, 2022 at 5:37 PM Chung Chung  wrote:
>>
>>>
>>> Hi all,
>>>
>>>   Sorry to bother you guys, I am just wondering if anyone reported
>>> having problems with pulp syncing with Fedora34 Update.  I am using Pulp in
>>> One Container.  I have been getting this error:
>>>
>>> .Error:
>>> Task /pulp/api/v3/tasks/7f920807-3ddf-4923-aceb-d770920e1850/ failed:
>>> 'Parse error '/var/lib/pulp/tmp/15271@c740d6e34c0a/tmpjd_12kcq/tmpsq08ffvz'
>>> at line: 120516 (PCDATA invalid Char value 27
>>> )'
>>>
>>>
>>> I am wondering if I hit a bug that is not able to parse some special
>>> characters.  I have the tmp file that is in question also.
>>>
>>> Thanks for your help.  If this is the wrong list, please let me know
>>> where.  Thanks!
>>>
>>> Chung
>>>
>> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://listman.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Configuration Recommendations for Distributed Enterprise Pulp Setup

[Robin Chan]

Hi Tim,

Apologies - I think your email may have missed our attention here. The
deployment plans and use case sounds extremely well suited for Pulp 3 and
pulp_squeezer. Thanks for providing a detailed email with links to what
resources you've found. That was helpful in understanding what you have
already seen/read.

Have you determined what storage you will be using? That might help guide
subsequent answers. Let's try to get you some answers - any updates since
this original email? There are likely a few more requirements on your setup
that we might need to know to provide some guidance here. This bump might
help get a few of the easier questions answered. My apologies again that
this email escaped our attention.

-Robin

On Tue, Nov 3, 2020 at 2:40 PM Tim Black  wrote:

> Hello pulp community! I'm investigating using pulp for my company's
> primary artifact management system, and am looking for some architectural
> and configuration recommendations, given our use case and needs:
>
>- use debian, python, container, and file content plugins
>- produce and consume private content
>-
>- use remotes to implement mirroring of "upstream" public debian,
>pypi, and docker repos for performance and stability reasons (probably
>using the excellent *on demand* feature, and filtering for our arches
>and distributions of interest)
>- replicate our initial pulp instance to new instances at multiple
>sites, implementing scheduled synchronization of all content
>-
>- our file content will dwarf the other plugin content, being counted
>in TB instead of GB
>
> What is your recommendation for configuring a new pulp instance to support
> quick crash recovery (backup/restore) and replication to other pulp
> instances at multiple sites, synchronizing all content with each other?
>
> Given that I am using ansible (pulp_installer) for all provisioning and
> configuration of my pulp instances (and perhaps also using pulp squeezer
>  to manage our repositories) what
> storage entities are required to backup/restore/replicate in a pulp
> instance?
>
> I've read about pulp architecture
>  and pulp storage
>  and
> also pulp deployment scenarios
> , but it's
> unclear to me whether backing up just the django-storages (local fs, sw,
> azure) is sufficient, or if the database also must be backed
> up/restored/synchronized. Please point me at any other documentation or
> discussions that would help shed light on how to achieve my goal of
> configuring multiple synchronized pulp instances that can be easily
> restored from backup.
>
> I understand that the pulp content plugins themselves implement
> synchronization with remotes, so perhaps the best solution is to configure
> a cluster of pulp instances using the same ansible playbooks, but defining
> one of them as the *primary*, and configure the others to use the
> *primary* as a remote? Are there techniques available to set up
> multi-directional sync, does it need to follow the *primary*/*secondary* 
> model?
> (i.e. can I also set up the primary to synchronize content from all of the
> *secondaries*, so that content added to a secondary will become present
> on the primary and all other secondaries?)
>
> It seems that pulp has been designed (and then redesigned) for my use
> case, but I'm having trouble putting together all the architectural and
> configuration pieces required to paint a complete picture of my goal.
> Thanks so much for your recommendations and your time!
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] [Pulp-dev] Pulp 3 CLI MVP

[Robin Chan]

Sorry to bring back this thread, but I am thinking through some operational
aspects and want to make sure that my assumptions are correct and won't
impact things later on (have been accounted for in the design).

1. Any documentation requirements? Since there are both core and plugin
commands. A user on a pulp instance would be able to find out what commands
are available to them (in other words not needing to know which plugins are
installed). Perhaps being able to ship/make available this same information
as an actor knowing which plugins I am shipping (or should be there).

2. I can see an operator wanting to update or patch the cli
without changing other pulp software - so I'm assuming this is a separate
deliverable or would this be just part of core/plugins? If they are
separate I guess a new cli may be released when needed but if not pulp
software can continue to be updated. My brain can't put together the words
to describe that dependency relationship, but hopefully that workflow is
clear.

I'm coming from a mindset that a minimal set of commands might be available
initially and that it will be easy to add new ones as the need becomes
clear and patch a system without causing concern that other things are
changed.  However I do see this might be a short term problem to solve.


On Mon, May 11, 2020 at 3:21 PM Brian Bouterse  wrote:

> I think having the commands namespaced by the plugin name would be an
> organized way to see what capabilities a given plugin is shipping. I
> imagine for pulpcore's commands they could be namespaced under 'core' or
> 'pulpcore'.
>
> Also +1 to 'pulp' command name.
>
> On Mon, May 11, 2020 at 6:42 AM David Davis  wrote:
>
>> In some places, the API in Pulp 3 is very different from Pulp 2 but where
>> it's possible and makes sense, I think we will want to do this. Perhaps
>> this is a good argument for having plugin name after the "pulp" command (eg
>> "pulp rpm ...", "pulp file ...").
>>
>> David
>>
>>
>> On Thu, May 7, 2020 at 8:47 AM Konstantin M. Khankin <
>> khankin.konstan...@gmail.com> wrote:
>>
>>> Is it an option to keep the Pulp 2 CLI syntax as much as possible?
>>>
>>> чт, 7 мая 2020 г. в 15:28, Dennis Kliban :
>>>
 On Thu, May 7, 2020 at 7:13 AM Tatiana Tereshchenko <
 ttere...@redhat.com> wrote:

> +1 to `pulp` command.
>
> I think for me as a user, the most logical would be to have a plugin
> name first and then follow the URL pattern.
> The majority of commands are plugin specific. If I work with multiple
> plugins, it also makes it easy for me as a user to just change the plugin
> name in front for the commands which have the same structure in different
> plugins.
> It also makes it visually clear that I work with a specific plugin, in
> comparison to when the plugin name is somewhere in the 3rd/4th place.
> It will also allow not to guess in commands like the `pulp
> repositories rpm rpm`  which one is the plugin name and which one is a 
> repo
> type.
>
> I agree that this would make much more clear to the user which 'rpm'
 is the plugin type and which 'rpm' is the resource type.


> +1 for
> pulp rpm content packages
> pulp rpm repositories rpm
> pulp rpm repositories mirror
> ...
>
> On Wed, May 6, 2020 at 7:58 PM Dennis Kliban 
> wrote:
>
>> On Wed, May 6, 2020 at 12:30 PM David Davis 
>> wrote:
>>
>>> Matthias and I met today to go over some plans for a prototype. I
>>> wrote some notes[0] down. As part of the prototype, we'd propose two
>>> deliverables (one this week and one next week):
>>>
>>> 1. A set of ~2-3 click commands that use the bindings to interact
>>> with Pulp
>>> 2. Some openapi-generator templates that will be able to generate
>>> such commands from the schema
>>>
>>> There is a question we had about how the commands for typed
>>> resources will be structured in the CLI. To illustrate with two 
>>> endpoints:
>>>
>>>
>> We should call the command 'pulp' instead of pulp-cli.
>>
>>
>>> # rpm.package content (/pulp/api/v3/content/rpm/packages/):
>>> - pulp-cli rpm content packages ...
>>> - pulp-cli content rpm packages ...
>>> - pulp-cli rpm packages content ...
>>> - ???
>>>
>>
>> I was thinking we should structure the commands similar to the URLs
>> in the REST API.
>>
>> pulp content rpm packages
>>
>>
>>>
>>> # file.file repositories (/pulp/api/v3/repositories/file/file/):
>>> - pulp-cli file repositories file ...
>>> - pulp-cli repositories file file ...
>>> - pulp-cli file file repositories ...
>>> - ???
>>>
>>> pulp repositories file
>>
>> Plugins that provide multiple types of the same resource would need
>> to be more descriptive. Though I can see a practical reason for requiring
>> all resources to be that descriptive.
>>

[Pulp-list] Pulp 3.0 Core RC Plan

[Robin Chan]

After our last update on the Pulp 3.0 Core RC [2], we have been working on
the list of open items for the Pulp 3.0 Core RC [1] and are closing in on
completion.  We felt it would be helpful to share our best estimates on
dates and an updated plan.

*Feature Freeze Target: March 6*

   - All items found in this query [1] merged


   - Any RC Blocker work coming out of the Upload discussion [2]


   - Beta release available on PyPI


   - Branch closed for merging


*2 Week Feature Freeze*

   - Critical bug fixes
   - Work on plugins compatible Pulp 3.0 Core RC


*Core RC Target: March 20 (**Feature** freeze + 2 week)*

   - Pulp 3.0 Core RC release


   - Pulp file plugin release


Feature Freeze will happen when all content [1] has been delivered. If that
date ends up being before or after March 6, the Core RC Target will be 2
weeks from the Feature Freeze date.

We expect new plugin releases shortly after or up to a few the weeks after
the Pulp 3.0 Core RC release. Please continue to follow the mailing list
and blog for announcements of plugin releases that will work with the new
Pulp 3 RC release.

Please bring up any concerns you have regarding content you do not see in
[1] that you believe should be. For more background information, please
also see our previous post [3].

[1] https://pulp.plan.io/issues?query_id=121
[2] https://www.redhat.com/archives/pulp-dev/2019-February/msg00052.html
[3] https://pulpproject.org/2019/01/30/pulp-3-rc-information/
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Mailman message delivery delay

[Robin Chan]

We noticed a delay in delivery of messages for both the pulp-dev,
pulp-list, and pulp-security mailman groups starting ~Aug 1, 2018. I
believe this issue has been resolved as the archives now match what is in
my Inbox and we will continue to monitor the archives and work with support
if this issue isn't resolved.

Apologies for any delays in our responses or confusion with late delivery
of messages. Please reach out to us on freenode #pulp or #pulp-dev if you
need to report future issues with the mail distribution list.
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Fwd: Pulp 3.0 Core Beta on 25-Apr-2018

[Robin Chan]

-- Forwarded message --
From: Robin Chan <rc...@redhat.com>
Date: Mon, Mar 12, 2018 at 1:26 PM
Subject: Pulp 3.0 Core Beta on 25-Apr-2018
To: Pulp-dev <pulp-...@redhat.com>


We have been busy working on the Pulp 3.0 Core Beta, the next step in our
roadmap to a Pulp 3 GA.

One of the Pulp 3 key objectives was to provide a true plugin API. Since
the September 2017 Pulp 3 Plugin API Alpha release, we have appreciated the
active reception and valuable feedback. The feedback we have received from
the community has been key in validating and informing some design
decisions shaping the Pulp core functionality.

We are working toward completing this core functionality and are excited to
announce our plans for the Pulp 3.0 Core Beta deliverable.  This email
describes the value proposition for Pulp 3.0 Core Beta, the feature set of
the deliverable, and when it will be delivered. We invite to you review
this milestone and provide feedback.

Value - What will this do for me?

Pulp 3.0 Core Beta will provide all major functionality required by most
users and a Plugin API to allow power users and plugin writers to start
planning the impact of moving to Pulp 3.0 from Pulp 2.0 including writing
plugins.

What is it?

We have loosely discussed and described most of the functionality in the
https://pulp.plan.io/projects/pulp/wiki/Pulp_3_Minimum_Viable_Product. The
Pulp 3.0 Core Beta will also commit to delivery of a few more items of
interest to our community that are described below.





















*Highlights of the Pulp 3.0 Core Beta Stack - Relational Database:
Compatible with PostgreSQL and SQLite- Scalable workers safely running
concurrent tasks- Compatible with Qpid Proton and RabbitMQ - Web servers:
Compatible with Apache, Nginx, and Django development server - Installation
via PyPI, source, or RPMFeatures: - REST API- Fetch (sync) content from
external sources- Upload content- Versioned Repositories- Fast Promotion
and RollbackPlugin API - Semantically versioned- Provides utilities:-
Concurrent downloads- Safely adding/removing units to repository versions
and publications- Tasking integration- Filesystem managementDocumentation -
Conceptual overview- Plugin Writer Docs- REST API documentationThe Pulp 3.0
core beta will be released alongside plugin betas: - pulp_file-
pulp_ansible- pulp_pythonRoadmaps for Pulp 3.0 beta compatible plugins: -
RPM- Docker- OSTreeThe Pulp 3.0 Core Beta will be available Wednesday April
25, 2018.-Robin Chan*
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Old python2-debpkgr package causing debian / ubuntu not to synchronize successfully

[Robin Chan]

So what should we do with the issue tracker that Sebastian wrote?
https://pulp.plan.io/issues/3311

On Fri, Feb 9, 2018 at 5:03 PM, Patrick Creech  wrote:

> On Fri, 2018-02-09 at 07:19 -0500, Patrick Creech wrote:
> > On Fri, 2018-02-09 at 10:30 +, Sebastian Sonne wrote:
> > > Hello everyone,
> > >
> > > ever since the stable release of 2.15, I’ve tried to synchronize
> debian, without any success. The error message given is "'md5sums' file not
> found, can't list MD5 sums“. In https://pulp.plan.io/is
> > > su
> > > es/3311, I have finally found out why: debpkg expects an md5sum, when
> apparently this is optional, causing everything else to fail. The fix for
> this is present in python-debpkgr version 1.0.2, but
> > > the only currently available version from pulp is 1.0.1.
> > >
> > > As I’m unsure where to turn to with the request to update
> python2-debpkgr, I’ll do it here.
> >
> > Thanks for reporting this Sebastion!
> >
> > As pulp's release engineer, I'll get right on updating python-debpkgr in
> our repos.  I'll send an update when the fix has been pushed
>
> The latest python2-debpkgr has been published to pulp's repos
> ___
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list