[Pulp-list] Pulp documentation error for Parent and Child node

[Bradley Davis]

I have been trying to get the Parent and Child node sync working but was
running into an issue where the sync would never complete forcing me to
cancel it. It wasn't until a cowker of mine erroneously had pulp_celerybeat
and pulp_resource_manager running on the Child that the Sync would
complete. The documentation in error is the Installation Chapter with the
last part of the Server section. It states this:

*3. There are two more services that need to be running, but it is
important that these two only run once each (i.e., do not enable either of
these on any more than one Pulp server). *

*Warning*

*pulp_celerybeat and pulp_resource_manager must both be singletons, so be
sure that you only enable each of these on one host if you are Pulp’s
clustered deployment.*



*On some Pulp system, configure, start and enable the Celerybeat process.
This process performs a job similar to a cron daemon for Pulp. Edit
/etc/default/pulp_celerybeat to your liking, and then enable and start it.
Again, do not enable this on more than one host. For Upstart:*

Content is found here:
https://pulp.readthedocs.org/en/latest/user-guide/installation.html#server

It makes it seems that pulp_resource_manager and pulp_celerybeat should not
be running on the child node server. When looking at the Nodes chapter,
neither services are mentioned.
Thanks,
Brad
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Pulp 2.6.4 beta available

[Dennis Kliban]

This beta contains a few bug fixes in platform and RPM plugin. The list can be 
found here [0-1]. The packages are available at their regular location [2]. 
Happy testing! 

 - Dennis


[0] 
https://pulp.plan.io/projects/pulp/issues?utf8=%E2%9C%93_filter=1%5B%5D=cf_4%5Bcf_4%5D=%3D%5Bcf_4%5D%5B%5D=2.6.4%5B%5D=%5B%5D=tracker%5B%5D=status%5B%5D=priority%5B%5D=cf_5%5B%5D=subject%5B%5D=author%5B%5D=assigned_to%5B%5D=cf_3_by=

[1] 
https://pulp.plan.io/projects/pulp_rpm/issues?utf8=%E2%9C%93_filter=1%5B%5D=cf_4%5Bcf_4%5D=%3D%5Bcf_4%5D%5B%5D=2.6.4%5B%5D=%5B%5D=tracker%5B%5D=status%5B%5D=priority%5B%5D=cf_5%5B%5D=subject%5B%5D=author%5B%5D=assigned_to%5B%5D=cf_3_by=

[2] https://repos.fedorapeople.org/repos/pulp/pulp/beta/2.6/

___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Re: [Pulp-list] Pulp 2.7.0 beta is available

[Webb, Reece]

After recreating all of my repositories, I ran into the offset and metadata 
errors again today. I can confirm the metadata error isn't generated due to 
problems with upstream; the repository I last saw it on was local and during 
publish.

I don't know if this helps, or if it's just coincidence, but the metadata 
errors have only (today) been on repositories where the feed has a cert. My 
other repos (local, CentOS, EPEL) had the offset error. I'm usually able to 
successfully sync after recreating the repository, but not always.

The new "pulp-admin tasks list" functionality was the main reason I upgraded, 
but I think I'm going to have to go back to 2.6.3 until things stabilize.

Reece



On 9/16/15, 2:56 PM, "Webb, Reece"  wrote:

>Thanks Randy, the CDN/metadata errors aren't showing up right now, so maybe it 
>was just a temporary fluke. I successfully synced right before the upgrade to 
>2.7, so I assumed the worse. :) In any case, I'll keep an eye on it for the 
>next couple of days to make sure the error doesn't return. And if there's any 
>other info I can provide to help get the email issue resolved, please let me 
>know.
>
>Reece
>
>
>
>
>On 9/16/15, 2:39 PM, "Randy Barlow"  wrote:
>
>>Webb, Reece wrote:
>>> Thanks, I upgraded from 2.6.3. As I said in my last follow-up email, 
>>> recreating a repository allow it to successfully sync/publish. But now I'm 
>>> running into two other issues... email notification events don't work at 
>>> all. I've also run into an error retrieving metadata on seemingly random 
>>> repositories that I've tried recreating to workaround the offset error. I 
>>> can recreate the repository, but I receive the same error. I found one bug 
>>> report with this error that turned out to be a selinux issue, but I have 
>>> selinux disabled... not good. Logs below.
>>
>>Hi Reece,
>>
>>Apologies, I misread your earlier e-mail and I thought it said you had
>>upgraded to 2.6.3, not from 2.6.3. I updated the bug report to reflect
>>that now.
>>
>>As for the e-mail error - would you mind filing a bug about that?
>>
>>The CDN error is more surprising. Perhaps it is possible that the CDN
>>really was missing that file for a bit? Can you confirm whether that
>>error is repeatable?
>>

___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

[Pulp-list] Synchronising Disconnected Pulp Servers

[ben . stanley]

Synchronising Disconnected Pulp Servers
===

Situation
-

I maintain two pulp servers:

pulp-A is connected to the internet, and downloads 583 repos.

pulp-B is disconnected from the rest of the world.

The goal is to transfer the 583 repos from pulp-A to pulp-B using only a
USB HDD.

Naive Solution
--

Initially, I tried to solve this problem using

pulp-admin rpm repo export run --repo-id=${REPO_ID} --export-dir=${DIR}

I constructed a script to run this command for all 583 repo-ids. This had
the following problems:
1) It wasn't going to finish copying inside a week
2) It was going to fill up more than the entire HDD with replicated binary
copies of every RPM package that is duplicated between separate
repositories.

Clearly the export command is a poor solution to this problem.

Workable Solution
-

After trying various other ideas and false starts, I have come up with the
following solution:
1) rsync the pulp internal representation to the USB HDD:
rsync --recursive --no-inc-recursive --links --hard-links --delete --times
--progress /var/lib/pulp ${EXPORT_DIR}/var_lib_pulp
The complete rsync takes about a day the first time.
2) All the rpms are symlinked to absolute paths starting with
/var/lib/pulp/content . This prevents the USB HDD from being used as a
repository itself. The symlinks must be re-written from absolute paths to
relative paths.
This process takes 4 days to complete with a bash script, perhaps less
with a custom C program.
Furthermore, on subsequent synchronisations (for updates), the rsync will
convert all the relative symlinks back to absolute symlinks, so that the
symlink conversion process must be repeated from scratch every time. This
is a big waste of time.

After performing the two steps above, the USB HDD becomes usable as a
bunch of repositories in its own right. It can also be used as the feed to
update the disconnected machine pulp-B. I have written some scripts to
achieve this.

Improving the Solution by Patching Pulp
---
Now, the critical observation is this:
If pulp stored its internal symlinks as relative paths instead of absolute
paths, or perhaps even as hardlinks, then the second step of converting
absolute symlinks to relative symlinks by bash script would be
un-necessary, saving much time in the synchronisation process.

Is there any objection to submitting a patch to amend pulp to use relative
symlinks instead of absolute symlinks internally?

Perhaps it would be better to use hardlinks? This implies that
/var/lib/pulp must be stored entirely within the same filesystem. I don't
see this as a big problem, but others might.

Ideal Solution
--

I agree that the method of disconnected synchronisation outlined above is
not ideal, but it is the only thing I have that works. Feel free to
propose a more comprehensive solution for later, but I need something that
works now.

Ben Stanley.


___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list