Hi Dennis, Sadly the RHEL 8 beta yum metadata appears to be in good order as a properly extensioned gz file with compressed data. I went ahead and ran a quick tree of the /var/lib/pulp/content/units/yum_repo_metadata_file/ path(*) to show how there's an odd mix of gzip and non-gzip metadata. If you have any ideas on how to inject a debug option that prints the file it's working on during the migration, I'd love to shim that in. I started poking around looking for the failed unit code in /usr/lib/python2.7/site-packages/pulp/server/db/migrations but came up empty.
Please also feel free to ping me on #pulp (I just hopped on) if you have any questions that you wanted less lag on response. Sadly I have a 14:00-15:00 but will be back after that. (*) https://paste.fedoraproject.org/paste/QEa8YPiB5O3mMGot2wo-gQ Thanks! - Kodiak On Mon, Apr 1, 2019 at 1:31 PM Dennis Kliban <[email protected]> wrote: > I think that RHEL beta repos just had a 'modules' file or 'modules.yaml' . > If you find that file, compress it and try running the migration again. > Please reply on list with your findings. > > On Mon, Apr 1, 2019 at 12:54 PM Kodiak Firesmith <[email protected]> > wrote: > >> Thanks very much for this lead. This is probably something that will be >> good for everyone to know about as RHEL 8 beta includes modular >> repositories. In my case, the only modular yum metadata was RHEL 8 and was >> in fact a gzip file (so sayeth the 'file' command and from head $file it >> appeared to be binary). >> >> I went ahead and traversed >> /var/lib/pulp/content/units/yum_repo_metadata_file -type f -name *.gz, fed >> them into the file command, and they all came back as compressed files. >> >> Should I run 'find' in any other paths by chance? Any other .gz files >> seem to be symlinks back to the file in /var/lib/pulp/content/units :( >> >> Thanks! >> - Kodiak >> >> On Mon, Apr 1, 2019 at 12:35 PM Dennis Kliban <[email protected]> wrote: >> >>> I did not provide enough information. This is only happening to your >>> Fedora 28+ modular repositories. The file that you are looking for is >>> supposed to be called modules.yaml.gz. It will have a hash prepended to it. >>> So the name will look more similar to >>> 256554a6d4e29b3ab128655ad4e72932765cc7e2da3192199efd11ae2cb7cf7a-modules.yaml.gz. >>> You want to find all these files and make sure that they are actually >>> gzipped. >>> >>> On Mon, Apr 1, 2019 at 12:01 PM Kodiak Firesmith <[email protected]> >>> wrote: >>> >>>> Thanks Dennis, >>>> I'm not sure I follow :( >>>> I'm guessing since there are a hundred YUM repos being served, there >>>> are YUM metadata files scattered all over the various symlink trees. Do >>>> you know which files I should be looking for, and if it's safe to just gzip >>>> them all in place? >>>> Apologies if this is basic stuff. I'm moving my priorities around to >>>> do a better job of twinning our beta and prod environment because it sounds >>>> like the bug is hitting something that might be out of the norm due to one >>>> of the repos that I'm syncing on prod but not on beta. >>>> >>>> - Kodiak >>>> >>>> On Mon, Apr 1, 2019 at 11:57 AM Dennis Kliban <[email protected]> >>>> wrote: >>>> >>>>> I was investigating a somewhat related problem and discovered that >>>>> when Pulp is downloading files that was encoded using gzip, Pulp decodes >>>>> the file and writes it to disk in decoded form. >>>>> >>>>> You can search for all the yum metadata files and make sure that they >>>>> are gzipped. Then try migrating again. >>>>> >>>>> On Mon, Apr 1, 2019 at 7:59 AM Kodiak Firesmith <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Folks, >>>>>> I have a beta and a prod pulp server, beta is much smaller but >>>>>> otherwise I keep them identical. Last week I updated the beta host from >>>>>> 2.16 to latest without any issues. Today I attempted to upgrade prod in >>>>>> the same way and ended up with a fatal migration error :( >>>>>> >>>>>> Any chance a Pulp dev can parse this traceback? The final error that >>>>>> bombs the migration is 'IOError: Not a gzipped file', on migration unit >>>>>> 'pulp_rpm.plugins.migrations.0043_add_modulemd_modulemd-defaults'. >>>>>> >>>>>> https://paste.fedoraproject.org/paste/wEIu5a3Tf8OejEzMNsXN9w >>>>>> >>>>>> >>>>>> Packages installed are: >>>>>> $rpm -qa | egrep 'pulp|qpid|gofer|celery|mongo' | sort >>>>>> mongodb-2.6.12-6.el7.x86_64 >>>>>> mongodb-server-2.6.12-6.el7.x86_64 >>>>>> pulp-admin-client-2.18.1-2.el7.noarch >>>>>> pulp-docker-admin-extensions-3.2.2-1.el7.noarch >>>>>> pulp-docker-plugins-3.2.2-1.el7.noarch >>>>>> pulp-puppet-admin-extensions-2.18.1-1.el7.noarch >>>>>> pulp-puppet-plugins-2.18.1-1.el7.noarch >>>>>> pulp-rpm-admin-extensions-2.18.1-1.el7.noarch >>>>>> pulp-rpm-plugins-2.18.1-1.el7.noarch >>>>>> pulp-selinux-2.18.1-2.el7.noarch >>>>>> pulp-server-2.18.1-2.el7.noarch >>>>>> python2-celery-4.0.2-7.el7.noarch >>>>>> python2-qpid-1.37.0-4.el7.noarch >>>>>> python2-qpid-qmf-1.39.0-1.el7.x86_64 >>>>>> python-gofer-2.12.5-1.el7.noarch >>>>>> python-gofer-qpid-2.12.5-1.el7.noarch >>>>>> python-isodate-0.5.0-4.pulp.el7.noarch >>>>>> python-mongoengine-0.10.5-1.el7.noarch >>>>>> python-pulp-bindings-2.18.1-2.el7.noarch >>>>>> python-pulp-client-lib-2.18.1-2.el7.noarch >>>>>> python-pulp-common-2.18.1-2.el7.noarch >>>>>> python-pulp-docker-common-3.2.2-1.el7.noarch >>>>>> python-pulp-oid_validation-2.18.1-2.el7.noarch >>>>>> python-pulp-puppet-common-2.18.1-1.el7.noarch >>>>>> python-pulp-repoauth-2.18.1-2.el7.noarch >>>>>> python-pulp-rpm-common-2.18.1-1.el7.noarch >>>>>> python-pymongo-3.2-2.el7.x86_64 >>>>>> python-pymongo-gridfs-3.2-2.el7.x86_64 >>>>>> qpid-cpp-client-1.39.0-1.el7.x86_64 >>>>>> qpid-cpp-server-1.39.0-1.el7.x86_64 >>>>>> qpid-cpp-server-linearstore-1.39.0-1.el7.x86_64 >>>>>> qpid-proton-c-0.26.0-1.el7.x86_64 >>>>>> qpid-qmf-1.39.0-1.el7.x86_64 >>>>>> qpid-tools-1.39.0-1.el7.noarch >>>>>> >>>>>> Thanks very much for any input you can give on this issue! >>>>>> - Kodiak Firesmith >>>>>> _______________________________________________ >>>>>> Pulp-list mailing list >>>>>> [email protected] >>>>>> https://www.redhat.com/mailman/listinfo/pulp-list >>>>> >>>>>
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
