Hi, I tried no to modify the /usr/lib/python2.6/site-packages/pulp_rpm/extension/admin/upload/package.py as suggested by Christina Plummer. (on the separate build-server, not on the pulp server) just replacing sha256 with sha didn't work.
_checksum_calculate_checksum(type,file) needs 'sha1' as type to generate
a correct sha1 hash, but the checksum type for the pulp-metadata needs
to be 'sha'.
Now I managed to get the pulp-metadata right when i check on the command
line:
# pulp-admin rpm repo content rpm --repo-id centos.5.x86_64.puzzle |
grep Data-Random -C 20
Arch: noarch
Buildhost: buildserver.not.puzzle.ch
Checksum: 4ad4b6eb87df53002f2c25223495a1503458420f
Checksumtype: sha
Description: A module used to generate random data. Useful mostly for test
programs.
Epoch: 0
Filename: perl-Data-Random-0.05-1.el5.noarch.rpm
License: Artistic/GPL
Name: perl-Data-Random
Provides: perl(Data::Random) = 0.05-0, perl(Data::Random::WordList) =
0.05-0, perl-Data-Random = 0.05-1.el5-0
Release: 1.el5
Requires: perl, perl(Carp), perl(Date::Calc), perl(Exporter), perl(GD)
Vendor: Dag Apt Repository, http://dag.wieers.com/apt/
Version: 0.05
BUT then I run the repo publish command to generate the xml metadata
# pulp-admin rpm repo publish run --repo-id centos.5.x86_64.puzzle
+----------------------------------------------------------------------+
Publishing Repository [centos.5.x86_64.puzzle]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed
Publishing packages...
[==================================================] 100%
Packages: 157/157 items
... completed
Generating metadata
[\]
... completed
Publishing repository over HTTPS
[-]
... completed
Publishing repository over HTTP
[\]
... completed
AND it still generates the WRONG Hash to the repodata/primary.xml.gz:
<package type="rpm">
<name>perl-Data-Random</name>
<arch>noarch</arch><version epoch="0" ver="0.05" rel="1.el5"/>
<checksum type="sha256"
pkgid="YES">c45114bb7c1b2331e1bc352a691d3eaac1a14f3a6a5011eeae73c1eb8c8f86a2</checksum>
...
It looks completely broken, inconsistent on various levels
(architecture/implementation/process).
- checksum calculated per upload
- checksum type per file in a repository
- checksum type as repository option (OK) but without effect.
- inconsistent checksum (type) between metadata in database and in xml
repodata
- type sha means hashlib.new('sha1')
- Manual Package upload should be a core functionality of pulp and it is
so broken
- Bug is known since September last year
- Bug is categorized to "user-experience"
- Pulp is supposed to be a core part of Satellite 6
Sorry for the rant, but I expect better quality from Red Hat.
We want to use pulp as yum repository mirror and Server for RHEL/CentOS
5 & 6, stumbling over such a blocker is very disappointing.
Regards,
Philipp Gassmann
Am 17.06.2014 15:36, schrieb Philipp Gassmann:
> Hi Barnaby,
>
> This option (--checksum-type) is not present in the current stable
> version of pulp (2.3.1) as Christina Plummer already mentioned in March.
>
> And as mentioned below, it is still hardcoded in the current master branch:
> # Checksum
> unit_key['checksumtype'] = 'sha256' # hardcoded to this in v1 so
> leaving this way for now
>
> https://github.com/pulp/pulp_rpm/blame/master/extensions_admin/pulp_rpm/extensions/admin/upload/package.py#L178
>
>
> from the details of repo list --detail (see below) it looks like
> "Checksum Type" is a property of the distributor of the repository. That
> makes sense to me, as the whole repository is to be consumed by a EL 5
> client.
> BTW. "Auto Publish: True" does not seem to work when manually uploading
> a package (it did in pulp v1).
>
> Regards,
> Philipp
>
> Am 17.06.2014 15:02, schrieb Barnaby Court:
>> Hi, Pulp does support EL5 platforms. The trick is that you have to specify
>> the checksum during the upload of RPM content units. Pulp treats files with
>> a different checksum as completely different entities. Use the "pulp-admin
>> rpm repo uploads rpm --checksum-type sha ..." for uploading the package for
>> RHEL5. This allows you to override the default checksum type of sha256
>> whenever you upload an RPM. Upload the same rpm twice, once with
>> --checksum-type sha for your el5 repo and once without the option specified
>> or specified as sha256 for your EL6 repo. Regards,
>>
>> Barnaby
>>
>> ----- Original Message -----
>> From: "Philipp Gassmann" <[email protected]>
>> To: "Martin Collins" <[email protected]>, "Christina Plummer"
>> <[email protected]>, "Barnaby Court" <[email protected]>
>> Cc: [email protected]
>> Sent: Tuesday, June 17, 2014 4:51:57 AM
>> Subject: Re: [Pulp-list] Repo configured to use SHA checksums, but uploaded
>> packages still get SHA256
>>
>> Thanks for the replys!
>> Unfortuately those workarounds are not really usable if you need to
>> upload packages to existing repos for EL5 and EL6.
>> I migrated the EL5 repos from pulp_v1 by initially syncing from those
>> repos. Now I need to add additional packages to those repositories.
>>
>> I'm quite disappointed to find out that Pulp 2 does support repositories
>> for EL5, even though RHEL 5 is listed as supported Platform.
>>
>> http://www.pulpproject.org/
>> Supported Platforms:
>> Fedora 18, Fedora 19, Red Hat Enterprise Linux 5, Red Hat Enterprise Linux 6
>>
>> The sha256 is still hardcoded:
>> https://github.com/pulp/pulp_rpm/blob/master/extensions_admin/pulp_rpm/extensions/admin/upload/package.py
>>
>> The checksumtype is a repository option so the checksum should not be
>> different for different packages of the same repository.
>> Pulp 1 had the command to generate the metadata.
>> Pulp 2 has the publish command which has a stage "generating metadata",
>> there the checksum should be created.
>> Am I wrong on that?
>> Why does pulp 2.4 add an option "--checksum-type" to the upload command?
>> Shouln't it use the repository checksum-type option?
>> What is the checksum-type option of the repository for?
>>
>> pulp-admin rpm repo update --repo-id centos.5.x86_64.custom
>> --checksum-type=sha1
>>
>> Id: centos.5.i386.updates
>> Display Name: centos.5.i386.updates
>> Description: Last checked by Puppet on Tue Jun 17 06:43:30 CEST 2014
>> Content Unit Counts:
>> Rpm: 408
>> Notes:
>> Importers:
>> Config:
>> Feed: http://mirror.switch.ch/ftp/mirror/centos/5/updates/i386/
>> Id: yum_importer
>> Importer Type Id: yum_importer
>> Last Sync: 2014-05-21T11:54:49+02:00
>> Repo Id: centos.5.i386.updates
>> Scheduled Syncs:
>> Distributors:
>> Auto Publish: True
>> Config:
>> Checksum Type: sha1
>> Http: True
>> Https: True
>> Relative URL: centos.5.i386.updates
>> Distributor Type Id: yum_distributor
>> Id: yum_distributor
>> Last Publish: 2014-05-21T11:55:24+02:00
>> Repo Id: centos.5.i386.updates
>> Scheduled Publishes:
>>
>>
>> The Bug Report is here:
>> https://bugzilla.redhat.com/show_bug.cgi?id=1004580
>>
>>
>> Am 16.06.2014 23:37, schrieb Martin Collins:
>>> My workaround (as I have a large number of rhel6 and only a few for
>>> rhel5) was to just sync from a file:/// feed URL for rhel5 then just put
>>> the rpms in that dir, createrepo -s sha, then sync and continue normally
>>> from there
>>>
>>> Cheers
>>> Martin
>>>
>>> On 16 Jun 2014 22:33, "Christina Plummer" <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>> Hi Philipp,
>>>
>>> My workaround was to modify the upload/package.py and explicitly
>>> force it to use "sha" instead of "sha256". On my system, the file
>>> was
>>>
>>> /usr/lib/python2.6/site-packages/pulp_rpm/extension/admin/upload/package.py
>>> . See attached diff.
>>>
>>> Be sure to set aside the package.pyc and package.pyo files as well,
>>> and restart httpd. Then delete and re-upload the package to your repos.
>>>
>>> Belated public thanks to Ryan Bowlby for the pointer on this one.
>>>
>>> Christina
>>>
>>>
>>> On Mon, Jun 16, 2014 at 10:19 AM, Philipp Gassmann
>>> <[email protected] <mailto:[email protected]>> wrote:
>>>
>>> Hello Christina
>>>
>>> I just stumbled over the same issue. I uploaded the same noarch
>>> package
>>> to multiple repositories. now the installation of the package on
>>> CentOS
>>> 5 fails because of the wrong checksum.
>>>
>>> Did you find a solution or workaround?
>>> I can install the package if I download it manually and install
>>> it with
>>> yum --nogpgcheck.
>>>
>>> Regards,
>>> Philipp
>>>
>>>
>>> Am 11.03.2014 14:26, schrieb Christina Plummer:
>>> > Hi Barnaby,
>>> >
>>> > Thanks for your note. Unfortunately, --checksum-type doesn't
>>> seem to be
>>> > an option for the "pulp-admin rpm repo uploads rpm" command:
>>> >
>>> > CWC\cplummer1@cbigdc-rhel6test ~ $ pulp-admin rpm repo uploads rpm
>>> > --repo-id=q1-ol5-x86_64 --file
>>> oracle-validated-1.0.0-34.el5.x86_64.rpm
>>> > --checksum-type sha
>>> > Command: rpm
>>> > Description: uploads one or more RPMs into a repository
>>> >
>>> > Available Arguments:
>>> >
>>> > --repo-id - (required) unique identifier; only
>>> alphanumeric, .,
>>> > -, and _
>>> > allowed
>>> > --file, -f - full path to a file to upload; may be
>>> specified multiple
>>> > times for multiple files
>>> > --dir, -d - full path to a directory containing files
>>> to upload;
>>> > may be
>>> > specified multiple times for multiple
>>> directories
>>> > -v - display extra information about the upload
>>> process
>>> > --skip-existing - if specified, RPMs that already exist on
>>> the server
>>> > will not
>>> > be uploaded
>>> > The following options were specified but do not exist on the
>>> command:
>>> > --checksum-type
>>> >
>>> > I am using the RPMs for 2.3.1-1. Shouldn't it default to the
>>> > checksum-type for the repository, if specified?
>>> >
>>> > Thanks,
>>> > Christina
>>> >
>>> >
>>> > On Mon, Mar 10, 2014 at 2:31 PM, Barnaby Court
>>> <[email protected] <mailto:[email protected]>
>>> > <mailto:[email protected] <mailto:[email protected]>>> wrote:
>>> >
>>> > Christina,
>>> >
>>> > The checksum on the RPM is set at upload time. Was
>>> "--checksum-type
>>> > SHA1" set on the "pulp-admin rpm repo uploads rpm"
>>> command? If no
>>> > checksum type is specified it defaults to SHA256.
>>> >
>>> > Regards,
>>> > Barnaby
>>> >
>>> > ----- Original Message -----
>>> > From: "Christina Plummer" <[email protected]
>>> <mailto:[email protected]>
>>> > <mailto:[email protected] <mailto:[email protected]>>>
>>> > To: [email protected] <mailto:[email protected]>
>>> <mailto:[email protected] <mailto:[email protected]>>
>>> > Sent: Monday, March 10, 2014 1:42:14 PM
>>> > Subject: [Pulp-list] Repo configured to use SHA checksums,
>>> but
>>> > uploaded packages still get SHA256
>>> >
>>> > On Pulp 2.1.3, I had run into issues with sha256 checksums
>>> being
>>> > created for some of the repo metadata files, even though
>>> the repo
>>> > was configured to use sha instead.
>>> >
>>> > On Pulp 2.3.1, this seems to have been fixed... however, I
>>> am now
>>> > seeing that packages manually uploaded into a repo are
>>> still getting
>>> > a sha256 checksum in the repodata, which means that RHEL5/OL5
>>> > clients can't handle them. Packages that were synced, or
>>> copied from
>>> > another repository, have the correct checksum type.
>>> >
>>> > OS: RHEL 6.5 x86_64
>>> > Pulp: 2.3.1-1 (RPMs from Pulp repo)
>>> > Steps to reproduce:
>>> > 1. Created "live" repo fed from Oracle public yum
>>> repositories,
>>> > checksum-type=sha, retain-old-count=1
>>> > 2. Created "q1" repo, checksum-type=sha
>>> > 3. Copied rpms from "live" to "q1
>>> > 4. Deleted "known-broken" versions of
>>> oracle-validated-1.1.0 package
>>> > 5. Uploaded "known-working" version of
>>> oracle-validated-1.0.0 package
>>> > 6. Published repo. Every package except for
>>> oracle-validated-1.0.0
>>> > has a 'sha' checksum; oracle-validated-1.0.0 alone has a
>>> 'sha256'
>>> > checksum.
>>> >
>>> > Is this a bug?
>>> >
>>> > Thanks,
>>> > Christina
>>> >
>>> > _______________________________________________
>>> > Pulp-list mailing list
>>> > [email protected] <mailto:[email protected]>
>>> <mailto:[email protected] <mailto:[email protected]>>
>>> > https://www.redhat.com/mailman/listinfo/pulp-list
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > Pulp-list mailing list
>>> > [email protected] <mailto:[email protected]>
>>> > https://www.redhat.com/mailman/listinfo/pulp-list
>>> >
>>>
>>>
>>> --
>>> Philipp Gassmann
>>>
>>> Puzzle ITC GmbH
>>> http://www.puzzle.ch
>>>
>>> Telefon +41 31 370 22 00 <tel:%2B41%2031%20370%2022%2000>
>>> Werfen Sie einen Blick in unseren Blog:
>>> http://www.puzzle.ch/blog
>>>
>>>
>>>
>>> _______________________________________________
>>> Pulp-list mailing list
>>> [email protected] <mailto:[email protected]>
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>>
>>
>>
>
>
--
Philipp Gassmann
Puzzle ITC GmbH
http://www.puzzle.ch
Telefon +41 31 370 22 00
Direkt +41 31 370 22 02
Fax +41 31 370 22 01
Werfen Sie einen Blick in unseren Blog:
http://www.puzzle.ch/blog
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
