Re: Apache archive RedHat repodata

[Stijn Vanden Brande (External)]

Hi Michael,

thank you for the feedback!
We will look for an internal solution in this case.

Kind regards,
Stijn

From: Michael Shuler 
Sent: Friday, March 18, 2022 18:19
To: user@cassandra.apache.org 
Subject: Re: Apache archive RedHat repodata

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.


Caution! External email. Do not open attachments or click links, unless this 
email comes from a known sender and you know the content is safe.


On 3/18/22 02:31, Stijn Vanden Brande (External) wrote:
>
> The solution should be simply running `createrepo` after the new package
> is added in archive repository to have the correct repo data.

You stated the issue in your initial observation. The contents of
archive.a.o are an automated ASF-wide rsync from downloads.a.o. There is
no human to run `createrepo` on the archives. Nor would the addition of
some sort of post-rsync script work for all Apache Software Foundation
projects, as this would be error prone. It is a rolling archive of all
ASF current release artifacts. That's it.

If users have a need to follow a specific version for their clusters,
the recommendation has always been that they download the specific
artifacts they need, verify sigs, etc. and toss them on their own web
server, using the package repository tools they like for deb/rpm
packages. It is very common for database clusters to have no direct
internet access at all, and for nodes to have an internal-only web
server in the infrastructure with their "golden" cluster package
repository. Test test test some new version in dev or from the public
repos, then upload those verified packages to the "golden" repo for
production deployment. Done.

There is no simple way to do this on archive.a.o, so you'll have to do
this yourself, which ends up being way more stable and predictable in
the long run. It's a great question that comes up from time to time, it
just hasn't been asked recently.

Warm regards,
Michael
Worldline and Ingenico Group are registered trademarks and trade names owned by 
the Worldline Group. This e-mail and any documents attached are confidential 
and intended solely for the addressee. If you are not the intended recipient of 
this e-mail, you are not authorized to copy, disclose, use or retain it. Please 
notify the sender immediately and delete this e-mail (including any 
attachments) from your systems. As e-mails may be intercepted, amended or lost, 
they are not secure. Worldline and its subsidiaries therefore cannot accept 
liability for any errors in their content. Although the Worldline Group 
endeavours to maintain a virus-free network, we do not warrant that this e-mail 
is virus-free and do not accept liability for any damages or losses resulting 
from any transmitted virus if any. The risks are deemed to be accepted by 
anyone who communicates with Worldline or its subsidiaries by e-mail.

Re: Apache archive RedHat repodata

[Michael Shuler]

On 3/18/22 02:31, Stijn Vanden Brande (External) wrote:


The solution should be simply running `createrepo` after the new package 
is added in archive repository to have the correct repo data.


You stated the issue in your initial observation. The contents of 
archive.a.o are an automated ASF-wide rsync from downloads.a.o. There is 
no human to run `createrepo` on the archives. Nor would the addition of 
some sort of post-rsync script work for all Apache Software Foundation 
projects, as this would be error prone. It is a rolling archive of all 
ASF current release artifacts. That's it.


If users have a need to follow a specific version for their clusters, 
the recommendation has always been that they download the specific 
artifacts they need, verify sigs, etc. and toss them on their own web 
server, using the package repository tools they like for deb/rpm 
packages. It is very common for database clusters to have no direct 
internet access at all, and for nodes to have an internal-only web 
server in the infrastructure with their "golden" cluster package 
repository. Test test test some new version in dev or from the public 
repos, then upload those verified packages to the "golden" repo for 
production deployment. Done.


There is no simple way to do this on archive.a.o, so you'll have to do 
this yourself, which ends up being way more stable and predictable in 
the long run. It's a great question that comes up from time to time, it 
just hasn't been asked recently.


Warm regards,
Michael

Re: Apache archive RedHat repodata

[Stijn Vanden Brande (External)]

Hi Bowen,

indeed, this is a work-around.

However, this disables the GPG check or other yum specific configuration 
options.
While you do offer the yum repo data to have it function as a proper repository.

The solution should be simply running `createrepo` after the new package is 
added in archive repository to have the correct repo data.

Stijn


From: Bowen Song 
Sent: Thursday, March 17, 2022 18:49
To: user@cassandra.apache.org 
Subject: Re: Apache archive RedHat repodata

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe.


Caution! External email. Do not open attachments or click links, unless this 
email comes from a known sender and you know the content is safe.

If you need a specific (and old) version of Cassandra, you may consider using 
"yum install 
https://archive.apache.org/dist/cassandra/redhat/40x/cassandra-4.0.0-1.noarch.rpm<https://urldefense.com/v3/__https://archive.apache.org/dist/cassandra/redhat/40x/cassandra-4.0.0-1.noarch.rpm__;!!KtIQeTNdMUQ6!qO1FJpyDi0pUZKYiDZ_2lncWPZ_qb1hTYFyTwTAIDCOJLh7gGcuqxcIV5L4mqyuJyy35SB8A$>"
 instead of setting up the YUM repository and install the package from it. The 
URL to the archived RPM files are pretty stable, even the Cassandra 2.x RPMs 
are still available today, and their URL remained unchanged since the day one 
they were published.


On 14/03/2022 12:10, Stijn Vanden Brande (External) wrote:
Hi,

when configuring the Apache archive for Cassandra RPMs, with file 
"/etc/yum.repos.d/cassandra-oss.repo":
[cassandra-oss]
name=Cassandra OSS RPM
baseurl=https://archive.apache.org/dist/cassandra/redhat/40x<https://urldefense.com/v3/__https://archive.apache.org/dist/cassandra/redhat/40x__;!!KtIQeTNdMUQ6!qO1FJpyDi0pUZKYiDZ_2lncWPZ_qb1hTYFyTwTAIDCOJLh7gGcuqxcIV5L4mqyuJy1ChvzF0$>
failovermethod=priority
enabled=1
Older package versions (non-latest) are not found in the repodata nor 
installable.
While older packages are present in the repository: 
https://archive.apache.org/dist/cassandra/redhat/40x/<https://urldefense.com/v3/__https://archive.apache.org/dist/cassandra/redhat/40x/__;!!KtIQeTNdMUQ6!qO1FJpyDi0pUZKYiDZ_2lncWPZ_qb1hTYFyTwTAIDCOJLh7gGcuqxcIV5L4mqyuJy7T8oZBJ$>


$ yum info cassandra --show-duplicates --disableplugin=versionlock

Loaded plugins: ovl, priorities, product-id, search-disabled-repos

148 packages excluded due to repository priority protections

Available Packages

Name   : cassandra

Arch   : noarch

Version: 4.0.3

Release: 1

Size   : 45 M

Repo   : cassandra-oss-artifactory

Summary: Cassandra is a highly scalable, eventually consistent, 
distributed, structured key-value store.

URL: 
http://cassandra.apache.org/<https://urldefense.com/v3/__http://cassandra.apache.org/__;!!KtIQeTNdMUQ6!qO1FJpyDi0pUZKYiDZ_2lncWPZ_qb1hTYFyTwTAIDCOJLh7gGcuqxcIV5L4mqyuJyz2TuS_0$>

License: Apache Software License 2.0

Description : Cassandra is a distributed (peer-to-peer) system for the 
management and storage of structured data.

I assume the repodata is also mirrored from the main repository: 
https://downloads.apache.org/cassandra/redhat/40x/<https://urldefense.com/v3/__https://downloads.apache.org/cassandra/redhat/40x/__;!!KtIQeTNdMUQ6!qO1FJpyDi0pUZKYiDZ_2lncWPZ_qb1hTYFyTwTAIDCOJLh7gGcuqxcIV5L4mqyuJyxdFRIHB$>
And thus, the file list does not contain the other packages:
https://archive.apache.org/dist/cassandra/redhat/40x/repodata/repomd.xml<https://urldefense.com/v3/__https://archive.apache.org/dist/cassandra/redhat/40x/repodata/repomd.xml__;!!KtIQeTNdMUQ6!qO1FJpyDi0pUZKYiDZ_2lncWPZ_qb1hTYFyTwTAIDCOJLh7gGcuqxcIV5L4mqyuJy3jG8aZG$>

Primary file list:
$ zgrep 'ver=' 
ece31517ddeb401b0eaff53c83ecabd59f4862df38efffe436b68d0781516e6d-primary.xml.gz 


 -1-
  
  
  
  
  
  
  
  
  
  
Other file list:
$ zgrep 'ver=' 
4c42c3645f88f578a7e38b34b516313de36d1ab223bfaced8b757ba58bf2c32d-other.xml.gz




I would assume older available versions are part of the repodata and thus 
installable.
Can you please advise on this?

Kind regards,
Stijn Vanden Brande
Worldline and Ingenico Group are registered trademarks and trade names owned by 
the Worldline Group. This e-mail and any documents attached are confidential 
and intended solely for the addressee. If you are not the intended recipient of 
this e-mail, you are not authorized to copy, disclose, use or retain it. Please 
notify the sender immediately and delete this e-mail (including any 
attachments) from your systems. As e-mails may be intercepted, amended or lost, 
they are n

Re: Apache archive RedHat repodata

[Bowen Song]

If you need a specific (and old) version of Cassandra, you may consider 
using "yum install 
https://archive.apache.org/dist/cassandra/redhat/40x/cassandra-4.0.0-1.noarch.rpm"; 
instead of setting up the YUM repository and install the package from 
it. The URL to the archived RPM files are pretty stable, even the 
Cassandra 2.x RPMs are still available today, and their URL remained 
unchanged since the day one they were published.



On 14/03/2022 12:10, Stijn Vanden Brande (External) wrote:

Hi,

when configuring the Apache archive for Cassandra RPMs, with file 
"/etc/yum.repos.d/cassandra-oss.repo":


[cassandra-oss]
name=Cassandra OSS RPM
baseurl=https://archive.apache.org/dist/cassandra/redhat/40x
failovermethod=priority
enabled=1

Older package versions (non-latest) are not found in the /repodata/ 
nor installable.
While older packages are present in the repository: 
https://archive.apache.org/dist/cassandra/redhat/40x/


$ yum info cassandra --show-duplicates --disableplugin=versionlock

Loaded plugins: ovl, priorities, product-id, search-disabled-repos

148 packages excluded due to repository priority protections

Available Packages

Name   : cassandra

Arch   : noarch

Version: 4.0.3

Release: 1

Size   : 45 M

Repo   : cassandra-oss-artifactory

Summary: Cassandra is a highly scalable, eventually
consistent, distributed, structured key-value store.

URL: http://cassandra.apache.org/


License: Apache Software License 2.0

Description : Cassandra is a distributed (peer-to-peer) system for
the management and storage of structured data.


I assume the /repodata/ is also mirrored from the main repository: 
https://downloads.apache.org/cassandra/redhat/40x/

And thus, the file list does not contain the other packages:
https://archive.apache.org/dist/cassandra/redhat/40x/repodata/repomd.xml

Primary file list:

$ zgrep 'ver='

ece31517ddeb401b0eaff53c83ecabd59f4862df38efffe436b68d0781516e6d-primary.xml.gz
                                                 -1-
  
      
      
      
  
      
      
  
      
      

Other file list:

$ zgrep 'ver='

4c42c3645f88f578a7e38b34b516313de36d1ab223bfaced8b757ba58bf2c32d-other.xml.gz

    
    
    

I would assume older available versions are part of the repodata and 
thus installable.

Can you please advise on this?

Kind regards,
Stijn Vanden Brande
Worldline and Ingenico Group are registered trademarks and trade names 
owned by the Worldline Group. This e-mail and any documents attached 
are confidential and intended solely for the addressee. If you are not 
the intended recipient of this e-mail, you are not authorized to copy, 
disclose, use or retain it. Please notify the sender immediately and 
delete this e-mail (including any attachments) from your systems. As 
e-mails may be intercepted, amended or lost, they are not secure. 
Worldline and its subsidiaries therefore cannot accept liability for 
any errors in their content. Although the Worldline Group endeavours 
to maintain a virus-free network, we do not warrant that this e-mail 
is virus-free and do not accept liability for any damages or losses 
resulting from any transmitted virus if any. The risks are deemed to 
be accepted by anyone who communicates with Worldline or its 
subsidiaries by e-mail. 

Apache archive RedHat repodata

[Stijn Vanden Brande (External)]

Hi,

when configuring the Apache archive for Cassandra RPMs, with file 
"/etc/yum.repos.d/cassandra-oss.repo":
[cassandra-oss]
name=Cassandra OSS RPM
baseurl=https://archive.apache.org/dist/cassandra/redhat/40x
failovermethod=priority
enabled=1
Older package versions (non-latest) are not found in the repodata nor 
installable.
While older packages are present in the repository: 
https://archive.apache.org/dist/cassandra/redhat/40x/


$ yum info cassandra --show-duplicates --disableplugin=versionlock

Loaded plugins: ovl, priorities, product-id, search-disabled-repos

148 packages excluded due to repository priority protections

Available Packages

Name   : cassandra

Arch   : noarch

Version: 4.0.3

Release: 1

Size   : 45 M

Repo   : cassandra-oss-artifactory

Summary: Cassandra is a highly scalable, eventually consistent, 
distributed, structured key-value store.

URL: http://cassandra.apache.org/

License: Apache Software License 2.0

Description : Cassandra is a distributed (peer-to-peer) system for the 
management and storage of structured data.

I assume the repodata is also mirrored from the main repository: 
https://downloads.apache.org/cassandra/redhat/40x/
And thus, the file list does not contain the other packages:
https://archive.apache.org/dist/cassandra/redhat/40x/repodata/repomd.xml

Primary file list:
$ zgrep 'ver=' 
ece31517ddeb401b0eaff53c83ecabd59f4862df38efffe436b68d0781516e6d-primary.xml.gz 


 -1-
  
  
  
  
  
  
  
  
  
  
Other file list:
$ zgrep 'ver=' 
4c42c3645f88f578a7e38b34b516313de36d1ab223bfaced8b757ba58bf2c32d-other.xml.gz




I would assume older available versions are part of the repodata and thus 
installable.
Can you please advise on this?

Kind regards,
Stijn Vanden Brande
Worldline and Ingenico Group are registered trademarks and trade names owned by 
the Worldline Group. This e-mail and any documents attached are confidential 
and intended solely for the addressee. If you are not the intended recipient of 
this e-mail, you are not authorized to copy, disclose, use or retain it. Please 
notify the sender immediately and delete this e-mail (including any 
attachments) from your systems. As e-mails may be intercepted, amended or lost, 
they are not secure. Worldline and its subsidiaries therefore cannot accept 
liability for any errors in their content. Although the Worldline Group 
endeavours to maintain a virus-free network, we do not warrant that this e-mail 
is virus-free and do not accept liability for any damages or losses resulting 
from any transmitted virus if any. The risks are deemed to be accepted by 
anyone who communicates with Worldline or its subsidiaries by e-mail.