Re: [Pulp-list] Error on Azure storage
Thanks very much for your help! From: Dennis Kliban [mailto:[email protected]] Sent: May 25, 2022 3:24 PM To: Briand, Sheldon Cc: [email protected] Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC I recommend filing an issue here: https://github.com/Azure/azure-sdk-for-python/issues It seems like that issue tracker has a few related issues in a closed state. Looks like they were resolved with a change on the Azure side. On Wed, May 25, 2022 at 2:19 PM Briand, Sheldon mailto:[email protected]>> wrote: I’m currently using: Django Storages: 1.11.1. Pulp core: 3.17.6 Pulp rpm: 3.17.4 From: Dennis Kliban [mailto:[email protected]<mailto:[email protected]>] Sent: May 25, 2022 3:10 PM To: Briand, Sheldon mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC What version of django-storages do you have installed? Could you try downgrading it to 1.11.1? Don't forget to restart pulpcore-content services after. I found a not completely related issue[0] with django-storages newer than 1.11.1. [0] https://github.com/jschneier/django-storages/issues/1116 On Wed, May 25, 2022 at 1:47 PM Briand, Sheldon mailto:[email protected]>> wrote: Both VMs are synced to the same time server using chronyd. From: Dennis Kliban [mailto:[email protected]<mailto:[email protected]>] Sent: May 25, 2022 2:46 PM To: Briand, Sheldon mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC Perhaps the time is wrong on the machine trying to download the content? On Wed, May 25, 2022 at 1:33 PM Briand, Sheldon mailto:[email protected]>> wrote: Thanks for getting back to me. The time on the server is correct (it is to the eastern time zone). From: Dennis Kliban [mailto:[email protected]<mailto:[email protected]>] Sent: May 25, 2022 2:29 PM To: Briand, Sheldon mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC I don't have much experience with S3, but is it possible that the date/time on your Pulp server is not correct? On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon mailto:[email protected]>> wrote: Hi, I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 pulp repository. From the dnf.log: error: Status code: 403 for https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- 10 The above error doesn’t show it but it includes a signature (I have purposely put xxx in for the storage account address). If I try to manually run the above query with curl and the URL in quotes I get: ERROR 403: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.. Signed expiry time [Wed, 25 May 2022 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 GMT] I ran a cron job which generated this error in the logs. The cron job ran at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time is set to a time before the request is even made and that gives me the 403 error. Is this a possible bug? Thanks, -Sheldon ___ Pulp-list mailing list [email protected]<mailto:[email protected]> https://listman.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
I recommend filing an issue here: https://github.com/Azure/azure-sdk-for-python/issues It seems like that issue tracker has a few related issues in a closed state. Looks like they were resolved with a change on the Azure side. On Wed, May 25, 2022 at 2:19 PM Briand, Sheldon < [email protected]> wrote: > I’m currently using: > > Django Storages: 1.11.1. > > Pulp core: 3.17.6 > > Pulp rpm: 3.17.4 > > > > *From:* Dennis Kliban [mailto:[email protected]] > *Sent:* May 25, 2022 3:10 PM > *To:* Briand, Sheldon > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Error on Azure storage > > > > ATTENTION*** This email originated from outside of the NRC. > ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC* > > What version of django-storages do you have installed? Could you try > downgrading it to 1.11.1? Don't forget to restart pulpcore-content services > after. > > > > I found a not completely related issue[0] with django-storages newer than > 1.11.1. > > > > [0] https://github.com/jschneier/django-storages/issues/1116 > > > > On Wed, May 25, 2022 at 1:47 PM Briand, Sheldon < > [email protected]> wrote: > > Both VMs are synced to the same time server using chronyd. > > > > *From:* Dennis Kliban [mailto:[email protected]] > *Sent:* May 25, 2022 2:46 PM > *To:* Briand, Sheldon > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Error on Azure storage > > > > ATTENTION*** This email originated from outside of the NRC. > ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC* > > Perhaps the time is wrong on the machine trying to download the content? > > > > On Wed, May 25, 2022 at 1:33 PM Briand, Sheldon < > [email protected]> wrote: > > Thanks for getting back to me. The time on the server is correct (it is > to the eastern time zone). > > > > *From:* Dennis Kliban [mailto:[email protected]] > *Sent:* May 25, 2022 2:29 PM > *To:* Briand, Sheldon > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Error on Azure storage > > > > ATTENTION*** This email originated from outside of the NRC. > ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC* > > I don't have much experience with S3, but is it possible that the > date/time on your Pulp server is not correct? > > > > On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon < > [email protected]> wrote: > > Hi, > > > > I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 > pulp repository. > > > > From the dnf.log: > > error: Status code: 403 for > https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 > > > cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- > > 10 > > > > The above error doesn’t show it but it includes a signature (I have > purposely put xxx in for the storage account address). > > > > If I try to manually run the above query with curl and the URL in quotes I > get: > > ERROR 403: Server failed to authenticate the request. Make sure the value > of Authorization header is formed correctly including the signature.. > > Signed expiry time [Wed, 25 May 2022 > 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 > GMT] > > > > I ran a cron job which generated this error in the logs. The cron job ran > at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time > is set to a time before the request is even made and that gives me the 403 > error. Is this a possible bug? > > > > Thanks, > > -Sheldon > > ___ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list > > ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
What version of django-storages do you have installed? Could you try downgrading it to 1.11.1? Don't forget to restart pulpcore-content services after. I found a not completely related issue[0] with django-storages newer than 1.11.1. [0] https://github.com/jschneier/django-storages/issues/1116 On Wed, May 25, 2022 at 1:47 PM Briand, Sheldon < [email protected]> wrote: > Both VMs are synced to the same time server using chronyd. > > > > *From:* Dennis Kliban [mailto:[email protected]] > *Sent:* May 25, 2022 2:46 PM > *To:* Briand, Sheldon > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Error on Azure storage > > > > ATTENTION*** This email originated from outside of the NRC. > ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC* > > Perhaps the time is wrong on the machine trying to download the content? > > > > On Wed, May 25, 2022 at 1:33 PM Briand, Sheldon < > [email protected]> wrote: > > Thanks for getting back to me. The time on the server is correct (it is > to the eastern time zone). > > > > *From:* Dennis Kliban [mailto:[email protected]] > *Sent:* May 25, 2022 2:29 PM > *To:* Briand, Sheldon > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Error on Azure storage > > > > ATTENTION*** This email originated from outside of the NRC. > ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC* > > I don't have much experience with S3, but is it possible that the > date/time on your Pulp server is not correct? > > > > On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon < > [email protected]> wrote: > > Hi, > > > > I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 > pulp repository. > > > > From the dnf.log: > > error: Status code: 403 for > https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 > > > cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- > > 10 > > > > The above error doesn’t show it but it includes a signature (I have > purposely put xxx in for the storage account address). > > > > If I try to manually run the above query with curl and the URL in quotes I > get: > > ERROR 403: Server failed to authenticate the request. Make sure the value > of Authorization header is formed correctly including the signature.. > > Signed expiry time [Wed, 25 May 2022 > 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 > GMT] > > > > I ran a cron job which generated this error in the logs. The cron job ran > at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time > is set to a time before the request is even made and that gives me the 403 > error. Is this a possible bug? > > > > Thanks, > > -Sheldon > > ___ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list > > ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
I’m currently using: Django Storages: 1.11.1. Pulp core: 3.17.6 Pulp rpm: 3.17.4 From: Dennis Kliban [mailto:[email protected]] Sent: May 25, 2022 3:10 PM To: Briand, Sheldon Cc: [email protected] Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC What version of django-storages do you have installed? Could you try downgrading it to 1.11.1? Don't forget to restart pulpcore-content services after. I found a not completely related issue[0] with django-storages newer than 1.11.1. [0] https://github.com/jschneier/django-storages/issues/1116 On Wed, May 25, 2022 at 1:47 PM Briand, Sheldon mailto:[email protected]>> wrote: Both VMs are synced to the same time server using chronyd. From: Dennis Kliban [mailto:[email protected]<mailto:[email protected]>] Sent: May 25, 2022 2:46 PM To: Briand, Sheldon mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC Perhaps the time is wrong on the machine trying to download the content? On Wed, May 25, 2022 at 1:33 PM Briand, Sheldon mailto:[email protected]>> wrote: Thanks for getting back to me. The time on the server is correct (it is to the eastern time zone). From: Dennis Kliban [mailto:[email protected]<mailto:[email protected]>] Sent: May 25, 2022 2:29 PM To: Briand, Sheldon mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC I don't have much experience with S3, but is it possible that the date/time on your Pulp server is not correct? On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon mailto:[email protected]>> wrote: Hi, I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 pulp repository. From the dnf.log: error: Status code: 403 for https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- 10 The above error doesn’t show it but it includes a signature (I have purposely put xxx in for the storage account address). If I try to manually run the above query with curl and the URL in quotes I get: ERROR 403: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.. Signed expiry time [Wed, 25 May 2022 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 GMT] I ran a cron job which generated this error in the logs. The cron job ran at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time is set to a time before the request is even made and that gives me the 403 error. Is this a possible bug? Thanks, -Sheldon ___ Pulp-list mailing list [email protected]<mailto:[email protected]> https://listman.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
Both VMs are synced to the same time server using chronyd. From: Dennis Kliban [mailto:[email protected]] Sent: May 25, 2022 2:46 PM To: Briand, Sheldon Cc: [email protected] Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC Perhaps the time is wrong on the machine trying to download the content? On Wed, May 25, 2022 at 1:33 PM Briand, Sheldon mailto:[email protected]>> wrote: Thanks for getting back to me. The time on the server is correct (it is to the eastern time zone). From: Dennis Kliban [mailto:[email protected]<mailto:[email protected]>] Sent: May 25, 2022 2:29 PM To: Briand, Sheldon mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC I don't have much experience with S3, but is it possible that the date/time on your Pulp server is not correct? On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon mailto:[email protected]>> wrote: Hi, I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 pulp repository. From the dnf.log: error: Status code: 403 for https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- 10 The above error doesn’t show it but it includes a signature (I have purposely put xxx in for the storage account address). If I try to manually run the above query with curl and the URL in quotes I get: ERROR 403: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.. Signed expiry time [Wed, 25 May 2022 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 GMT] I ran a cron job which generated this error in the logs. The cron job ran at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time is set to a time before the request is even made and that gives me the 403 error. Is this a possible bug? Thanks, -Sheldon ___ Pulp-list mailing list [email protected]<mailto:[email protected]> https://listman.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
Perhaps the time is wrong on the machine trying to download the content? On Wed, May 25, 2022 at 1:33 PM Briand, Sheldon < [email protected]> wrote: > Thanks for getting back to me. The time on the server is correct (it is > to the eastern time zone). > > > > *From:* Dennis Kliban [mailto:[email protected]] > *Sent:* May 25, 2022 2:29 PM > *To:* Briand, Sheldon > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Error on Azure storage > > > > ATTENTION*** This email originated from outside of the NRC. > ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC* > > I don't have much experience with S3, but is it possible that the > date/time on your Pulp server is not correct? > > > > On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon < > [email protected]> wrote: > > Hi, > > > > I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 > pulp repository. > > > > From the dnf.log: > > error: Status code: 403 for > https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 > > > cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- > > 10 > > > > The above error doesn’t show it but it includes a signature (I have > purposely put xxx in for the storage account address). > > > > If I try to manually run the above query with curl and the URL in quotes I > get: > > ERROR 403: Server failed to authenticate the request. Make sure the value > of Authorization header is formed correctly including the signature.. > > Signed expiry time [Wed, 25 May 2022 > 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 > GMT] > > > > I ran a cron job which generated this error in the logs. The cron job ran > at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time > is set to a time before the request is even made and that gives me the 403 > error. Is this a possible bug? > > > > Thanks, > > -Sheldon > > ___ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list > > ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
Thanks for getting back to me. The time on the server is correct (it is to the eastern time zone). From: Dennis Kliban [mailto:[email protected]] Sent: May 25, 2022 2:29 PM To: Briand, Sheldon Cc: [email protected] Subject: Re: [Pulp-list] Error on Azure storage ***ATTENTION*** This email originated from outside of the NRC. ***ATTENTION*** Ce courriel provient de l'extérieur du CNRC I don't have much experience with S3, but is it possible that the date/time on your Pulp server is not correct? On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon mailto:[email protected]>> wrote: Hi, I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 pulp repository. From the dnf.log: error: Status code: 403 for https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- 10 The above error doesn’t show it but it includes a signature (I have purposely put xxx in for the storage account address). If I try to manually run the above query with curl and the URL in quotes I get: ERROR 403: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.. Signed expiry time [Wed, 25 May 2022 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 GMT] I ran a cron job which generated this error in the logs. The cron job ran at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time is set to a time before the request is even made and that gives me the 403 error. Is this a possible bug? Thanks, -Sheldon ___ Pulp-list mailing list [email protected]<mailto:[email protected]> https://listman.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Error on Azure storage
I don't have much experience with S3, but is it possible that the date/time on your Pulp server is not correct? On Wed, May 25, 2022 at 7:58 AM Briand, Sheldon < [email protected]> wrote: > Hi, > > > > I’m getting an error when I’m accessing Azure storage for a AlmaLinux 8 > pulp repository. > > > > From the dnf.log: > > error: Status code: 403 for > https://xx/pulp-storage/artifact/37/e8d520f7f51a02b3530d1e51 > > > cc1b74e9b9f10c46c358ed78cca91ddba0?se=2022-05-23T20%3A01%3A52Z&sp=rt&sv=2021-04- > > 10 > > > > The above error doesn’t show it but it includes a signature (I have > purposely put xxx in for the storage account address). > > > > If I try to manually run the above query with curl and the URL in quotes I > get: > > ERROR 403: Server failed to authenticate the request. Make sure the value > of Authorization header is formed correctly including the signature.. > > Signed expiry time [Wed, 25 May 2022 > 09:51:09 GMT] must be after signed start time [Wed, 25 May 2022 10:00:16 > GMT] > > > > I ran a cron job which generated this error in the logs. The cron job ran > at Wed, 25 May 2022 09:56:00 GMT. So to me it looks like the expiry time > is set to a time before the request is even made and that gives me the 403 > error. Is this a possible bug? > > > > Thanks, > > -Sheldon > ___ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list > ___ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
