Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
I continued the investigation and ended up filing a bug against Django[0]. I updated my initial patch to reflect the deeper understanding of the problem. The latest patch should generate fewer queries to the DB[1] during migration from Pulp 2 to 3. [0] https://code.djangoproject.com/ticket/32089 [1] https://github.com/pulp/pulpcore/pull/937 On Tue, Sep 29, 2020 at 9:58 AM Daniel Alley wrote: > The createrepo_c issue is a packaging problem, our Python packages include > the correct patch but the RPMs don't have the patch applied. We're going > to get this fixed within the next day or two. > > On Mon, Sep 28, 2020 at 3:00 AM Winberg Adam wrote: > >> Thanks! That's a very nice patch, after trying for a week to get the >> migration to work and continously searching for and deleting duplicate >> pulp2 content, the migration worked right away after this patch. :) >> >> >> Or, at least regarding the duplicate content errors, however I still have >> problems with a createrepo_c error, same as described in >> https://pulp.plan.io/issues/7193 (huge input lookup). >> >> >> That error only arise for a few packages so that is manageable for me to >> purge from pulp2 before migration, but it would be nice to not have to do >> that.. >> >> >> //Adam >> >> >> >> >> >> ------ >> *From:* Dennis Kliban >> *Sent:* 27 September 2020 17:35 >> *To:* Winberg Adam >> *Cc:* Ina Panova; pulp-list@redhat.com >> *Subject:* Re: [Pulp-list] 2to3migration fails on 'Package' object has >> no attribute '_remote_artifact_saver_cas' >> >> Here is a patch for pulpcore that fixes this problem[0]. >> >> [0] https://github.com/pulp/pulpcore/pull/937 >> >> On Mon, Sep 21, 2020 at 7:17 AM Winberg Adam >> wrote: >> >>> Thank you for your reply - yes I did clean orphans after i removed the >>> pulp2 repos that i suspected might be the cause. But as you say, there >>> might be some other cause for this in my case. >>> >>> >>> At the moment I have adjusted the code so the iteration only runs if the >>> '_remote_artifact_saver_cas' attribute is present. Don't know if the >>> result will be any good though, running the migration right now. >>> >>> >>> //Adam >>> >>> >>> -- >>> *From:* Ina Panova >>> *Sent:* 21 September 2020 13:09 >>> *To:* Winberg Adam >>> *Cc:* pulp-list@redhat.com >>> *Subject:* Re: [Pulp-list] 2to3migration fails on 'Package' object has >>> no attribute '_remote_artifact_saver_cas' >>> >>> Hi, >>> the provided steps in the mentioned issue are the steps to reproduce the >>> issue, however, unfortunately, this does not necessarily mean that this is >>> the root cause of the manifested problem. >>> Apparently we need to find a fix to properly handle duplicated >>> declarative content in a batch. >>> >>> Looking at the steps you have tried to bypass the issue, have you run >>> orphan clean up after pulp2 repos removal? >>> >>> >>> >>> Regards, >>> >>> Ina Panova >>> Senior Software Engineer| Pulp| Red Hat Inc. >>> >>> "Do not go where the path may lead, >>> go instead where there is no path and leave a trail." >>> >>> >>> On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam >>> wrote: >>> >>>> Hi, >>>> >>>> >>>> When running the 2to3migration for the 'rpm' plugin, I get the >>>> following error: >>>> >>>> >>>> AttributeError: 'Package' object has no attribute >>>> '_remote_artifact_saver_cas' >>>> >>>> >>>> >>>> This is the same as specified in https://pulp.plan.io/issues/7147, and >>>> I actually had a couple of repos in pulp2 which shared the same feed. I >>>> removed the redundant repos, flushed the pulp3 db and reran the >>>> 2to3-migration but still got stuck on the same error. >>>> >>>> >>>> Anyone got any pointers how to resolve this? >>>> >>>> >>>> //Adam >>>> ___ >>>> Pulp-list mailing list >>>> Pulp-list@redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-list >>> >>> ___ >>> Pulp-list mailing list >>> Pulp-list@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> ___ >> Pulp-list mailing list >> Pulp-list@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
The createrepo_c issue is a packaging problem, our Python packages include the correct patch but the RPMs don't have the patch applied. We're going to get this fixed within the next day or two. On Mon, Sep 28, 2020 at 3:00 AM Winberg Adam wrote: > Thanks! That's a very nice patch, after trying for a week to get the > migration to work and continously searching for and deleting duplicate > pulp2 content, the migration worked right away after this patch. :) > > > Or, at least regarding the duplicate content errors, however I still have > problems with a createrepo_c error, same as described in > https://pulp.plan.io/issues/7193 (huge input lookup). > > > That error only arise for a few packages so that is manageable for me to > purge from pulp2 before migration, but it would be nice to not have to do > that.. > > > //Adam > > > > > > -- > *From:* Dennis Kliban > *Sent:* 27 September 2020 17:35 > *To:* Winberg Adam > *Cc:* Ina Panova; pulp-list@redhat.com > *Subject:* Re: [Pulp-list] 2to3migration fails on 'Package' object has no > attribute '_remote_artifact_saver_cas' > > Here is a patch for pulpcore that fixes this problem[0]. > > [0] https://github.com/pulp/pulpcore/pull/937 > > On Mon, Sep 21, 2020 at 7:17 AM Winberg Adam wrote: > >> Thank you for your reply - yes I did clean orphans after i removed the >> pulp2 repos that i suspected might be the cause. But as you say, there >> might be some other cause for this in my case. >> >> >> At the moment I have adjusted the code so the iteration only runs if the >> '_remote_artifact_saver_cas' >> attribute is present. Don't know if the result will be any good though, >> running the migration right now. >> >> >> //Adam >> >> >> -- >> *From:* Ina Panova >> *Sent:* 21 September 2020 13:09 >> *To:* Winberg Adam >> *Cc:* pulp-list@redhat.com >> *Subject:* Re: [Pulp-list] 2to3migration fails on 'Package' object has >> no attribute '_remote_artifact_saver_cas' >> >> Hi, >> the provided steps in the mentioned issue are the steps to reproduce the >> issue, however, unfortunately, this does not necessarily mean that this is >> the root cause of the manifested problem. >> Apparently we need to find a fix to properly handle duplicated >> declarative content in a batch. >> >> Looking at the steps you have tried to bypass the issue, have you run >> orphan clean up after pulp2 repos removal? >> >> >> >> Regards, >> >> Ina Panova >> Senior Software Engineer| Pulp| Red Hat Inc. >> >> "Do not go where the path may lead, >> go instead where there is no path and leave a trail." >> >> >> On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam >> wrote: >> >>> Hi, >>> >>> >>> When running the 2to3migration for the 'rpm' plugin, I get the following >>> error: >>> >>> >>> AttributeError: 'Package' object has no attribute >>> '_remote_artifact_saver_cas' >>> >>> >>> >>> This is the same as specified in https://pulp.plan.io/issues/7147, and >>> I actually had a couple of repos in pulp2 which shared the same feed. I >>> removed the redundant repos, flushed the pulp3 db and reran the >>> 2to3-migration but still got stuck on the same error. >>> >>> >>> Anyone got any pointers how to resolve this? >>> >>> >>> //Adam >>> ___ >>> Pulp-list mailing list >>> Pulp-list@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> ___ >> Pulp-list mailing list >> Pulp-list@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > ___ > Pulp-list mailing list > Pulp-list@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
Thanks! That's a very nice patch, after trying for a week to get the migration to work and continously searching for and deleting duplicate pulp2 content, the migration worked right away after this patch. :) Or, at least regarding the duplicate content errors, however I still have problems with a createrepo_c error, same as described in https://pulp.plan.io/issues/7193 (huge input lookup). That error only arise for a few packages so that is manageable for me to purge from pulp2 before migration, but it would be nice to not have to do that.. //Adam From: Dennis Kliban Sent: 27 September 2020 17:35 To: Winberg Adam Cc: Ina Panova; pulp-list@redhat.com Subject: Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas' Here is a patch for pulpcore that fixes this problem[0]. [0] https://github.com/pulp/pulpcore/pull/937 On Mon, Sep 21, 2020 at 7:17 AM Winberg Adam mailto:adam.winb...@smhi.se>> wrote: Thank you for your reply - yes I did clean orphans after i removed the pulp2 repos that i suspected might be the cause. But as you say, there might be some other cause for this in my case. At the moment I have adjusted the code so the iteration only runs if the '_remote_artifact_saver_cas' attribute is present. Don't know if the result will be any good though, running the migration right now. //Adam From: Ina Panova mailto:ipan...@redhat.com>> Sent: 21 September 2020 13:09 To: Winberg Adam Cc: pulp-list@redhat.com<mailto:pulp-list@redhat.com> Subject: Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas' Hi, the provided steps in the mentioned issue are the steps to reproduce the issue, however, unfortunately, this does not necessarily mean that this is the root cause of the manifested problem. Apparently we need to find a fix to properly handle duplicated declarative content in a batch. Looking at the steps you have tried to bypass the issue, have you run orphan clean up after pulp2 repos removal? Regards, Ina Panova Senior Software Engineer| Pulp| Red Hat Inc. "Do not go where the path may lead, go instead where there is no path and leave a trail." On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam mailto:adam.winb...@smhi.se>> wrote: Hi, When running the 2to3migration for the 'rpm' plugin, I get the following error: AttributeError: 'Package' object has no attribute '_remote_artifact_saver_cas' This is the same as specified in https://pulp.plan.io/issues/7147, and I actually had a couple of repos in pulp2 which shared the same feed. I removed the redundant repos, flushed the pulp3 db and reran the 2to3-migration but still got stuck on the same error. Anyone got any pointers how to resolve this? //Adam ___ Pulp-list mailing list Pulp-list@redhat.com<mailto:Pulp-list@redhat.com> https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com<mailto:Pulp-list@redhat.com> https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
Here is a patch for pulpcore that fixes this problem[0]. [0] https://github.com/pulp/pulpcore/pull/937 On Mon, Sep 21, 2020 at 7:17 AM Winberg Adam wrote: > Thank you for your reply - yes I did clean orphans after i removed the > pulp2 repos that i suspected might be the cause. But as you say, there > might be some other cause for this in my case. > > > At the moment I have adjusted the code so the iteration only runs if the > '_remote_artifact_saver_cas' > attribute is present. Don't know if the result will be any good though, > running the migration right now. > > > //Adam > > > -- > *From:* Ina Panova > *Sent:* 21 September 2020 13:09 > *To:* Winberg Adam > *Cc:* pulp-list@redhat.com > *Subject:* Re: [Pulp-list] 2to3migration fails on 'Package' object has no > attribute '_remote_artifact_saver_cas' > > Hi, > the provided steps in the mentioned issue are the steps to reproduce the > issue, however, unfortunately, this does not necessarily mean that this is > the root cause of the manifested problem. > Apparently we need to find a fix to properly handle duplicated declarative > content in a batch. > > Looking at the steps you have tried to bypass the issue, have you run > orphan clean up after pulp2 repos removal? > > > > Regards, > > Ina Panova > Senior Software Engineer| Pulp| Red Hat Inc. > > "Do not go where the path may lead, > go instead where there is no path and leave a trail." > > > On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam wrote: > >> Hi, >> >> >> When running the 2to3migration for the 'rpm' plugin, I get the following >> error: >> >> >> AttributeError: 'Package' object has no attribute >> '_remote_artifact_saver_cas' >> >> >> >> This is the same as specified in https://pulp.plan.io/issues/7147, and I >> actually had a couple of repos in pulp2 which shared the same feed. I >> removed the redundant repos, flushed the pulp3 db and reran the >> 2to3-migration but still got stuck on the same error. >> >> >> Anyone got any pointers how to resolve this? >> >> >> //Adam >> ___ >> Pulp-list mailing list >> Pulp-list@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > ___ > Pulp-list mailing list > Pulp-list@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
Thank you for your reply - yes I did clean orphans after i removed the pulp2 repos that i suspected might be the cause. But as you say, there might be some other cause for this in my case. At the moment I have adjusted the code so the iteration only runs if the '_remote_artifact_saver_cas' attribute is present. Don't know if the result will be any good though, running the migration right now. //Adam From: Ina Panova Sent: 21 September 2020 13:09 To: Winberg Adam Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas' Hi, the provided steps in the mentioned issue are the steps to reproduce the issue, however, unfortunately, this does not necessarily mean that this is the root cause of the manifested problem. Apparently we need to find a fix to properly handle duplicated declarative content in a batch. Looking at the steps you have tried to bypass the issue, have you run orphan clean up after pulp2 repos removal? Regards, Ina Panova Senior Software Engineer| Pulp| Red Hat Inc. "Do not go where the path may lead, go instead where there is no path and leave a trail." On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam mailto:adam.winb...@smhi.se>> wrote: Hi, When running the 2to3migration for the 'rpm' plugin, I get the following error: AttributeError: 'Package' object has no attribute '_remote_artifact_saver_cas' This is the same as specified in https://pulp.plan.io/issues/7147, and I actually had a couple of repos in pulp2 which shared the same feed. I removed the redundant repos, flushed the pulp3 db and reran the 2to3-migration but still got stuck on the same error. Anyone got any pointers how to resolve this? //Adam ___ Pulp-list mailing list Pulp-list@redhat.com<mailto:Pulp-list@redhat.com> https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
Hi, the provided steps in the mentioned issue are the steps to reproduce the issue, however, unfortunately, this does not necessarily mean that this is the root cause of the manifested problem. Apparently we need to find a fix to properly handle duplicated declarative content in a batch. Looking at the steps you have tried to bypass the issue, have you run orphan clean up after pulp2 repos removal? Regards, Ina Panova Senior Software Engineer| Pulp| Red Hat Inc. "Do not go where the path may lead, go instead where there is no path and leave a trail." On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam wrote: > Hi, > > > When running the 2to3migration for the 'rpm' plugin, I get the following > error: > > > AttributeError: 'Package' object has no attribute > '_remote_artifact_saver_cas' > > > > This is the same as specified in https://pulp.plan.io/issues/7147, and I > actually had a couple of repos in pulp2 which shared the same feed. I > removed the redundant repos, flushed the pulp3 db and reran the > 2to3-migration but still got stuck on the same error. > > > Anyone got any pointers how to resolve this? > > > //Adam > ___ > Pulp-list mailing list > Pulp-list@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
[Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'
Hi, When running the 2to3migration for the 'rpm' plugin, I get the following error: AttributeError: 'Package' object has no attribute '_remote_artifact_saver_cas' This is the same as specified in https://pulp.plan.io/issues/7147, and I actually had a couple of repos in pulp2 which shared the same feed. I removed the redundant repos, flushed the pulp3 db and reran the 2to3-migration but still got stuck on the same error. Anyone got any pointers how to resolve this? //Adam ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
