Do you mean that the bash script was the cause of the file descriptor issue, or the memory issue, or both?
On Thu, Nov 5, 2020 at 3:25 AM Winberg Adam <[email protected]> wrote: > sorry, ignore my problem. It was totally unrelated to pulp, it was another > process going haywire at the same time as I was running the migration (bug > in bash while loop in an unrelated housekeeping script). I was so focused > on the migration that I didn't even notice that the issue was elsewhere. > > > //Adam > > > ------------------------------ > *From:* [email protected] <[email protected]> on > behalf of Winberg Adam <[email protected]> > *Sent:* 05 November 2020 07:42 > *To:* Daniel Alley > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] 2to3 migration: memory usage and open file > handles > > > thats weird. I had a lot of memory usage while I was running on pulpcore > 3.4, but after upgrading to 3.7 there was hardly any memory usage. > > > I will reboot and run with 2 workers instead of 4 (don't know if that is > even relevant in migration) and run a new migration from scratch before > filing an issue. > > > //Adam > > > ------------------------------ > *From:* Daniel Alley <[email protected]> > *Sent:* 05 November 2020 01:58 > *To:* Winberg Adam > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] 2to3 migration: memory usage and open file > handles > > Hi Adam, > > We discovered (and fixed) some memory leaks in a library that we are using > [0] [1], which happens to be a Python extension written in C. Right now > we're still waiting on the maintainers of that library to review the > changes before we package them and ship the RPM, but we'll definitely let > you know what that happens. > > However, these issues would have affected every previous version equally, > so it's a little strange that you're only running into it now. Nothing > else about your setup has changed I assume? > > re: file descriptors, we've been testing migrating much larger systems > (300k RPMs, 600k errata) without running into problems, so I'm perplexed > about what could be causing that. File an issue and list which > repositories you're attempting to migrate and we'll see if we can reproduce. > > > [0] https://github.com/rpm-software-management/createrepo_c/pull/231 > [1] https://github.com/rpm-software-management/createrepo_c/pull/233 > > On Wed, Nov 4, 2020 at 12:40 PM Winberg Adam <[email protected]> wrote: > >> Hi, >> >> >> running a 2to3 migration with 2to3-migration-0.5.1 seems to consume a >> whole lot more memory than previous versions. My 12G RAM was quickly spent, >> i increased to 16G which wasnt enough either. Earlier migrations with 0.5.0 >> didnt spend anywhere near that amount. >> >> >> Also - the migration fails with >> >> OSError: [Errno 23] Too many open files in system: .. >> >> >> The memory usage increases while running the 'Migrating rpm content to >> Pulp 3 rpm' subtask. With 16G RAM I only get to about 114000/152000 pkgs in >> that task before the memory is more or less all consumed and the OSError >> appears. >> >> >> So it seems to me that there is some type of regression here. Any >> pointers on how I can further debug or work around this? >> >> >> This is on RHEL8 with >> >> python3-pulp-rpm-3.7.0-1.el8.noarch >> python3-pulpcore-3.7.3-1.el8.noarch >> python3-pulp-2to3-migration-0.5.1-1.el8.noarch >> >> //Adam >> >> _______________________________________________ >> Pulp-list mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/pulp-list > >
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
