both! After fixing that script the memory usage is very manageable.
//Adam ________________________________ From: Daniel Alley <[email protected]> Sent: 05 November 2020 15:30 To: Winberg Adam Cc: [email protected] Subject: Re: [Pulp-list] 2to3 migration: memory usage and open file handles Do you mean that the bash script was the cause of the file descriptor issue, or the memory issue, or both? On Thu, Nov 5, 2020 at 3:25 AM Winberg Adam <[email protected]<mailto:[email protected]>> wrote: sorry, ignore my problem. It was totally unrelated to pulp, it was another process going haywire at the same time as I was running the migration (bug in bash while loop in an unrelated housekeeping script). I was so focused on the migration that I didn't even notice that the issue was elsewhere. //Adam ________________________________ From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> on behalf of Winberg Adam <[email protected]<mailto:[email protected]>> Sent: 05 November 2020 07:42 To: Daniel Alley Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] 2to3 migration: memory usage and open file handles thats weird. I had a lot of memory usage while I was running on pulpcore 3.4, but after upgrading to 3.7 there was hardly any memory usage. I will reboot and run with 2 workers instead of 4 (don't know if that is even relevant in migration) and run a new migration from scratch before filing an issue. //Adam ________________________________ From: Daniel Alley <[email protected]<mailto:[email protected]>> Sent: 05 November 2020 01:58 To: Winberg Adam Cc: [email protected]<mailto:[email protected]> Subject: Re: [Pulp-list] 2to3 migration: memory usage and open file handles Hi Adam, We discovered (and fixed) some memory leaks in a library that we are using [0] [1], which happens to be a Python extension written in C. Right now we're still waiting on the maintainers of that library to review the changes before we package them and ship the RPM, but we'll definitely let you know what that happens. However, these issues would have affected every previous version equally, so it's a little strange that you're only running into it now. Nothing else about your setup has changed I assume? re: file descriptors, we've been testing migrating much larger systems (300k RPMs, 600k errata) without running into problems, so I'm perplexed about what could be causing that. File an issue and list which repositories you're attempting to migrate and we'll see if we can reproduce. [0] https://github.com/rpm-software-management/createrepo_c/pull/231 [1] https://github.com/rpm-software-management/createrepo_c/pull/233 On Wed, Nov 4, 2020 at 12:40 PM Winberg Adam <[email protected]<mailto:[email protected]>> wrote: Hi, running a 2to3 migration with 2to3-migration-0.5.1 seems to consume a whole lot more memory than previous versions. My 12G RAM was quickly spent, i increased to 16G which wasnt enough either. Earlier migrations with 0.5.0 didnt spend anywhere near that amount. Also - the migration fails with OSError: [Errno 23] Too many open files in system: .. The memory usage increases while running the 'Migrating rpm content to Pulp 3 rpm' subtask. With 16G RAM I only get to about 114000/152000 pkgs in that task before the memory is more or less all consumed and the OSError appears. So it seems to me that there is some type of regression here. Any pointers on how I can further debug or work around this? This is on RHEL8 with python3-pulp-rpm-3.7.0-1.el8.noarch python3-pulpcore-3.7.3-1.el8.noarch python3-pulp-2to3-migration-0.5.1-1.el8.noarch //Adam _______________________________________________ Pulp-list mailing list [email protected]<mailto:[email protected]> https://www.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
