Re: Deprecation of portsnap (was: Proposed ports git transition schedule)
On 2021-Apr-11 14:27:27 +0200, Helge Oldach wrote: >Peter Jeremy via freebsd-ports wrote on Sun, 11 Apr 2021 00:52:11 +0200 (CEST): >> Following the SVN to GIT migration, portsnap is now the only practical >> way to use ports on a low-memory system. I've done some experiments >> and standard git has a 2GB working set to checkout a ports tree. > >However checking out is a one-time action with ports as there is only >one branch (switching frequently between main and quarterly is probably >not very sensible on a limited machine). git pull is significantly more >lightweight, I've just seen some 200M RSS. That should work well even on >a 512M machine. Probably much better than gitup in constrained memory? Except that git will arbitrarily and randomly decide that it needs to run "gc" - which is similarly extravagant in memory usage. Last time I found one running, it thrashed that poor VM for 3 days. Ignoring that, a "git up -ff" on a ports tree peaks with 2×1GB processes, though it looks like the working set size might only be ~350MB. -- Peter Jeremy signature.asc Description: PGP signature
gitup issues [was Re: No update for a day on ports?]
On 2021-Apr-01 22:36:02 +1100, Trev wrote: >> I just tried gitup and was told tha I had no permissions for >> /var/db/gitup. "sudo chown $USER /var/db/gitup" fixes that. >I just tried gitup and it was killed after apparently exhausting my >swap space. I had no such problem with portsnap :-( gitup walks the destination tree (ie /usr/ports), hashing every file it finds (by reading the complete file into memory). Whilst the default gitup configuration ignores /usr/ports/packages and /usr/ports/distfiles, it reads and hashes all the files down both paths before ignoring them. One consequence is that gitup is unusable on small memory environments - in my case, I saw peak memory usage hit 5GB (though the largest file I have is 1.8GB, so I'm not sure why it's eating so much memory). >My system (Vultr VPS) has 512K memory and 1.5G swap. git is barely usable on such a system (it's peak memory usage is about 2GB whilst processing a ports tree). gitup is unusable unless you either delete all packages and distfiles, or symlink them out of the way. > > Can I not run gitup as a normal user? > >I don't believe so. Well, git or gitup need write access to the working directory and the associated metadata directory. -- Peter Jeremy signature.asc Description: PGP signature
Re: Deprecation of portsnap (was: Proposed ports git transition schedule)
On 2021-Apr-01 12:19:08 +0200, Felix Palmen wrote: >* Christoph Moench-Tegeder [20210326 19:45]: >> ## Felix Palmen (fe...@palmen-it.de): >> >> > I'd assume (someone may correct me) that portsnap will still be >> > supported, >> >> https://lists.freebsd.org/pipermail/freebsd-ports/2020-August/119098.html > >Is this finally decided, and is there a timeline? Right now, it seems >portsnap is still built by default on releng/13.0. Following the SVN to GIT migration, portsnap is now the only practical way to use ports on a low-memory system. I've done some experiments and standard git has a 2GB working set to checkout a ports tree. gitup reached a 5GB working set size before I gave up. Typical small VPSs are around the 1GB RAM size and moving to something that can support 2GB or 5GB processes is a big price jump. -- Peter Jeremy signature.asc Description: PGP signature
