Re: Getting rid of port binaries in /software
On May 22, 2016, at 3:18 AM, Joshua Root wrote: > Eric A. Borisch wrote: >> If you are just looking to save some space at the expense of time, you >> could set: >> >> portarchivetype txz >> >> in macports.conf; on some of the big clang/llvm archives this is ~2x >> improvement... > > Nobody has yet mentioned the very important fact that you can't just change > the config file and have this work. Apple does not ship xz with OS X, so > there is an obvious bootstrap problem (How do you build the archive of the xz > port?) and a somewhat less obvious problem when upgrading xz (you need to > extract the archive of the new version *after* the old version has been > deactivated). > > So in fact you have to install xz somewhere outside of MacPorts and configure > base to use it, before you can make this config file change. Can we discuss the feasibility of bundling the necessary xz software with MacPorts base, like we already do for tcl? ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Getting rid of port binaries in /software
Eric A. Borisch wrote: If you are just looking to save some space at the expense of time, you could set: portarchivetype txz in macports.conf; on some of the big clang/llvm archives this is ~2x improvement... Nobody has yet mentioned the very important fact that you can't just change the config file and have this work. Apple does not ship xz with OS X, so there is an obvious bootstrap problem (How do you build the archive of the xz port?) and a somewhat less obvious problem when upgrading xz (you need to extract the archive of the new version *after* the old version has been deactivated). So in fact you have to install xz somewhere outside of MacPorts and configure base to use it, before you can make this config file change. - Josh ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
On Thu, May 19, 2016 at 5:43 AM, Ryan Schmidt wrote: > >> On May 18, 2016, at 6:05 PM, Mojca Miklavec wrote: >> >> On 19 May 2016 at 00:25, Eric A. Borisch wrote: >>> If you are just looking to save some space at the expense of time, you could >>> set: >>> >>> portarchivetype txz >>> >>> in macports.conf; on some of the big clang/llvm archives this is ~2x >>> improvement... >> >> But in current implementation that probably means that all the >> packages have to be compiled locally? > > Yes. Which will probably bring in build-only dependencies that weren't > necessary before. Which will remove some of the space savings. So this is > probably not a good change for users to make. > > Instead, we should switch the pre-built binaries to txz, as we've though > about doing before. Indeed, which is where I hoped the discussion would go. ;) ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
> On May 18, 2016, at 6:05 PM, Mojca Miklavec wrote: > > On 19 May 2016 at 00:25, Eric A. Borisch wrote: >> If you are just looking to save some space at the expense of time, you could >> set: >> >> portarchivetype txz >> >> in macports.conf; on some of the big clang/llvm archives this is ~2x >> improvement... > > But in current implementation that probably means that all the > packages have to be compiled locally? Yes. Which will probably bring in build-only dependencies that weren't necessary before. Which will remove some of the space savings. So this is probably not a good change for users to make. Instead, we should switch the pre-built binaries to txz, as we've though about doing before. ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
Hi, On Tue, May 17, 2016 at 06:38:24PM -0400, Nicolas Martin wrote: > > If you delete those archives you can no longer deactivate and > > re-activate a port. In addition to the use case above, this is also > > helpful when one of the files installed by the port was corrupted > > for some reason -- just de- and re-activate it. > > I suppose that if I were to manually delete those archives, MacPorts > would not be so kind as to detect this and just start the build > process over again, if he needs to? I thought about this earlier but then ended up forgetting to mention it in my initial reply: Deleting these archive files may trick an old script that was used to do the transition towards the archive-based approach into thinking that you're doing a migration from an old macports version when you run selfupdate. If that's the case, it will try to re-create the archives you deleted from the files you have on disk, which will fail. Since a couple of people ran into this before without asking on the list on possible side-effects, I think we should actually remove this upgrade code path soonish. It's been years since MacPorts 1.8 (?) that introduced this change, and it's unlikely that anybody still has a working 1.8 installation they want to upgrade. -- Clemens ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
Hi, On Wed, May 18, 2016 at 07:14:49PM -0400, Brandon Allbery wrote: > > Would it be safe then to have some of /opt/local/var/macports > > symlink-ed on an external hard drive ? > > I used to do this on my MacBook Air. One trick was to have .../sources > symlinked as well as .../software and .../build; this meant port > sync/install/activate/deactivate and friends failed safely if I didn't > have the drive attached. (I don't know if this is still true; haven't > used it in a while, and there have been a few updates to MacPorts > itself since then.) I haven't seen the code paths that use these foldes change a lot recently, so you should be fine. -- Clemens ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
On Tue, May 17, 2016 at 6:38 PM, Nicolas Martin wrote: > Would it be safe then to have some of /opt/local/var/macports symlink-ed > on an external hard drive ? I used to do this on my MacBook Air. One trick was to have .../sources symlinked as well as .../software and .../build; this meant port sync/install/activate/deactivate and friends failed safely if I didn't have the drive attached. (I don't know if this is still true; haven't used it in a while, and there have been a few updates to MacPorts itself since then.) -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
On 19 May 2016 at 00:25, Eric A. Borisch wrote: > If you are just looking to save some space at the expense of time, you could > set: > > portarchivetype txz > > in macports.conf; on some of the big clang/llvm archives this is ~2x > improvement... But in current implementation that probably means that all the packages have to be compiled locally? Mojca ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Getting rid of port binaries in /software
If you are just looking to save some space at the expense of time, you could set: portarchivetype txz in macports.conf; on some of the big clang/llvm archives this is ~2x improvement... On Wed, May 18, 2016 at 12:16 PM, Ryan Schmidt > wrote: > >> On May 17, 2016, at 5:38 PM, Nicolas Martin > wrote: >> >>> >>> On May 17, 2016, at 5:15 PM, Clemens Lang > wrote: >>> >>> On Tue, May 17, 2016 at 10:58:56AM -0400, Nicolas Martin wrote: I have looked for real answers regarding this question through the mailing list, but did not really understand the purpose of these files. >>> >>> MacPorts always keeps a tarball of the files installed by a certain >>> port in this directory. This allows you to switch between installed >>> versions or between ports that would otherwise conflict without >>> re-installing them completely. port activate/port deactivate implement >>> this. >> >> Is there a way to have MacPorts behave so as to completely reinstall a port if one needs to ? > > Regarding switching between installed versions of a port: MacPorts does not have the capability to build an arbitrary version of a port, only the current version of a port. So if you might want to switch back to an older version, you should keep it installed. Otherwise you have to perform manual steps to rebuild the old version as described here: > > https://trac.macports.org/wiki/howto/InstallingOlderPort > > Regarding switching between conflicting ports: If you don't want to keep both ports installed, for example for disk space reasons, you can of course uninstall one port and install the other. Then, to switch back, you can uninstall the second port and reinstall the first. > > >> I would prefer to wait through the process of building and activating the port again if I need to, than to lose quite a lot of space with duplicated binaries I almost never have to activate again. >> >>> The rationale here is that after an update you can test the updated >>> version of a software for a while, and if you notice it causes problems >>> you can file a ticket and easily go back to the old version with a >>> simple >>> sudo port activate @oldversion >> >> I understand this, but if you never have to revert to an older version of a package, this is quite a waste of space. > > It's also not entirely uncommon for a user to discover that some of a port's files have vanished or been replaced with the wrong contents, for example by running a third-party installer that installed older files into the MacPorts prefix. Some of these types of problems can be resolved by deactivating and then reactivating the affected ports, which you couldn't do if you didn't have an archive there. > > The decision to store these archives on the disk was made with the assumption that disk space is cheap, and becoming cheaper. And while disk space is indeed getting cheaper, we didn't anticipate the rise of more-expensive-yet-smaller SSDs. > >> >>> >>> MacPorts used to keep these files in a directory and just hard-link them >>> into $prefix, but that (a) means modifications to files in $prefix >>> affect the supposed-to-be prinstine copy, and (b) isn't easy to download >>> as pre-built binary. For this reason, we switched to tarballs a while >>> back and now provide pre-built binaries for some of these tar balls. > > Another reason was that hard links didn't play nicely with Time Machine. > > I have almost 5Gb of archives (tbz2) in /opt/local/var/macports/software. >>> >>> You seem to have quite a few ports installed. My software directory is >>> 3.4G with 652 installed ports. >> >> Well clang and llvm occupies a whooping 1.2G by themselves alone. > > True, clang and llvm are large. But according to > > https://packages.macports.org/llvm-3.7/?C=M;O=D > > llvm-3.7 is 377MB and according to > > https://packages.macports.org/clang-3.7/?C=M;O=D > > clang-3.7 is 357MB. If you're seeing twice that size on your system, maybe you've installed it with the universal variant. > > I have already run the port uninstall inactive command, so from my understanding, what remains in this path is from currently active and used ports. >>> >>> Correct. >>> What I don’t understand, and does not seem to be clear from any posts I have read regarding the matter, is why should these archives be kept? >>> >>> If you delete those archives you can no longer deactivate and >>> re-activate a port. In addition to the use case above, this is also >>> helpful when one of the files installed by the port was corrupted for >>> some reason -- just de- and re-activate it. >> >> I suppose that if I were to manually delete those archives, MacPorts would not be so kind as to detect this and just start the build process over again, if he needs to ? > > Deactivating a port for which you've deleted the archive would probably succeed, since the information about what files to remove comes from the registry and not from the archive, but attempting to reactivate it would p
Re: Getting rid of port binaries in /software
> On May 17, 2016, at 5:38 PM, Nicolas Martin > wrote: > >> >> On May 17, 2016, at 5:15 PM, Clemens Lang wrote: >> >> On Tue, May 17, 2016 at 10:58:56AM -0400, Nicolas Martin wrote: >>> I have looked for real answers regarding this question through the >>> mailing list, but did not really understand the purpose of these >>> files. >> >> MacPorts always keeps a tarball of the files installed by a certain >> port in this directory. This allows you to switch between installed >> versions or between ports that would otherwise conflict without >> re-installing them completely. port activate/port deactivate implement >> this. > > Is there a way to have MacPorts behave so as to completely reinstall a port > if one needs to ? Regarding switching between installed versions of a port: MacPorts does not have the capability to build an arbitrary version of a port, only the current version of a port. So if you might want to switch back to an older version, you should keep it installed. Otherwise you have to perform manual steps to rebuild the old version as described here: https://trac.macports.org/wiki/howto/InstallingOlderPort Regarding switching between conflicting ports: If you don't want to keep both ports installed, for example for disk space reasons, you can of course uninstall one port and install the other. Then, to switch back, you can uninstall the second port and reinstall the first. > I would prefer to wait through the process of building and activating the > port again if I need to, than to lose quite a lot of space with duplicated > binaries I almost never have to activate again. > >> The rationale here is that after an update you can test the updated >> version of a software for a while, and if you notice it causes problems >> you can file a ticket and easily go back to the old version with a >> simple >> sudo port activate @oldversion > > I understand this, but if you never have to revert to an older version of a > package, this is quite a waste of space. It's also not entirely uncommon for a user to discover that some of a port's files have vanished or been replaced with the wrong contents, for example by running a third-party installer that installed older files into the MacPorts prefix. Some of these types of problems can be resolved by deactivating and then reactivating the affected ports, which you couldn't do if you didn't have an archive there. The decision to store these archives on the disk was made with the assumption that disk space is cheap, and becoming cheaper. And while disk space is indeed getting cheaper, we didn't anticipate the rise of more-expensive-yet-smaller SSDs. > >> >> MacPorts used to keep these files in a directory and just hard-link them >> into $prefix, but that (a) means modifications to files in $prefix >> affect the supposed-to-be prinstine copy, and (b) isn't easy to download >> as pre-built binary. For this reason, we switched to tarballs a while >> back and now provide pre-built binaries for some of these tar balls. Another reason was that hard links didn't play nicely with Time Machine. >>> I have almost 5Gb of archives (tbz2) in >>> /opt/local/var/macports/software. >> >> You seem to have quite a few ports installed. My software directory is >> 3.4G with 652 installed ports. > > Well clang and llvm occupies a whooping 1.2G by themselves alone. True, clang and llvm are large. But according to https://packages.macports.org/llvm-3.7/?C=M;O=D llvm-3.7 is 377MB and according to https://packages.macports.org/clang-3.7/?C=M;O=D clang-3.7 is 357MB. If you're seeing twice that size on your system, maybe you've installed it with the universal variant. >>> I have already run the port uninstall inactive command, so from my >>> understanding, what remains in this path is from currently active and >>> used ports. >> >> Correct. >> >>> What I don’t understand, and does not seem to be clear from any posts >>> I have read regarding the matter, is why should these archives be >>> kept? >> >> If you delete those archives you can no longer deactivate and >> re-activate a port. In addition to the use case above, this is also >> helpful when one of the files installed by the port was corrupted for >> some reason -- just de- and re-activate it. > > I suppose that if I were to manually delete those archives, MacPorts would > not be so kind as to detect this and just start the build process over again, > if he needs to ? Deactivating a port for which you've deleted the archive would probably succeed, since the information about what files to remove comes from the registry and not from the archive, but attempting to reactivate it would probably give you an error message. Manually removing archives of installed ports from the software directory is not something we support at this time. MacPorts does not expect you to do that, and its behavior if you do is undefined. > Would it be safe then to have some of /opt/local/var/mac
Re: Getting rid of port binaries in /software
> On May 17, 2016, at 5:15 PM, Clemens Lang wrote: > > On Tue, May 17, 2016 at 10:58:56AM -0400, Nicolas Martin wrote: >> I have looked for real answers regarding this question through the >> mailing list, but did not really understand the purpose of these >> files. > > MacPorts always keeps a tarball of the files installed by a certain > port in this directory. This allows you to switch between installed > versions or between ports that would otherwise conflict without > re-installing them completely. port activate/port deactivate implement > this. Is there a way to have MacPorts behave so as to completely reinstall a port if one needs to ? I would prefer to wait through the process of building and activating the port again if I need to, than to lose quite a lot of space with duplicated binaries I almost never have to activate again. > The rationale here is that after an update you can test the updated > version of a software for a while, and if you notice it causes problems > you can file a ticket and easily go back to the old version with a > simple > sudo port activate @oldversion I understand this, but if you never have to revert to an older version of a package, this is quite a waste of space. > > MacPorts used to keep these files in a directory and just hard-link them > into $prefix, but that (a) means modifications to files in $prefix > affect the supposed-to-be prinstine copy, and (b) isn't easy to download > as pre-built binary. For this reason, we switched to tarballs a while > back and now provide pre-built binaries for some of these tar balls. > >> I have almost 5Gb of archives (tbz2) in >> /opt/local/var/macports/software. > > You seem to have quite a few ports installed. My software directory is > 3.4G with 652 installed ports. Well clang and llvm occupies a whooping 1.2G by themselves alone. > >> I have already run the port uninstall inactive command, so from my >> understanding, what remains in this path is from currently active and >> used ports. > > Correct. > >> What I don’t understand, and does not seem to be clear from any posts >> I have read regarding the matter, is why should these archives be >> kept? > > If you delete those archives you can no longer deactivate and > re-activate a port. In addition to the use case above, this is also > helpful when one of the files installed by the port was corrupted for > some reason -- just de- and re-activate it. I suppose that if I were to manually delete those archives, MacPorts would not be so kind as to detect this and just start the build process over again, if he needs to ? Would it be safe then to have some of /opt/local/var/macports symlink-ed on an external hard drive ? I would only plug the drive if I need to activate/deactivate those old (and large) ports, but small ports would be stored on my main drive. Would MacPorts still function with such a setup ? Thanks, Nicolas ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Re: Getting rid of port binaries in /software
On Tue, May 17, 2016 at 10:58:56AM -0400, Nicolas Martin wrote: > I have looked for real answers regarding this question through the > mailing list, but did not really understand the purpose of these > files. MacPorts always keeps a tarball of the files installed by a certain port in this directory. This allows you to switch between installed versions or between ports that would otherwise conflict without re-installing them completely. port activate/port deactivate implement this. The rationale here is that after an update you can test the updated version of a software for a while, and if you notice it causes problems you can file a ticket and easily go back to the old version with a simple sudo port activate @oldversion MacPorts used to keep these files in a directory and just hard-link them into $prefix, but that (a) means modifications to files in $prefix affect the supposed-to-be prinstine copy, and (b) isn't easy to download as pre-built binary. For this reason, we switched to tarballs a while back and now provide pre-built binaries for some of these tarballs. > I have almost 5Gb of archives (tbz2) in > /opt/local/var/macports/software. You seem to have quite a few ports installed. My software directory is 3.4G with 652 installed ports. > I have already run the port uninstall inactive command, so from my > understanding, what remains in this path is from currently active and > used ports. Correct. > What I don’t understand, and does not seem to be clear from any posts > I have read regarding the matter, is why should these archives be > kept? If you delete those archives you can no longer deactivate and re-activate a port. In addition to the use case above, this is also helpful when one of the files installed by the port was corrupted for some reason -- just de- and re-activate it. -- Clemens ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
Getting rid of port binaries in /software
Hi, I have looked for real answers regarding this question through the mailing list, but did not really understand the purpose of these files. I have almost 5Gb of archives (tbz2) in /opt/local/var/macports/software. I have already run the port uninstall inactive command, so from my understanding, what remains in this path is from currently active and used ports. What I don’t understand, and does not seem to be clear from any posts I have read regarding the matter, is why should these archives be kept ? I understand that port first compress the binaries in those files, and later extracts them to /opt/local during the activation. But once those ports are activated, why should we still keep them ? Thanks! ___ macports-users mailing list macports-users@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-users
