Re: [MacPorts] #51689: preserve_runtime_libraries convenience PortGroup

2016-09-16 Thread René J . V . Bertin
On Friday September 16 2016 16:06:54 Rainer Müller wrote:

>Moving the discussion to macports-dev as this is about the
>implementation, nothing users should care about.

You're not entirely right: users can and should care about ease of updating. I 
posted to macports-users on purpose, to get the largest possible awareness.

>That would be solved with dependency resolution using a SAT solver and
>version information in dependency specifications. We already started to
>work on that in last GSoC.

I'm not entirely sure if versioned dependencies are a solution to this 
particular problem. They shouldn't even become necessary when old runtime 
libraries can be left around; any port using them that gets updated will use 
the latest version. I don't see how you could end up in a situation where a 
non-rebuilt/reinstalled port that worked with a previous version will stop to 
work if you upgrade that dependency while keeping the required old runtime 
libraries around. A bump in the required dependency version will by definition 
be a result of an upgrade of the dependent.

>Putting the files into the destroot for the new archive is the wrong
>solution to this problem.

But the only workable workaround at the moment, as far as I can tell.
Even if it'll remain acceptable only in custom ports trees I'd still appreciate 
suggestions to improve it.

>Libraries that would not be replaced, but still have linking information
>need to be preserved on disk and in the registry indicating which port
>they belonged to. After all other ports linking to the files are
>upgraded, they can be safely removed.

Are you thinking of an automatic feature here? You're right that this should be 
feasible but that'd be so elegant even I didn't dare dreaming of it :)

>version) changes. This does not happen that often and as long as the
>major part of ports comes from our ports tree, the current solution of

It happens often enough with far-reaching enough consequences, in my perception!

>and cannot introduce the rev-bump at the same time. However, the
>automatic scanning in rev-upgrade will detect the problem and either
>upgrade the broken port with a new binary archive or eventually rebuild
>the port locally, ensuring the user has a working installation of all ports.

And hopefully for him/her, one that doesn't have any ports that require manual 
rebuilding, have pending changes, or any other whatever-it-is that means it 
shouldn't be upgraded completely automatically.

>needs to be done in base. As you said, that will require changes to the
>registry format. As we currently handle upgrade as a completely
>transparent deactivate old/activate new cycle, that would also require
>significant changes to the internal phases.

I know next to nothing about db internals, so I have no idea if one can store 
new data fields in a database and still keep it forwards compatible (i.e. old 
db file readable by the new code). I'd hope this is or can be more or less 
transparent, but if not, couldn't that new data be stored in an additional file?
And wouldn't that reduce the overhead of operating on a single huge db file, at 
least in theory?

R
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev


Re: [MacPorts] #51689: preserve_runtime_libraries convenience PortGroup

2016-09-16 Thread Rainer Müller
https://trac.macports.org/ticket/51689

Moving the discussion to macports-dev as this is about the
implementation, nothing users should care about.

On 2016-09-16 10:06, René J.V. Bertin wrote:
> My proposal aims to introduce (and provide a proof of concept) of an
> optional other solution, which could be used by select ports and give
> users with specific needs an easier way to adjust the update
> behaviour and cycle to their requirements. Easier than backing out of
> an upgrade cycle that turns out to have unwelcome consequences, copy
> the retrieved outdated culprit ports to a personal ports tree and
> start over again.
> 
> It'd be more elegant if implemented in "base", with more control over
> which libraries get preserved from an existing install, but I think
> it's obvious a non-default variant should be used to activate the
> feature.
> 
> It'd be even more elegant if the implementation actually reactivated
> select files from an older software image, meaning they'd remain
> members of that installed version only and be removed from the system
> when the user uninstalls that or those older versions. I'd love to
> try my hand at that but it's clearly going to involve substantial
> changes to "base", possibly even a change to the registry database,
> and I'm not comfortable with that. Maybe the additional metadata
> could be stored in an additional *sql file?
> 
> If that additional metadata includes a list of which ports depend on
> what "outdated" libraries it should even become trivial to present a
> list of recommended updates, i.e. the ports that ought to be rebuilt
> in order to use the latest dependencies available. That's the same
> list of ports that would break when uninstalling a particular version
> of a particular dependency.

That would be solved with dependency resolution using a SAT solver and
version information in dependency specifications. We already started to
work on that in last GSoC.

> I know that users can already decide not to upgrade select ports (and
> back out of an unwelcome upgrade by reactivating old versions), but
> then you quickly end up with an increasing list of outdated ports
> (making it hard to see and figure out which can and should be
> upgraded) and at some point you almost stop upgrading (and
> selfupdating) at all.

Overall, handling these kinds of upgrades could be improved. You are
correct that the proposed way would be what other package managers do,
for example Gentoo's portage:

https://wiki.gentoo.org/wiki/Preserve-libs

Putting the files into the destroot for the new archive is the wrong
solution to this problem.

Libraries that would not be replaced, but still have linking information
need to be preserved on disk and in the registry indicating which port
they belonged to. After all other ports linking to the files are
upgraded, they can be safely removed.

This problem will only appear when the SONAME (or dylib compatibility
version) changes. This does not happen that often and as long as the
major part of ports comes from our ports tree, the current solution of
rev-bumping works. The problem exists mainly for custom ports trees with
ports linking with the main ports tree. We have no control over these
and cannot introduce the rev-bump at the same time. However, the
automatic scanning in rev-upgrade will detect the problem and either
upgrade the broken port with a new binary archive or eventually rebuild
the port locally, ensuring the user has a working installation of all ports.

There is really no way to add such a functionality in a port group, it
needs to be done in base. As you said, that will require changes to the
registry format. As we currently handle upgrade as a completely
transparent deactivate old/activate new cycle, that would also require
significant changes to the internal phases.

Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev