Re: [opkg 0/3] Rework criteria for dependent package selection

2023-01-19 Thread Robert Marko
On Mon, 17 Oct 2022 at 19:07, Eneas U de Queiroz  wrote:
>
> This fixes a problem when generating an image using the firmware
> building, to include libwolfsslcpu-crypto.  Before they are sent to the
> asu server, the firmware builder strips ABI version from the packages
> and sort they alphabetically.  That means libustream-wolfssl will be
> installed before libwolfsslcpu-crypto.
>
> Opkg will see that libustream-wolfssl depends on
> libwolfssl5.5.1.b24d5f87.  Since it matches the name of the regular
> libwolfssl package, it is chosen and installed.  When it comes
> libwolfsslcpu-crypto's turn, it will fail because of a clash with the
> regular package.
>
> If you were to run it in the cmdline with the full name of
> libwolfsslcpu-crypto5.5.1.b24d5f87, or list it before any dpeendents,
> then it would work as expected.  However, because the firmware selector
> sripts ABI version and changes the order of the packages, there's no way
> to build an image with both libustrem-wolfssl and libwolfsslcpu-crypto.
>
> The first two commits attempt to add some order to the way they are
> currently chosen, by prioritizing packages chosen "by hand" and by
> preferring packages listed in the command line arguments over new
> packages chosen automatically.
>
> The third commit adds matching the package without ABI suffix, and
> establishes a hierarchy among the criteria, prioritizing user choices
> (i.e. package names given in as command line arguments), then developer
> choices (chosen package names), and resort to alphabetical order as a
> last resort.
>
> When resolving dependencies, packages listed in the cli may not have the
> ABI version, and they should have a higher priority over anything picked
> automatically.
>
> Use powers of two when computing the score to avoid ties due to
> different criteria, and so that it reflects what was matched.  The
> resulting priorities after this change are:
>
> 1. base score is 0
>
> ---USER CHOICES (cmdline)---
> 2. packages "picked by hand" (local file given in the cli) have absolute
>priority, ending the search regardless of score
> 3. package whose full name is in the cli: score += 4
> 4. package whose name stripped of ABI matches one in the cli: score += 2
>
> ---DEVELOPER CHOICE (pkg data)---
> 5. package whose full name matches the dependency name: score += 1
>Note: the ABI is recorded in the dependency, so I'm not using the
>stripped name here.
>
> 6. in case of a tie, the last package that was looked at is chosen
>(equivalent to being first in alphabetical order)
>
> I tried not to change things so much--aside from restoring the "picked
> by hand" case, I just created tie-breakers.  However, I still have some
> questions about the necessity of some of this.  For example: if more
> than one dependency is listed in the cli, does it matter which package
> is chosen?  I imagine it would be equivalent of the picked-by-hand case,
> so it would be simpler and faster to end the search.  It could make a
> difference if one were to install clashing packages with
> --force-overwrite in the same invocation, but I can't see a scenario
> where this would be useful.
>
> This was tested with the Image Builder, and by running opkg from command
> line on Linksys E8450 (mediatek/mt7622, aarch64_cortex-a53).

Hi,
I have just rediscovered this issue since ipq807x uses the libwolfsslcpu-crypto
as a target package by default, however, this is breaking image builder.

So, I wanted to bring some attention to this patchset that is fixing this issue
as I am sure we are gonna hit more of these in the future are well.

Tested-by: Robert Marko  # ipq807x

Regards,
Robert
>
> Signed-off-by: Eneas U de Queiroz 
>
> Eneas U de Queiroz (3):
>   libopkg: pkg_hash: restore picked by hand priority
>   libopkg: pkg_hash: bump score of packages in cli
>   libopkg: pkg_hash: consider names stripped of ABI
>
>  libopkg/pkg_hash.c | 35 +--
>  1 file changed, 29 insertions(+), 6 deletions(-)
>
>
> ___
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


[opkg 0/3] Rework criteria for dependent package selection

2022-10-17 Thread Eneas U de Queiroz
This fixes a problem when generating an image using the firmware
building, to include libwolfsslcpu-crypto.  Before they are sent to the
asu server, the firmware builder strips ABI version from the packages
and sort they alphabetically.  That means libustream-wolfssl will be
installed before libwolfsslcpu-crypto.

Opkg will see that libustream-wolfssl depends on
libwolfssl5.5.1.b24d5f87.  Since it matches the name of the regular
libwolfssl package, it is chosen and installed.  When it comes
libwolfsslcpu-crypto's turn, it will fail because of a clash with the
regular package.  

If you were to run it in the cmdline with the full name of
libwolfsslcpu-crypto5.5.1.b24d5f87, or list it before any dpeendents, 
then it would work as expected.  However, because the firmware selector
sripts ABI version and changes the order of the packages, there's no way
to build an image with both libustrem-wolfssl and libwolfsslcpu-crypto.

The first two commits attempt to add some order to the way they are
currently chosen, by prioritizing packages chosen "by hand" and by
preferring packages listed in the command line arguments over new
packages chosen automatically.

The third commit adds matching the package without ABI suffix, and
establishes a hierarchy among the criteria, prioritizing user choices
(i.e. package names given in as command line arguments), then developer
choices (chosen package names), and resort to alphabetical order as a
last resort.

When resolving dependencies, packages listed in the cli may not have the
ABI version, and they should have a higher priority over anything picked
automatically.

Use powers of two when computing the score to avoid ties due to
different criteria, and so that it reflects what was matched.  The
resulting priorities after this change are:

1. base score is 0

---USER CHOICES (cmdline)---
2. packages "picked by hand" (local file given in the cli) have absolute
   priority, ending the search regardless of score
3. package whose full name is in the cli: score += 4
4. package whose name stripped of ABI matches one in the cli: score += 2

---DEVELOPER CHOICE (pkg data)---
5. package whose full name matches the dependency name: score += 1
   Note: the ABI is recorded in the dependency, so I'm not using the
   stripped name here.

6. in case of a tie, the last package that was looked at is chosen
   (equivalent to being first in alphabetical order)

I tried not to change things so much--aside from restoring the "picked
by hand" case, I just created tie-breakers.  However, I still have some
questions about the necessity of some of this.  For example: if more
than one dependency is listed in the cli, does it matter which package
is chosen?  I imagine it would be equivalent of the picked-by-hand case,
so it would be simpler and faster to end the search.  It could make a
difference if one were to install clashing packages with
--force-overwrite in the same invocation, but I can't see a scenario
where this would be useful.

This was tested with the Image Builder, and by running opkg from command
line on Linksys E8450 (mediatek/mt7622, aarch64_cortex-a53).

Signed-off-by: Eneas U de Queiroz 

Eneas U de Queiroz (3):
  libopkg: pkg_hash: restore picked by hand priority
  libopkg: pkg_hash: bump score of packages in cli
  libopkg: pkg_hash: consider names stripped of ABI

 libopkg/pkg_hash.c | 35 +--
 1 file changed, 29 insertions(+), 6 deletions(-)


___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel