Hello community, here is the log from the commit of package autoyast2 for openSUSE:Factory checked in at 2018-02-02 22:18:45 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/autoyast2 (Old) and /work/SRC/openSUSE:Factory/.autoyast2.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "autoyast2" Fri Feb 2 22:18:45 2018 rev:231 rq:571704 version:4.0.28 Changes: -------- --- /work/SRC/openSUSE:Factory/autoyast2/autoyast2.changes 2018-01-28 00:37:28.202984101 +0100 +++ /work/SRC/openSUSE:Factory/.autoyast2.new/autoyast2.changes 2018-02-02 22:18:46.783686363 +0100 @@ -1,0 +2,30 @@ +Thu Feb 1 12:59:08 CET 2018 - [email protected] + +- Report packages which cannot be select for installation + (except those packages not included in the AutoYaST profile) + (bnc#1077292) +- 4.0.28 + +------------------------------------------------------------------- +Wed Jan 31 13:04:57 UTC 2018 - [email protected] + +- Speed optimization for the previous fix, the "clone_system" + client spent several minutes processing the packages + (related to bsc#1077882) +- 4.0.27 + +------------------------------------------------------------------- +Mon Jan 29 14:58:35 UTC 2018 - [email protected] + +- Avoid using Pkg.ResolvableProperties("", :package, "") calls + which require too much memory (bsc#1077882) +- 4.0.26 + +------------------------------------------------------------------- +Fri Jan 26 12:58:17 CET 2018 - [email protected] + +- Reporting packages which cannot be selected for installation. + (bnc#1077292) +- 4.0.25 + +------------------------------------------------------------------- Old: ---- autoyast2-4.0.24.tar.bz2 New: ---- autoyast2-4.0.28.tar.bz2 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ autoyast2.spec ++++++ --- /var/tmp/diff_new_pack.gk9kNC/_old 2018-02-02 22:18:47.527651631 +0100 +++ /var/tmp/diff_new_pack.gk9kNC/_new 2018-02-02 22:18:47.531651445 +0100 @@ -22,7 +22,7 @@ %endif Name: autoyast2 -Version: 4.0.24 +Version: 4.0.28 Release: 0 BuildRoot: %{_tmppath}/%{name}-%{version}-build @@ -119,8 +119,8 @@ # storage-ng based version Requires: yast2-update >= 3.3.0 Requires: yast2-xml -# pkgGpgCheck callback -Requires: yast2-pkg-bindings >= 3.1.31 +# "transact_by" key in PkgPropertiesAll() +Requires: yast2-pkg-bindings >= 4.0.7 # Y2Storage::AutoinstIssues containing section information BuildRequires: yast2-storage-ng >= 4.0.15 Provides: yast2-trans-autoinst ++++++ autoyast2-4.0.24.tar.bz2 -> autoyast2-4.0.28.tar.bz2 ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/autoyast2-4.0.24/package/autoyast2.changes new/autoyast2-4.0.28/package/autoyast2.changes --- old/autoyast2-4.0.24/package/autoyast2.changes 2018-01-24 16:46:07.000000000 +0100 +++ new/autoyast2-4.0.28/package/autoyast2.changes 2018-02-01 15:05:23.000000000 +0100 @@ -1,4 +1,34 @@ ------------------------------------------------------------------- +Thu Feb 1 12:59:08 CET 2018 - [email protected] + +- Report packages which cannot be select for installation + (except those packages not included in the AutoYaST profile) + (bnc#1077292) +- 4.0.28 + +------------------------------------------------------------------- +Wed Jan 31 13:04:57 UTC 2018 - [email protected] + +- Speed optimization for the previous fix, the "clone_system" + client spent several minutes processing the packages + (related to bsc#1077882) +- 4.0.27 + +------------------------------------------------------------------- +Mon Jan 29 14:58:35 UTC 2018 - [email protected] + +- Avoid using Pkg.ResolvableProperties("", :package, "") calls + which require too much memory (bsc#1077882) +- 4.0.26 + +------------------------------------------------------------------- +Fri Jan 26 12:58:17 CET 2018 - [email protected] + +- Reporting packages which cannot be selected for installation. + (bnc#1077292) +- 4.0.25 + +------------------------------------------------------------------- Wed Jan 24 16:20:01 CET 2018 - [email protected] - Selecting evaluated/given product for installation. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/autoyast2-4.0.24/package/autoyast2.spec new/autoyast2-4.0.28/package/autoyast2.spec --- old/autoyast2-4.0.24/package/autoyast2.spec 2018-01-24 16:46:07.000000000 +0100 +++ new/autoyast2-4.0.28/package/autoyast2.spec 2018-02-01 15:05:23.000000000 +0100 @@ -22,7 +22,7 @@ %endif Name: autoyast2 -Version: 4.0.24 +Version: 4.0.28 Release: 0 BuildRoot: %{_tmppath}/%{name}-%{version}-build @@ -119,8 +119,8 @@ # storage-ng based version Requires: yast2-update >= 3.3.0 Requires: yast2-xml -# pkgGpgCheck callback -Requires: yast2-pkg-bindings >= 3.1.31 +# "transact_by" key in PkgPropertiesAll() +Requires: yast2-pkg-bindings >= 4.0.7 # Y2Storage::AutoinstIssues containing section information BuildRequires: yast2-storage-ng >= 4.0.15 Provides: yast2-trans-autoinst diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/autoyast2-4.0.24/src/clients/inst_store_upgrade_software.rb new/autoyast2-4.0.28/src/clients/inst_store_upgrade_software.rb --- old/autoyast2-4.0.24/src/clients/inst_store_upgrade_software.rb 2018-01-24 16:46:07.000000000 +0100 +++ new/autoyast2-4.0.28/src/clients/inst_store_upgrade_software.rb 2018-02-01 15:05:23.000000000 +0100 @@ -41,30 +41,13 @@ Builtins.y2milestone("Patterns to install: %1", @patterns_to_install) Builtins.y2milestone("Patterns to remove: %1", @patterns_to_remove) - # find out status of packages - @packages = Pkg.ResolvableProperties("", :package, "") - @packages = Builtins.filter(@packages) do |p| - Ops.get(p, "transact_by") == :user || - Ops.get(p, "transact_by") == :app_high - end + @packages_to_remove = transactional_packages(:removed).concat( + transactional_packages(:available) + ) + @packages_to_install = transactional_packages(:selected).concat( + transactional_packages(:installed) + ) - @packages_to_remove = [] - @packages_to_install = Builtins.maplist(@packages) do |p| - if Ops.get(p, "status") == :selected || - Ops.get(p, "status") == :installed - next Ops.get_string(p, "name", "") - elsif Ops.get(p, "status") == :removed || - Ops.get(p, "status") == :available - @packages_to_remove = Builtins.add( - @packages_to_remove, - Ops.get_string(p, "name", "") - ) - end - nil - end - @packages_to_install = Builtins.filter(@packages_to_install) do |p| - p != nil - end Builtins.y2milestone("Packages to install: %1", @packages_to_install) Builtins.y2milestone("Packages to remove: %1", @packages_to_remove) @@ -111,6 +94,24 @@ :auto end + + private + + # get packages which are in requested state, ignore the packages changed + # by the solver + # @param status [Symbol] package status (:available, :selected, :installed, + # :removed) + # @return [Array<String>] package names + def transactional_packages(status) + # only package names (without version) + names_only = true + names = Pkg.GetPackages(status, names_only) + + names.select do |name| + packages = Pkg.ResolvableProperties(name, :package, "") + packages.any?{ |p| p["transact_by"] == :user || p["transact_by"] == :app_high } + end + end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/autoyast2-4.0.24/src/modules/AutoinstSoftware.rb new/autoyast2-4.0.28/src/modules/AutoinstSoftware.rb --- old/autoyast2-4.0.24/src/modules/AutoinstSoftware.rb 2018-01-24 16:46:07.000000000 +0100 +++ new/autoyast2-4.0.28/src/modules/AutoinstSoftware.rb 2018-02-01 15:05:23.000000000 +0100 @@ -19,6 +19,10 @@ # it contains some details of the failure BAD_LIST_FILE = "/var/log/YaST2/badlist".freeze + # Maximal amount of packages which will be shown + # in a popup. + MAX_PACKAGE_VIEW = 5 + def main Yast.import "UI" Yast.import "Pkg" @@ -105,8 +109,6 @@ @patterns = settings.fetch("patterns",[]) @instsource = settings.fetch("instsource","") - notFound = "" - @packagesAvailable = Pkg.GetPackages(:available, true) @patternsAvailable = [] allPatterns = Pkg.ResolvableDependencies("", :pattern, "") @@ -163,26 +165,6 @@ :to => "list <string>" ) - Builtins.foreach(Ops.get_list(settings, "packages", [])) do |pack| - if Stage.initial && # We are in the first installation stage - !Mode.config && # but not cloning system to autoinst.xml (bnc#901747) - !Pkg.IsAvailable(pack) # and package is NOT on installation medium - notFound = Ops.add(Ops.add(notFound, pack), "\n") - end - end - if Ops.greater_than(Builtins.size(notFound), 0) - Builtins.y2error("packages not found: %1", notFound) - # warning text during the installation. %1 is a list of package names - Report.Error( - Builtins.sformat( - _( - "These packages could not be found in the software repositories:\n%1" - ), - notFound - ) - ) - end - PackageAI.toinstall = settings.fetch("packages",[]) @kernel = settings.fetch("kernel","") @@ -896,27 +878,9 @@ log.info "No product has been selected for installation" end - autoinstPacks = autoinstPackages - # FIXME: optimization for package list evaluation turned off because it optimized it - # into an unbootable state (no kernel) bnc#427731 - # - # list<string> autoinstPacks = PackageAI::toinstall; - Builtins.y2milestone( - "Packages selected in autoinstall mode: %1", - autoinstPacks - ) - - if Ops.greater_than(Builtins.size(autoinstPacks), 0) - Builtins.y2milestone( - "Installing individual packages: %1", - Pkg.DoProvide(autoinstPacks) - ) - end + SelectPackagesForInstallation() computed_packages = Packages.ComputeSystemPackageList - Builtins.y2debug("Computed list of packages: %1", computed_packages) - Pkg.DoProvide(computed_packages) - Builtins.foreach(computed_packages) do |pack2| if Ops.greater_than(Builtins.size(@kernel), 0) && pack2 != @kernel && Builtins.search(pack2, "kernel-") == 0 @@ -992,12 +956,11 @@ nil end + # returns (hard and soft) locked packages + # @return [Array<String>] list of package names def locked_packages - packages = Pkg.ResolvableProperties("", :package, "").select do |package| - # hard AND soft locks - package["transact_by"] == :user && (package["locked"] || package["status"] == :available) - end - packages.map! {|p| p["name"] } + # hard AND soft locks + user_transact_packages(:taboo).concat(user_transact_packages(:available)) end def install_packages @@ -1010,7 +973,7 @@ # user selected packages which have already been installed installed_by_user = Pkg.GetPackages(:installed, true).select{ |pkg_name| - Pkg.PkgPropertiesAll(pkg_name).any? { |package| package["on_system_by_user"] } + Pkg.PkgPropertiesAll(pkg_name).any? { |p| p["on_system_by_user"] && p["status"] == :installed } } # Filter out kernel and pattern packages @@ -1156,6 +1119,51 @@ Profile.softwareCompat end + def SelectPackagesForInstallation + log.info "Individual Packages for installation: #{autoinstPackages}" + failed_packages = {} + failed_packages = Pkg.DoProvide(autoinstPackages) unless autoinstPackages.empty? + computed_packages = Packages.ComputeSystemPackageList + log.info "Computed packages for installation: #{computed_packages}" + failed_packages = failed_packages.merge(Pkg.DoProvide(computed_packages)) unless computed_packages.empty? + + # Blaming only packages which have been selected by the AutoYaST configuration file + log.error "Cannot select following packages for installation:" unless failed_packages.empty? + failed_packages.reject! do |name,reason| + if @Software["packages"] && @Software["packages"].include?(name) + log.error(" #{name} : #{reason} (selected by AutoYaST configuration file)") + false + else + log.error(" #{name} : #{reason} (selected by YAST automatically)") + true + end + end + + unless failed_packages.empty? + not_selected = "" + suggest_y2log = false + failed_count = failed_packages.size + if failed_packages.size > MAX_PACKAGE_VIEW + failed_packages = failed_packages.first(MAX_PACKAGE_VIEW).to_h + suggest_y2log = true + end + failed_packages.each do |name,reason| + not_selected << "#{name}: #{reason}\n" + end + # TRANSLATORS: Warning text during the installation. %s is a list of package + error_message = _("These packages cannot be found in the software repositories:\n%s") % not_selected + if suggest_y2log + # TRANSLATORS: Error message, %d is replaced by the amount of failed packages. + error_message += _("and %d additional packages") % (failed_count - MAX_PACKAGE_VIEW) + # TRANSLATORS: Error message, %s is replaced by "/var/log/YaST2/y2log" + error_message += "\n\n" + _("Details can be found in the %s file.") % + "/var/log/YaST2/y2log" + end + + Report.Error(error_message) + end + end + publish :function => :merge_product, :type => "void (string)" publish :variable => :Software, :type => "map" publish :variable => :image, :type => "map <string, any>" @@ -1187,6 +1195,23 @@ publish :function => :addPostPackages, :type => "void (list <string>)" publish :function => :ReadHelper, :type => "map <string, any> ()" publish :function => :Read, :type => "boolean ()" + + private + + # Get user transacted packages, include only the packages in the requested state + # @param status [Symbol] package status (:available, :selected, :installed, + # :removed) + # @return [Array<String>] package names + def user_transact_packages(status) + # only package names (without version) + names_only = true + packages = Pkg.GetPackages(status, names_only) + + # iterate over each package, Pkg.ResolvableProperties("", :package, "") requires a lot of memory + packages.select do |package| + Pkg.PkgPropertiesAll(package).any? { |p| p["transact_by"] == :user && p["status"] == status } + end + end end AutoinstSoftware = AutoinstSoftwareClass.new diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/autoyast2-4.0.24/test/AutoinstSoftware_test.rb new/autoyast2-4.0.28/test/AutoinstSoftware_test.rb --- old/autoyast2-4.0.24/test/AutoinstSoftware_test.rb 2018-01-24 16:46:07.000000000 +0100 +++ new/autoyast2-4.0.28/test/AutoinstSoftware_test.rb 2018-02-01 15:05:23.000000000 +0100 @@ -53,4 +53,57 @@ expect { subject.Export }.to raise_error(RuntimeError, "Found multiple base products") end end + + describe "selecting packages for installation" do + + before(:each) do + allow(subject).to receive(:autoinstPackages).and_return(["a1"]) + expect(Yast::Packages).to receive(:ComputeSystemPackageList).and_return(["a2","a3","a4"]) + expect(Yast::Pkg).to receive(:DoProvide).with(["a1"]).and_return({"a1" => "not found"}) + end + + it "shows a popup if some packages have not been found" do + expect(Yast::Pkg).to receive(:DoProvide).with(["a2","a3", "a4"]).and_return({}) + expect(Yast::Report).to receive(:Error) + subject.SelectPackagesForInstallation() + end + + it "shows a popup for not found packages which have been selected by AY configuration only" do + subject.Import(Yast::Profile.current["software"]) + expect(Yast::Pkg).to receive(:DoProvide).with(["a2","a3","a4"]).and_return({"a4" => "not found"}) + # a4 is not in the software/packages section + expect(Yast::Report).to receive(:Error).with("These packages cannot be found in the software repositories:\na1: not found\n") + subject.SelectPackagesForInstallation() + end + + it "shows no popup if no software section has been defined in the AY configuration" do + subject.Import({}) + expect(Yast::Pkg).to receive(:DoProvide).with(["a2","a3","a4"]).and_return({"a4" => "not found"}) + expect(Yast::Report).to_not receive(:Error) + subject.SelectPackagesForInstallation() + end + + end + + describe "#locked_packages" do + before do + expect(Yast::Pkg).to receive(:GetPackages).with(:taboo, true).and_return(["foo"]) + expect(Yast::Pkg).to receive(:GetPackages).with(:available, true).and_return(["bar"]) + end + + it "returns packages locked by user" do + # just mock only the needed attributes + expect(Yast::Pkg).to receive(:PkgPropertiesAll).with("foo").and_return(["transact_by" => :user, "status" => :taboo]) + expect(Yast::Pkg).to receive(:PkgPropertiesAll).with("bar").and_return(["transact_by" => :user, "status" => :available]) + expect(subject.locked_packages).to include("foo").and include("bar") + end + + it "ignores packages changed by the solver" do + # just mock only the needed attributes + expect(Yast::Pkg).to receive(:PkgPropertiesAll).with("foo").and_return(["transact_by" => :solver, "status" => :taboo]) + expect(Yast::Pkg).to receive(:PkgPropertiesAll).with("bar").and_return(["transact_by" => :solver, "status" => :available]) + expect(subject.locked_packages).to be_empty + end + end + end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/autoyast2-4.0.24/test/fixtures/profiles/software.xml new/autoyast2-4.0.28/test/fixtures/profiles/software.xml --- old/autoyast2-4.0.24/test/fixtures/profiles/software.xml 2018-01-24 16:46:07.000000000 +0100 +++ new/autoyast2-4.0.28/test/fixtures/profiles/software.xml 2018-02-01 15:05:23.000000000 +0100 @@ -9,5 +9,10 @@ <post-package>a2</post-package> <post-package>a3</post-package> </post-packages> + <packages config:type="list"> + <post-package>a1</post-package> + <post-package>a2</post-package> + <post-package>a3</post-package> + </packages> </software> </profile>
