[yocto] [Recipe reporting system] Upgradable recipe name list
This mail was sent out by Recipe reporting system. This message list those recipes which need to be upgraded. If maintainers believe some of them needn't to upgrade at this time, they can fill RECIPE_NO_UPDATE_REASON in respective recipe file to ignore this remainder until newer upstream version was detected. Example: RECIPE_NO_UPDATE_REASON = "Version 2.0 is unstable" You can check the detail information at: http://recipes.yoctoproject.org/ Package Version Upstream version Maintainer NoUpgradeReason --- -- webkitgtk 2.18.6 2.20.1Alexander Kanavin gcr 3.20.0 3.28.0Alexander Kanavin gnome-desktop33.26.2 3.28.1Alexander Kanavin libsecret 0.18.5 0.18.6Alexander Kanavin libwebp 0.6.11.0.0 Alexander Kanavin meson 0.44.1 0.46.0Alexander Kanavin ffmpeg3.4.24.0 Alexander Kanavin dtc 1.4.51.4.6 Alexander Kanavin bash-completion 2.7 2.8 Alexander Kanavin gobject-introspec... 1.54.1 1.56.1Alexander Kanavin sysprof 3.26.1 3.28.1Alexander Kanavin bjam-native 1.66.0 1.67.0Alexander Kanavin libidn1.33 1.34 Alexander Kanavin epiphany 3.26.6 3.28.1.1 Alexander Kanavin pcmanfm 1.2.51.3.0 Alexander Kanavin boost 1.66.0 1.67.0Alexander Kanavin ca-certificates 20170717 20180409 Alexander Kanavin vala 0.38.8 0.40.4Alexander Kanavin libfm-extra 1.2.51.3.0 Alexander Kanavin libfm 1.2.51.3.0 Alexander Kanavin gtk-doc 1.27 1.28 Alexander Kanavin libaio0.3.110 0.3.111 Alexander Kanavin lighttpd 1.4.48 1.4.49Alexander Kanavin procps3.3.12 3.3.14Alexander Kanavin btrfs-tools 4.13.3 4.16 Alexander Kanavin Waiting for resolution of h... babeltrace1.5.41.5.5 Alexander Kanavin icu 60.2 61.1 Alexander Kanavin apt 1.2.24 1.6.1 Aníbal Limón dpkg 1.18.24 1.19.0.5 Aníbal Limón apt-native1.2.24 1.6.1 Aníbal Limón gstreamer1.0-vaapi1.12.4 1.14.0Anuj Mittal gstreamer1.0-plug... 1.12.4 1.14.0Anuj Mittal gstreamer1.0-rtsp... 1.12.4 1.14.0Anuj Mittal gstreamer1.0-plug... 1.12.4 1.14.0Anuj Mittal gst-validate 1.12.4 1.14.0Anuj Mittal gstreamer1.0-libav1.12.4 1.14.0Anuj Mittal gstreamer1.0-plug... 1.12.4 1.14.0Anuj Mittal gstreamer1.0-plug... 1.12.4 1.14.0Anuj Mittal gstreamer1.0 1.12.4 1.14.0Anuj Mittal gstreamer1.0-omx 1.12.4 1.14.0Anuj Mittal gstreamer1.0-python 1.12.4 1.14.0Anuj Mittal gst-examples 0.0.1+gitX 0.0.1-new-commits... Anuj Mittal mkfontscale 1.1.21.1.3 Armin Kuster xeyes 1.1.11.1.2 Armin Kuster xinit 1.3.41.4.0 Armin Kuster xkbcomp 1.4.01.4.1 Armin Kuster bind 9.10.6 9.12.1Armin Kuster xprop 1.2.21.2.3 Armin Kuster xset 1.2.31.2.4 Armin Kuster xwininfo 1.1.31.1.4 Armin Kuster openssh 7.6p17.7p1 Armin Kuster xf86-video-vesa 2.3.42.4.0 Armin Kuster libxcb1.12 1.13 Armin Kuster libxshmfence 1.2 1.3 Armin Kuster xkeyboard-config 2.22 2.23.1Armin Kuster calibrateproto0.0+gitX 0.0-new-commits-a... Armin Kuster xcb-proto 1.12
Re: [yocto] [PATCH 4/4] update_layer.py: move layer validation to update.py (Performance improve)
Yes, you're right, I updated the two comments in the repo: +if layerbranch: +if layerbranch.actual_branch: +branchname = layerbranch.actual_branch +branchdesc = "%s (%s)" % (branch, branchname) +else: +# LayerBranch doesn't exist for this branch, create it temporarily +# (we won't save this - update_layer.py will do the actual creation +# if it gets called). +newbranch = True +layerbranch = LayerBranch() +layerbranch.layer = layer ... +if layerbranch.vcs_last_rev == topcommit.hexsha and not update.reload: +logger.info("Layer %s is already up-to-date for branch %s" % (layer.name, branchdesc)) +collections.add((layerbranch.collection, layerbranch.version)) +continue +else: +# Check out appropriate branch +if not options.nocheckout: +utils.checkout_layer_branch(layerbranch, repodir, logger=logger) +layerdir = os.path.join(repodir, layerbranch.vcs_subdir) +if layerbranch.vcs_subdir and not os.path.exists(layerdir): +print_subdir_error(newbranch, layer.name, layerbranch.vcs_subdir, branchdesc) +continue + +if not os.path.exists(os.path.join(layerdir, 'conf/layer.conf')): +logger.error("conf/layer.conf not found for layer %s - is subdirectory set correctly?" % layer.name) +continue // Robert On 04/23/2018 11:35 AM, Paul Eggleton wrote: On Monday, 23 April 2018 3:06:43 PM NZST Robert Yang wrote: On 04/23/2018 09:55 AM, Paul Eggleton wrote: It seems that you are missing a call somewhere to layerbranch.save() later on, guarded with "if newbranch:". This was easy to miss though when copying over from update_layer.py seeing as we were relying upon saving it later on there (and that is still needed). We don't need it to save the new branch here, we just want to know whether it is a new branch or not, the logic is: if newbranch and "the branch doesn't exist" in the repo: continue else: # "the branch exits in the repo" update_layer.py So we don't have to save the new branch here, the update_layer.py will save the new branch when needed, this can reducing a lot of calls to update_layer, which can save a lot of time. OK - I had missed that you hadn't removed the code from update_layer.py that does the actual creation, so the logic is fine. I think however we need to adjust the "LayerBranch doesn't exist for this branch, create it" comment here though, something like "LayerBranch doesn't exist for this branch, create it temporarily (we won't save this - update_layer.py will do the actual creation if it gets called)." +if layerbranch.vcs_last_rev == topcommit.hexsha and not update.reload: +logger.info("Layer %s is already up-to-date for branch %s" % (layer.name, branchdesc)) +collections.add((layerbranch.collection, layerbranch.version)) +continue + +if layerbranch.vcs_last_rev != topcommit.hexsha or update.reload: +# Check out appropriate branch +if not options.nocheckout: +utils.checkout_layer_branch(layerbranch, repodir, logger=logger) Isn't "if layerbranch.vcs_last_rev != topcommit.hexsha or update.reload:" superfluous here? We're skipping to the next layer if that wasn't true via "continue" above, so it should always evaluate to true. Let's ignore the update.reload here, then the logic will be clear: if layerbranch.vcs_last_rev == topcommit.hexsha: continue else: checkout_layer_branch() So we can't ignore the else block here, the "if" block doesn't cover the "else" block. And If update.reload, then we can't continue, so the first "if" is: "if layerbranch.vcs_last_rev == topcommit.hexsha and not update.reload" And for the second "if", we need checkout the branch when: "layerbranch.vcs_last_rev != topcommit.hexsha or update.reload" And check whether there is a conf/layer.layer in the repo, we have a lot of layers which are only for specific releases, for example, a layer is for morty branch only, then we cleanup its master branch, just leave a README (no conf/layer.conf) file on master branch which says this layer is for morty only, check conf/layer.conf in update.py rather than in update_layer.py can save a lot of time, too. Right, I understand the logic, what I'm saying is that the two if statements are opposites, and the first one
Re: [yocto] [PATCH 4/4] update_layer.py: move layer validation to update.py (Performance improve)
On Monday, 23 April 2018 3:06:43 PM NZST Robert Yang wrote: > On 04/23/2018 09:55 AM, Paul Eggleton wrote: > > It seems that you are missing a call somewhere to layerbranch.save() later > > on, guarded with "if newbranch:". This was easy to miss though when copying > > over from update_layer.py seeing as we were relying upon saving it later on > > there (and that is still needed). > > We don't need it to save the new branch here, we just want to know whether it > is > a new branch or not, the logic is: > > if newbranch and "the branch doesn't exist" in the repo: > continue > else: # "the branch exits in the repo" > update_layer.py > > So we don't have to save the new branch here, the update_layer.py will save > the > new branch when needed, this can reducing a lot of calls to update_layer, > which > can save a lot of time. OK - I had missed that you hadn't removed the code from update_layer.py that does the actual creation, so the logic is fine. I think however we need to adjust the "LayerBranch doesn't exist for this branch, create it" comment here though, something like "LayerBranch doesn't exist for this branch, create it temporarily (we won't save this - update_layer.py will do the actual creation if it gets called)." > >> +if layerbranch.vcs_last_rev == topcommit.hexsha and > >> not update.reload: > >> +logger.info("Layer %s is already up-to-date for > >> branch %s" % (layer.name, branchdesc)) > >> +collections.add((layerbranch.collection, > >> layerbranch.version)) > >> +continue > >> + > >> +if layerbranch.vcs_last_rev != topcommit.hexsha or > >> update.reload: > >> +# Check out appropriate branch > >> +if not options.nocheckout: > >> +utils.checkout_layer_branch(layerbranch, > >> repodir, logger=logger) > > > > Isn't "if layerbranch.vcs_last_rev != topcommit.hexsha or update.reload:" > > superfluous here? We're skipping to the next layer if that wasn't true via > > "continue" above, so it should always evaluate to true. > > Let's ignore the update.reload here, then the logic will be clear: > > if layerbranch.vcs_last_rev == topcommit.hexsha: > continue > else: > checkout_layer_branch() > > So we can't ignore the else block here, the "if" block doesn't cover the > "else" > block. And If update.reload, then we can't continue, so the first "if" is: > "if layerbranch.vcs_last_rev == topcommit.hexsha and not update.reload" > > And for the second "if", we need checkout the branch when: > "layerbranch.vcs_last_rev != topcommit.hexsha or update.reload" > > And check whether there is a conf/layer.layer in the repo, we have a lot of > layers which are only for specific releases, for example, a layer is for morty > branch only, then we cleanup its master branch, just leave a README > (no conf/layer.conf) file on master branch which says this layer is for morty > only, check conf/layer.conf in update.py rather than in update_layer.py can > save a lot of time, too. Right, I understand the logic, what I'm saying is that the two if statements are opposites, and the first one results in a "continue" if true, so the second one can logically never evaluate to false, thus there's no point doing the check - just put the code inside the block immediately afterwards, i.e.: if layerbranch.vcs_last_rev == topcommit.hexsha and not update.reload: logger.info("Layer %s is already up-to-date for branch %s" % (layer.name, branchdesc)) collections.add((layerbranch.collection, layerbranch.version)) continue # Check out appropriate branch if not options.nocheckout: utils.checkout_layer_branch(layerbranch, repodir, logger=logger) ... If you wanted to make it even more clear you could have it in an "else" block instead, but that's not strictly necessary. Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH 4/4] update_layer.py: move layer validation to update.py (Performance improve)
Hi Paul, Thanks for the reply, please see my comments inline. On 04/23/2018 09:55 AM, Paul Eggleton wrote: Hi Robert I like the performance improvements, thanks. A few comments below though. On Wednesday, 18 April 2018 11:04:33 PM NZST Robert Yang wrote: The utils.setup_django() costs a lot of time, but both update.py and update_layer.py calls it, so move layer validation from update_layer.py to update.py to avoid calling update_layer.py when possible can save a lot of time. Now we don't have to call update_layer.py in the following cases: * The branch doesn't exist * The layer is already update to date on specified branch (when no reload) * The layer dir or conf/layer.layer doesn't exist We can save up to 98% time in my testing: $ update.py -b master --nofetch [--fullreload] BeforeNow Reduced No update: 276s 3.6s 98% Partial update:312s 87s 72% Full repload: 1016s 980s 3% Note: * All of the testing are based on --nofetch * "No update" means all layers on the branch is up-to-date, for example, when we run it twice, there is no update in the second run, so we only need about 3s now, which is the most common case when we use cron to run it per half an hour. * "Partly update" means part of the layers have been updated. * "Fullreload" means all of the layers have been updated. Signed-off-by: Robert Yang--- layerindex/update.py | 79 +- layerindex/update_layer.py | 39 +++ 2 files changed, 82 insertions(+), 36 deletions(-) diff --git a/layerindex/update.py b/layerindex/update.py index 44a7b90..2bb2df7 100755 --- a/layerindex/update.py +++ b/layerindex/update.py @@ -140,6 +140,15 @@ def fetch_repo(vcs_url, repodir, urldir, fetchdir, layer_name): logger.error("Fetch of layer %s failed: %s" % (layer_name, e.output)) return (vcs_url, e.output) +def print_subdir_error(newbranch, layername, vcs_subdir, branchdesc): +# This will error out if the directory is completely invalid or had never existed at this point +# If it previously existed but has since been deleted, you will get the revision where it was +# deleted - so we need to handle that case separately later +if newbranch: +logger.info("Skipping update of layer %s for branch %s - subdirectory %s does not exist on this branch" % (layername, branchdesc, vcs_subdir)) +elif vcs_subdir: +logger.error("Subdirectory for layer %s does not exist on branch %s - if this is legitimate, the layer branch record should be deleted" % (layername, branchdesc)) + def main(): if LooseVersion(git.__version__) < '0.3.1': logger.error("Version of GitPython is too old, please install GitPython (python-git) 0.3.1 or later in order to use this script") @@ -195,7 +204,7 @@ def main(): utils.setup_django() import settings -from layerindex.models import Branch, LayerItem, Update, LayerUpdate +from layerindex.models import Branch, LayerItem, Update, LayerUpdate, LayerBranch logger.setLevel(options.loglevel) @@ -337,6 +346,74 @@ def main(): collections.add((layerbranch.collection, layerbranch.version)) for layer in layerquery: +layerbranch = layer.get_layerbranch(branch) +branchname = branch +branchdesc = branch +newbranch = False +branchobj = utils.get_branch(branch) +if layerbranch: +if layerbranch.actual_branch: +branchname = layerbranch.actual_branch +branchdesc = "%s (%s)" % (branch, branchname) +else: +# LayerBranch doesn't exist for this branch, create it +newbranch = True +layerbranch = LayerBranch() +layerbranch.layer = layer +layerbranch.branch = branchobj +layerbranch_source = layer.get_layerbranch(branchobj) +if not layerbranch_source: +layerbranch_source = layer.get_layerbranch(None) +if layerbranch_source: +layerbranch.vcs_subdir = layerbranch_source.vcs_subdir + It seems that you are missing a call somewhere to layerbranch.save() later on, guarded with "if newbranch:". This was easy to miss though when copying over from update_layer.py seeing as we were relying upon saving it later on there (and that is still needed). We don't need it to save the new branch here, we just want to know whether it is a new branch or not, the logic is: if newbranch and "the branch doesn't exist" in the repo: continue else: # "the branch exits in the repo"
[yocto] [layerindex-web][PATCH 00/16] Django 1.11 upgrade (cover letter only)
Django 1.8 has gone out of support upstream, so here are the necessary changes to upgrade to Django 1.11.x (the next LTS release). Along the way I switched the Bulk Change feature to use OE's code for patching recipes, since that part was broken by the upgrade (and possibly broken otherwise). I've posted the cover letter only to avoid too much noise for folks that aren't interested - see the branch for details. The following changes since commit 611c96883c35240d3c291951146154d828745774: requirements.txt: use the most recent Django 1.8 version (2018-03-26 08:29:27 +1300) are available in the Git repository at: git://git.yoctoproject.org/layerindex-web paule/django-upgrade http://git.yoctoproject.org/cgit.cgi//log/?h=paule/django-upgrade Paul Eggleton (16): views: replace mimetype with content_type settings.py: switch to django.template.context_processors Make manage.py executable and tidy shebang Use new TEMPLATES setting Replace usage of model._meta.get_all_field_names() Stop using string name for publish view in urls.py urls*: Use new urlpatterns list syntax templatetags/addurlparameter: remove unused reference templates: replace use of = with == views: fix history tracking to work with current django-reversion urls: add required exception parameter for page_not_found views: Adapt to template rendering API change utils: add function to add meta/lib/oe to sys.path bulkchange.py: fix regression bulkchange.py: use oe.recipeutils code to patch recipes requirements.txt: bump Django and other dependency versions README | 4 +- layerindex/admin.py| 12 +-- layerindex/bulkchange.py | 134 ++--- layerindex/querysethelper.py | 2 +- layerindex/recipeparse.py | 14 --- layerindex/templatetags/addurlparameter.py | 2 +- layerindex/urls.py | 10 +-- layerindex/urls_branch.py | 4 +- layerindex/utils.py| 12 +++ layerindex/views.py| 92 +++- manage.py | 4 +- requirements.txt | 16 ++-- settings.py| 42 - templates/base_toplevel.html | 4 +- templates/layerindex/detail.html | 6 +- templates/layerindex/duplicates.html | 4 +- templates/layerindex/editlayer.html| 8 +- templates/layerindex/history.html | 8 +- templates/layerindex/reviewdetail.html | 6 +- urls.py| 7 +- 20 files changed, 139 insertions(+), 252 deletions(-) mode change 100644 => 100755 manage.py -- 2.14.3 -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [PATCH 4/4] update_layer.py: move layer validation to update.py (Performance improve)
Hi Robert I like the performance improvements, thanks. A few comments below though. On Wednesday, 18 April 2018 11:04:33 PM NZST Robert Yang wrote: > The utils.setup_django() costs a lot of time, but both update.py and > update_layer.py calls it, so move layer validation from update_layer.py to > update.py to avoid calling update_layer.py when possible can save a lot of > time. > > Now we don't have to call update_layer.py in the following cases: > * The branch doesn't exist > * The layer is already update to date on specified branch (when no > reload) > * The layer dir or conf/layer.layer doesn't exist > > We can save up to 98% time in my testing: > > $ update.py -b master --nofetch [--fullreload] > >BeforeNow Reduced > No update: 276s 3.6s 98% > Partial update:312s 87s 72% > Full repload: 1016s 980s 3% > > Note: > * All of the testing are based on --nofetch > > * "No update" means all layers on the branch is up-to-date, for > example, when we run it twice, there is no update in the second run, so we > only need about 3s now, which is the most common case when we use cron to > run > it per half an hour. > > * "Partly update" means part of the layers have been updated. > > * "Fullreload" means all of the layers have been updated. > > Signed-off-by: Robert Yang> --- > layerindex/update.py | 79 > +- > layerindex/update_layer.py | 39 +++ > 2 files changed, 82 insertions(+), 36 deletions(-) > > diff --git a/layerindex/update.py b/layerindex/update.py > index 44a7b90..2bb2df7 100755 > --- a/layerindex/update.py > +++ b/layerindex/update.py > @@ -140,6 +140,15 @@ def fetch_repo(vcs_url, repodir, urldir, fetchdir, > layer_name): > logger.error("Fetch of layer %s failed: %s" % (layer_name, e.output)) > return (vcs_url, e.output) > > +def print_subdir_error(newbranch, layername, vcs_subdir, branchdesc): > +# This will error out if the directory is completely invalid or had > never existed at this point > +# If it previously existed but has since been deleted, you will get the > revision where it was > +# deleted - so we need to handle that case separately later > +if newbranch: > +logger.info("Skipping update of layer %s for branch %s - > subdirectory %s does not exist on this branch" % (layername, branchdesc, > vcs_subdir)) > +elif vcs_subdir: > +logger.error("Subdirectory for layer %s does not exist on branch %s > - if this is legitimate, the layer branch record should be deleted" % > (layername, branchdesc)) > + > def main(): > if LooseVersion(git.__version__) < '0.3.1': > logger.error("Version of GitPython is too old, please install > GitPython (python-git) 0.3.1 or later in order to use this script") > @@ -195,7 +204,7 @@ def main(): > > utils.setup_django() > import settings > -from layerindex.models import Branch, LayerItem, Update, LayerUpdate > +from layerindex.models import Branch, LayerItem, Update, LayerUpdate, > LayerBranch > > logger.setLevel(options.loglevel) > > @@ -337,6 +346,74 @@ def main(): > collections.add((layerbranch.collection, > layerbranch.version)) > > for layer in layerquery: > +layerbranch = layer.get_layerbranch(branch) > +branchname = branch > +branchdesc = branch > +newbranch = False > +branchobj = utils.get_branch(branch) > +if layerbranch: > +if layerbranch.actual_branch: > +branchname = layerbranch.actual_branch > +branchdesc = "%s (%s)" % (branch, branchname) > +else: > +# LayerBranch doesn't exist for this branch, create > it > +newbranch = True > +layerbranch = LayerBranch() > +layerbranch.layer = layer > +layerbranch.branch = branchobj > +layerbranch_source = layer.get_layerbranch(branchobj) > +if not layerbranch_source: > +layerbranch_source = layer.get_layerbranch(None) > +if layerbranch_source: > +layerbranch.vcs_subdir = > layerbranch_source.vcs_subdir > + It seems that you are missing a call somewhere to layerbranch.save() later on, guarded with "if newbranch:". This was easy to miss though when copying over from update_layer.py seeing as we were relying upon saving it later on there (and that is still needed). > +if layerbranch.vcs_last_rev == topcommit.hexsha and not > update.reload: > +logger.info("Layer %s is
Re: [yocto] [meta-qt3][v2][PATCH 2/2] layer: Change LAYERSERIES_COMPAT to sumo
On Sunday, 22 April 2018 3:15:18 AM NZST akuster wrote: > > On 04/13/2018 03:47 PM, Armin Kuster wrote: > > [v2] > > fix typo in layer name > > > > Signed-off-by: Armin Kuster> Ping Also pushed. Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] [meta-qt4][v2][PATCH 2/2] layers: move LAYERSERIES_COMPAT to sumo
Hi Armin On Sunday, 22 April 2018 3:16:18 AM NZST akuster wrote: > On 04/13/2018 03:39 PM, Armin Kuster wrote: > > [v2] > > Fix cut error > > > > Signed-off-by: Armin Kuster> > --- > > conf/layer.conf | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/conf/layer.conf b/conf/layer.conf > > index 1ce54af..915d89c 100644 > > --- a/conf/layer.conf > > +++ b/conf/layer.conf > > @@ -8,4 +8,4 @@ BBFILE_COLLECTIONS += "qt4-layer" > > BBFILE_PATTERN_qt4-layer = "^${LAYERDIR}/" > > BBFILE_PRIORITY_qt4-layer = "6" > > > > -LAYERSERIES_COMPAT_qt4-layer = "rocko" > > +LAYERSERIES_COMPAT_qt4-layer = "sumo" > Ping for the series I have pushed this (and dropped the v2 text, that doesn't belong in the commit history - strictly speaking there should be some sort of message but I have let these pass). Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto