Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/recipe-vocab-many-ppas-timeout into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/recipe-vocab-many-ppas-timeout/+merge/348569 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/recipe-vocab-many-ppas-timeout into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/recipe-vocab-many-ppas-timeout into lp:launchpad. Commit message: Optimise BuildableDistroSeries.findSeries. Requested reviews: Launchpad code reviewers (launchpad-reviewers) Related bugs: Bug #1778732 in Launchpad itself: "Recipe creation times out" https://bugs.launchpad.net/launchpad/+bug/1778732 For more details, see: https://code.launchpad.net/~cjwatson/launchpad/recipe-vocab-many-ppas-timeout/+merge/348569 I'm not sure whether this will be all of it or whether target_ppas_vocabulary will need some work too, but it's a start. -- Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/recipe-vocab-many-ppas-timeout into lp:launchpad. === modified file 'lib/lp/code/vocabularies/sourcepackagerecipe.py' --- lib/lp/code/vocabularies/sourcepackagerecipe.py 2015-07-08 16:05:11 + +++ lib/lp/code/vocabularies/sourcepackagerecipe.py 2018-06-27 01:01:06 + @@ -1,4 +1,4 @@ -# Copyright 2010-2013 Canonical Ltd. This software is licensed under the +# Copyright 2010-2018 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Source Package Recipe vocabularies used in the lp/code modules.""" @@ -37,8 +37,8 @@ @classmethod def findSeries(self, user): -ppas = getUtility(IArchiveSet).getPPAsForUser(user) -supported_distros = set([ppa.distribution for ppa in ppas]) +supported_distros = set( +getUtility(IArchiveSet).getPPADistributionsForUser(user)) # Now add in Ubuntu. supported_distros.add(getUtility(ILaunchpadCelebrities).ubuntu) all_series = getUtility(IDistroSeriesSet).search() === modified file 'lib/lp/soyuz/doc/archive.txt' --- lib/lp/soyuz/doc/archive.txt 2018-05-27 18:32:33 + +++ lib/lp/soyuz/doc/archive.txt 2018-06-27 01:01:06 + @@ -1208,6 +1208,22 @@ >>> jblack_ppas.count() 0 +'getPPADistributionsForUser' returns the distinct distributions for all the +PPAs that a given user participates in. + +>>> for distribution in archive_set.getPPADistributionsForUser(cprov): +... print(distribution.display_name) +Ubuntu +>>> for distribution in archive_set.getPPADistributionsForUser(no_priv): +... print(distribution.display_name) +Ubuntu +>>> for distribution in archive_set.getPPADistributionsForUser( +... indirect_uploader): +... print(distribution.display_name) +Ubuntu +>>> for distribution in archive_set.getPPADistributionsForUser(jblack): +... print(distribution.display_name) + The method getPrivatePPAs() will return a result set of all PPAs that are private. === modified file 'lib/lp/soyuz/interfaces/archive.py' --- lib/lp/soyuz/interfaces/archive.py 2018-05-04 21:59:32 + +++ lib/lp/soyuz/interfaces/archive.py 2018-06-27 01:01:06 + @@ -2383,6 +2383,9 @@ The result is ordered by PPA displayname. """ +def getPPADistributionsForUser(user): +"""Return the `Distribution`s of all PPAs for the given user.""" + def getPPAsPendingSigningKey(): """Return all PPAs pending signing key generation. === modified file 'lib/lp/soyuz/model/archive.py' --- lib/lp/soyuz/model/archive.py 2018-05-04 21:59:32 + +++ lib/lp/soyuz/model/archive.py 2018-06-27 01:01:06 + @@ -33,6 +33,7 @@ Or, Select, Sum, +Union, ) from storm.properties import ( Int, @@ -40,7 +41,10 @@ Unicode, ) from storm.references import Reference -from storm.store import Store +from storm.store import ( +EmptyResultSet, +Store, +) from zope.component import ( getAdapter, getUtility, @@ -2747,29 +2751,35 @@ SourcePackagePublishingHistory.archive == Archive.id) return store.find(Archive, *clause).order_by(Archive.id).first() +def _getPPAsForUserClause(self, user): +"""Base clause for getPPAsForUser and getPPADistributionsForUser.""" +direct_membership = Select( +Archive.id, +where=And( +Archive._enabled == True, +Archive.purpose == ArchivePurpose.PPA, +TeamParticipation.team == Archive.ownerID, +TeamParticipation.person == user, +)) +third_party_upload_acl = Select( +Archive.id, +where=And( +Archive.purpose == ArchivePurpose.PPA, +ArchivePermission.archiveID == Archive.id, +TeamParticipation.person == user, +TeamParticipation.team == ArchivePermission.personID, +)) +return Archive.id.is_in( +Union(direct_membership, third_party_upload_acl)) + def getPPAsForUser(self, user): """See `IArchiveSet`.""" from lp.registry.model.person import Person -# If there's no user
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-product-release-finder into lp:launchpad
Colin Watson has proposed merging
lp:~cjwatson/launchpad/explicit-proxy-product-release-finder into lp:launchpad.
Commit message:
Convert the product release finder to urlfetch.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-product-release-finder/+merge/348564
--
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~cjwatson/launchpad/explicit-proxy-product-release-finder into lp:launchpad.
=== modified file 'lib/lp/registry/scripts/productreleasefinder/finder.py'
--- lib/lp/registry/scripts/productreleasefinder/finder.py 2014-06-03 10:43:24 +
+++ lib/lp/registry/scripts/productreleasefinder/finder.py 2018-06-26 20:55:58 +
@@ -1,4 +1,4 @@
-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2018 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
__metaclass__ = type
@@ -12,11 +12,12 @@
import mimetypes
import os
import re
-import urllib
+import tempfile
import urlparse
from cscvs.dircompare import path
import pytz
+import requests
from zope.component import getUtility
from lp.app.validators.name import invalid_name_pattern
@@ -32,12 +33,17 @@
from lp.registry.model.productseries import ProductSeries
from lp.registry.scripts.productreleasefinder.filter import FilterPattern
from lp.registry.scripts.productreleasefinder.hose import Hose
+from lp.services.config import config
from lp.services.database import (
read_transaction,
write_transaction,
)
from lp.services.database.interfaces import IStore
from lp.services.librarian.model import LibraryFileAlias
+from lp.services.timeout import (
+default_timeout,
+urlfetch,
+)
processors = '|'.join([
@@ -104,8 +110,9 @@
def findReleases(self):
"""Scan for new releases in all products."""
-for product_name, filters in self.getFilters():
-self.handleProduct(product_name, filters)
+with default_timeout(config.productreleasefinder.timeout):
+for product_name, filters in self.getFilters():
+self.handleProduct(product_name, filters)
@read_transaction
def getFilters(self):
@@ -220,21 +227,27 @@
mimetype = 'application/octet-stream'
self.log.info("Downloading %s", url)
-try:
-local, headers = urllib.urlretrieve(url)
-stat = os.stat(local)
-except IOError:
-self.log.error("Download of %s failed", url)
-raise
-except OSError:
-self.log.error("Unable to stat downloaded file")
-raise
+with tempfile.TemporaryFile(prefix="product-release-finder") as fp:
+try:
+response = urlfetch(
+url, trust_env=False, use_proxy=True, output_file=fp)
+# XXX cjwatson 2018-06-26: This will all change with
+# requests 3.x. See:
+# https://blog.petrzemek.net/2018/04/22/
+expected_length = response.headers.get("Content-Length")
+if expected_length is not None:
+actual_length = response.raw.tell()
+expected_length = int(expected_length)
+if actual_length < expected_length:
+raise IOError(
+"Incomplete read: got %d, expected %d" %
+(actual_length, expected_length))
+except (IOError, requests.RequestException):
+self.log.exception("Download of %s failed", url)
+raise
+stat = os.fstat(fp.fileno())
+fp.seek(0)
-try:
-fp = open(local, 'r')
-os.unlink(local)
self.addReleaseTarball(product_name, series_name, version,
filename, stat.st_size, fp, mimetype)
file_names.add(filename)
-finally:
-fp.close()
=== modified file 'lib/lp/registry/tests/test_prf_finder.py'
--- lib/lp/registry/tests/test_prf_finder.py 2018-01-02 10:54:31 +
+++ lib/lp/registry/tests/test_prf_finder.py 2018-06-26 20:55:58 +
@@ -1,4 +1,4 @@
-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2018 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
import logging
@@ -8,6 +8,7 @@
import tempfile
import unittest
+import responses
import transaction
from zope.component import getUtility
from zope.interface.verify import verifyObject
@@ -219,22 +220,29 @@
def create_tarball(self, file_name):
"""create a release tarball for testing"""
file_path = os.path.join(self.release_root, file_name)
-try:
-fp =
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/no-default-http-proxy into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/no-default-http-proxy into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/no-default-http-proxy/+merge/348540 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/no-default-http-proxy into lp:launchpad
Review: Approve I'm going to self-approve this, as it's pretty trivial and it fixes a current test failure on buildbot. -- https://code.launchpad.net/~cjwatson/launchpad/no-default-http-proxy/+merge/348540 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-mirror-prober into lp:launchpad
Colin Watson has proposed merging
lp:~cjwatson/launchpad/explicit-proxy-mirror-prober into lp:launchpad with
lp:~cjwatson/launchpad/no-default-http-proxy as a prerequisite.
Commit message:
Convert the mirror prober to use explicit proxy configuration.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-mirror-prober/+merge/348543
--
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~cjwatson/launchpad/explicit-proxy-mirror-prober into lp:launchpad.
=== modified file 'cronscripts/distributionmirror-prober.py'
--- cronscripts/distributionmirror-prober.py 2013-01-07 02:40:55 +
+++ cronscripts/distributionmirror-prober.py 2018-06-26 15:14:17 +
@@ -1,14 +1,12 @@
#!/usr/bin/python -S
#
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2018 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Script to probe distribution mirrors and check how up-to-date they are."""
import _pythonpath
-import os
-
from lp.registry.interfaces.distributionmirror import MirrorContent
from lp.registry.scripts.distributionmirror_prober import DistroMirrorProber
from lp.services.config import config
@@ -16,6 +14,7 @@
LaunchpadCronScript,
LaunchpadScriptFailure,
)
+from lp.services.timeout import set_default_timeout_function
class DistroMirrorProberScript(LaunchpadCronScript):
@@ -49,12 +48,8 @@
'Wrong value for argument --content-type: %s'
% self.options.content_type)
-if config.distributionmirrorprober.use_proxy:
-os.environ['http_proxy'] = config.launchpad.http_proxy
-self.logger.debug("Using %s as proxy." % os.environ['http_proxy'])
-else:
-self.logger.debug("Not using any proxy.")
-
+set_default_timeout_function(
+lambda: config.distributionmirrorprober.timeout)
DistroMirrorProber(self.txn, self.logger).probe(
content_type, self.options.no_remote_hosts, self.options.force,
self.options.max_mirrors, not self.options.no_owner_notification)
=== modified file 'lib/lp/registry/scripts/distributionmirror_prober.py'
--- lib/lp/registry/scripts/distributionmirror_prober.py 2017-05-22 09:33:45 +
+++ lib/lp/registry/scripts/distributionmirror_prober.py 2018-06-26 15:14:17 +
@@ -1,4 +1,4 @@
-# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2018 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
__metaclass__ = type
@@ -10,12 +10,12 @@
import httplib
import itertools
import logging
-import os
+import os.path
from StringIO import StringIO
import urllib
-import urllib2
import urlparse
+import requests
from twisted.internet import (
defer,
protocol,
@@ -36,6 +36,7 @@
from lp.registry.interfaces.distroseries import IDistroSeries
from lp.services.config import config
from lp.services.librarian.interfaces import ILibraryFileAliasSet
+from lp.services.timeout import urlfetch
from lp.services.webapp import canonical_url
from lp.soyuz.interfaces.distroarchseries import IDistroArchSeries
@@ -202,9 +203,9 @@
request_path = None
# Details of the URL of the host in which we'll connect, which will only
-# be different from request_* in case we have an http_proxy environment
-# variable --in that case the scheme, host and port will be the ones
-# extracted from http_proxy and the path will be self.url
+# be different from request_* in case we have a configured http_proxy --
+# in that case the scheme, host and port will be the ones extracted from
+# http_proxy and the path will be self.url.
connect_scheme = None
connect_host = None
connect_port = None
@@ -279,7 +280,7 @@
# XXX Guilherme Salgado 2006-09-19:
# We don't actually know how to handle FTP responses, but we
# expect to be behind a squid HTTP proxy with the patch at
-# http://www.squid-cache.org/bugs/show_bug.cgi?id=1758 applied. So, if
+# https://bugs.squid-cache.org/show_bug.cgi?id=1758 applied. So, if
# you encounter any problems with FTP URLs you'll probably have to nag
# the sysadmins to fix squid for you.
if scheme not in ('http', 'ftp'):
@@ -296,9 +297,9 @@
if self.request_host not in host_timeouts:
host_timeouts[self.request_host] = 0
-# If the http_proxy variable is set, we want to use it as the host
-# we're going to connect to.
-proxy = os.getenv('http_proxy')
+# If launchpad.http_proxy is set in our configuration, we want to
+# use it as the host we're going to connect to.
+proxy = config.launchpad.http_proxy
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-sitesearch into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-sitesearch into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-sitesearch/+merge/348437 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/no-default-http-proxy into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/no-default-http-proxy into lp:launchpad. Commit message: Don't set http_proxy by default. Requested reviews: Launchpad code reviewers (launchpad-reviewers) For more details, see: https://code.launchpad.net/~cjwatson/launchpad/no-default-http-proxy/+merge/348540 It's now set in lp-production-configs for deployments on Canonical networks, and anyone else is unlikely to want squid.internal. -- Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/no-default-http-proxy into lp:launchpad. === modified file 'lib/lp/services/config/schema-lazr.conf' --- lib/lp/services/config/schema-lazr.conf 2018-06-06 12:46:56 + +++ lib/lp/services/config/schema-lazr.conf 2018-06-26 15:01:25 + @@ -915,7 +915,7 @@ # not the Launchpad section? At the moment, there are only scripts # using this. # datatype: string -http_proxy: http://squid.internal:3128/ +http_proxy: none # Session cookies being sent to a subdomain of the parent # domains listed here will be sent to the parent domain, ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-bugtracker-cleanup into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-bugtracker-cleanup into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-bugtracker-cleanup/+merge/348436 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-trac into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-trac into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-trac/+merge/348433 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-bugzilla into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-bugzilla into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-bugzilla/+merge/348432 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-mantis into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-mantis into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-mantis/+merge/348431 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-sourceforge into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-sourceforge into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-sourceforge/+merge/348430 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-rt into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-rt into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-rt/+merge/348429 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-roundup into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-roundup into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-roundup/+merge/348428 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-github into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/explicit-proxy-github into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-github/+merge/348427 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/snap-parse-architectures into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/snap-parse-architectures/+merge/347998 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/snap-initial-name-bzr into lp:launchpad
Review: Approve code
Diff comments:
>
> === modified file 'lib/lp/code/model/branch.py'
> --- lib/lp/code/model/branch.py 2018-06-14 17:09:31 +
> +++ lib/lp/code/model/branch.py 2018-06-14 17:09:31 +
> @@ -783,6 +789,63 @@
> RevisionAuthor, revisions, ['revision_author_id'])
> return DecoratedResultSet(result, pre_iter_hook=eager_load)
>
> +def getBlob(self, filename, revision_id=None, enable_memcache=None,
> +logger=None):
> +"""See `IBranch`."""
> +hosting_client = getUtility(IBranchHostingClient)
> +if enable_memcache is None:
> +enable_memcache = not getFeatureFlag(
> +u'code.bzr.blob.disable_memcache')
> +if revision_id is None:
> +revision_id = self.last_scanned_id
> +if revision_id is None:
> +# revision_id may still be None if the branch scanner hasn't
> +# scanned this branch yet. In this case, we won't be able to
> +# guarantee that subsequent calls to this method within the same
> +# transaction with revision_id=None will see the same revision,
> +# and we won't be able to cache file lists. Neither is fatal,
> +# and this should be relatively rare.
> +enable_memcache = False
> +dirname = os.path.dirname(filename)
> +unset = object()
> +file_list = unset
Can you just use None here, since it's otherwise unused?
> +if enable_memcache:
> +memcache_client = getUtility(IMemcacheClient)
> +instance_name = urlsplit(
> +config.codehosting.internal_bzr_api_endpoint).hostname
> +memcache_key = '%s:bzr-file-list:%s:%s:%s' % (
> +instance_name, self.id, revision_id, dirname)
> +if not isinstance(memcache_key, bytes):
> +memcache_key = memcache_key.encode('UTF-8')
> +cached_file_list = memcache_client.get(memcache_key)
> +if cached_file_list is not None:
> +try:
> +file_list = json.loads(cached_file_list)
> +except Exception:
> +logger.exception(
> +'Cannot load cached file list for %s:%s:%s;
> deleting' %
> +(self.unique_name, revision_id, dirname))
> +memcache_client.delete(memcache_key)
> +if file_list is unset:
> +try:
> +inventory = hosting_client.getInventory(
> +self.unique_name, dirname, rev=revision_id)
> +file_list = {
> +entry['filename']: entry['file_id']
> +for entry in inventory['filelist']}
> +except BranchFileNotFound:
> +file_list = None
> +if enable_memcache:
> +# Cache the file list in case there's a request for another
> +# file in the same directory.
> +memcache_client.set(memcache_key, json.dumps(file_list))
> +file_id = (file_list or {}).get(os.path.basename(filename))
> +if file_id is None:
> +raise BranchFileNotFound(
> +self.unique_name, filename=filename, rev=revision_id)
> +return hosting_client.getBlob(
> +self.unique_name, file_id, rev=revision_id)
> +
> def getDiff(self, new, old=None):
> """See `IBranch`."""
> hosting_client = getUtility(IBranchHostingClient)
>
> === modified file 'lib/lp/snappy/browser/snap.py'
> --- lib/lp/snappy/browser/snap.py 2018-04-21 10:01:22 +
> +++ lib/lp/snappy/browser/snap.py 2018-06-14 17:09:31 +
> @@ -424,34 +422,19 @@
> @property
> def initial_values(self):
> store_name = None
> -if self.has_snappy_distro_series and
> IGitRef.providedBy(self.context):
> +if self.has_snappy_distro_series:
> # Try to extract Snap store name from snapcraft.yaml file.
> try:
> -paths = (
> -'snap/snapcraft.yaml',
> -'snapcraft.yaml',
> -'.snapcraft.yaml',
> -)
> -for i, path in enumerate(paths):
> -try:
> -blob = self.context.repository.getBlob(
> -path, self.context.name)
> -break
> -except GitRepositoryBlobNotFound:
> -if i == len(paths) - 1:
> -raise
> -# Beware of unsafe yaml.load()!
> -store_name = yaml.safe_load(blob).get('name')
> -except GitRepositoryScanFault:
> -log.exception("Failed to get Snap manifest from Git %s",
> - self.context.unique_name)
> -except (AttributeError, yaml.YAMLError):
> -
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-sitesearch into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-sitesearch/+merge/348437 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-bugtracker-cleanup into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-bugtracker-cleanup/+merge/348436 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-trac into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-trac/+merge/348433 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-bugzilla into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-bugzilla/+merge/348432 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-mantis into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-mantis/+merge/348431 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-github into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-github/+merge/348427 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-rt into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-rt/+merge/348429 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-sourceforge into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-sourceforge/+merge/348430 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/explicit-proxy-roundup into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/explicit-proxy-roundup/+merge/348428 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
