Re: [Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/postgresql-10-db into lp:launchpad
Review: Approve db -- https://code.launchpad.net/~wgrant/launchpad/postgresql-10-db/+merge/345516 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/no-tsearch2-residue into lp:launchpad
William Grant has proposed merging lp:~wgrant/launchpad/no-tsearch2-residue
into lp:launchpad with lp:~wgrant/launchpad/no-tsearch2 as a prerequisite.
Commit message:
Drop search_path and ts2 reside. All our stuff is now in public.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~wgrant/launchpad/no-tsearch2-residue/+merge/345521
--
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~wgrant/launchpad/no-tsearch2-residue into lp:launchpad.
=== modified file 'database/replication/Makefile'
--- database/replication/Makefile 2017-11-16 03:18:49 +
+++ database/replication/Makefile 2018-05-14 13:14:29 +
@@ -59,7 +59,7 @@
# XXX wgrant 2017-11-16: Exclude tsearch2 bits that are
# erreonously being dumped now.
pg_restore --list ${STAGING_DUMP} | grep -v 'TRIGGER public _sl_' \
- | grep -v 'OPERATOR FAMILY ts2' > ${DUMPLIST}
+ > ${DUMPLIST}
# Deny new connections to the main DBs and kill any leftovers.
${STAGING_PGBOUNCER} -c 'DISABLE launchpad_staging'
=== modified file 'database/schema/Makefile'
--- database/schema/Makefile 2012-11-21 17:43:58 +
+++ database/schema/Makefile 2018-05-14 13:14:29 +
@@ -135,23 +135,9 @@
# Confirm that launchpad-XX-00-0.sql hasn't been messed with - this file
# is our baseline telling us what was installed into production
-check: search_path
+check:
@if [ "`md5sum ${BASELINE}`" != "${MD5SUM}" ]; then echo "* ${BASELINE} is corrupt or has been modified"; exit 1; else echo "* Using ${BASELINE} as baseline"; fi
-search_path:
- @psql -d template1 -q -A -t -c 'show search_path'
- @if [ `psql -d template1 -q -A -t -c 'show search_path'` != '$$user,public,ts2' ] && [ `psql -d template1 -q -A -t -c 'show search_path'` != '"$$user",public,ts2' ]; then \
- echo "* It appears your search path is unconfigured."; \
- echo " Have you read ?"; \
- echo; \
- echo "* Add the following to /etc/postgresql/X.X/main/postgresql.conf"; \
- echo " (where X.X is the version of the PostgreSQL DB you're connecting to):"; \
- echo "search_path='\$$user,public,ts2'"; \
- echo "* Then reload PostgreSQL:"; \
- echo "sudo /etc/init.d/postgresql reload"; \
- exit 1; \
- fi
-
all: dev test
@ echo "* All done"
@@ -172,5 +158,5 @@
$(call build_new_sampledata,${TEMPLATE_WITH_TEST_SAMPLEDATA},${LINTDATA})
$(call build_new_sampledata,${TEMPLATE_WITH_DEV_SAMPLEDATA},${LINTDATA_DEV})
-.PHONY: default test dev create check search_path all doc diagram newsampledata_test newsampledata_dev newsampledata lintdata
+.PHONY: default test dev create check all doc diagram newsampledata_test newsampledata_dev newsampledata lintdata
=== modified file 'lib/lp/services/database/doc/textsearching.txt'
--- lib/lp/services/database/doc/textsearching.txt 2018-05-14 09:25:45 +
+++ lib/lp/services/database/doc/textsearching.txt 2018-05-14 13:14:29 +
@@ -3,8 +3,7 @@
Introduction
-We are currently using tsearch2 to provide full text searching
-capabilities inside the PostgreSQL database.
+We are currently using PostgreSQL's built-in full text search capabilities.
Although in a lot of cases simple substring searches using LIKE would be
fine or even preferred, the disadvantage of using LIKE is that PostgreSQL
@@ -12,31 +11,6 @@
when we need to search tens of thousands of entries (packages) or hundreds
of thousands of entries (people) or millions of entries (translations).
-Installation
-
-tsearch2 is part of the `postgresql-contrib` package. This is installed
-into the launchpad database using the script `database/schema/fti.py`,
-which is run automatically during database creation.
-
-This script does the following:
-
-* Hooks `tsearch2` into the launchpad database, installing it into
- the ts2 schema.
-
-* Creates all the full text indexes. These are in the form of a new
- column called `fti` on a number of tables. Which tables and
- which columns are indexed is defined inside the fti.py script.
-
-* Creates the `ftq(text)` stored procedure. This procedure makes
- queries against the indexes simpler and consistant throughout
- our applications.
-
-It is the Database Administrator's responsibility to rerun `fti.py` as
-part of the production rollouts if this script has been modified. It does
-not need to be rerun if the script has not changed. Rebuilding the indexes
-currently takes several minutes on production, during which time the
-production database is unavailable.
-
Querying
=== modified file 'test_on_merge.py'
--- test_on_merge.py 2014-01-15 10:46:59 +
+++ test_on_merge.py 2018-05-14 13:14:29 +
@@ -120,14 +120,6 @@
# bedrock is crumbling.
con = psycopg2.connect('dbname=launchpad_ftest_template')
cur = con.cursor()
-cur.execute('show search_path')
-search_path = cur.fetchone()[0]
-
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/upgrade-python-debian into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/upgrade-python-debian into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/upgrade-python-debian/+merge/345511 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/postgresql-10-db into lp:launchpad
William Grant has proposed merging lp:~wgrant/launchpad/postgresql-10-db into lp:launchpad with lp:~wgrant/launchpad/no-tsearch2 as a prerequisite. Commit message: Update activity() for PostgreSQL 10 support, in addition to 9.6, 9.5 and 9.3. Requested reviews: Launchpad code reviewers (launchpad-reviewers) For more details, see: https://code.launchpad.net/~wgrant/launchpad/postgresql-10-db/+merge/345516 Update activity() for PostgreSQL 10 support, in addition to 9.6, 9.5 and 9.3. -- Your team Launchpad code reviewers is requested to review the proposed merge of lp:~wgrant/launchpad/postgresql-10-db into lp:launchpad. === added file 'database/schema/patch-2209-84-1.sql' --- database/schema/patch-2209-84-1.sql 1970-01-01 00:00:00 + +++ database/schema/patch-2209-84-1.sql 2018-05-14 11:26:32 + @@ -0,0 +1,94 @@ +-- Copyright 2018 Canonical Ltd. This software is licensed under the +-- GNU Affero General Public License version 3 (see the file LICENSE). + +SET client_min_messages = ERROR; + +-- Update functions for PostgreSQL 10 support, in addition to 9.6, 9.5 and 9.3. + +-- From 2209-81-0 +CREATE OR REPLACE FUNCTION activity() +RETURNS SETOF pg_stat_activity +VOLATILE SECURITY DEFINER SET search_path = public +LANGUAGE plpgsql AS $$ +DECLARE +a pg_stat_activity%ROWTYPE; +BEGIN +IF EXISTS ( +SELECT 1 FROM pg_attribute WHERE +attrelid = +(SELECT oid FROM pg_class + WHERE relname = 'pg_stat_activity') +AND attname = 'backend_type') THEN +-- >= 10 +RETURN QUERY SELECT +datid, datname, pid, usesysid, usename, application_name, +client_addr, client_hostname, client_port, backend_start, +xact_start, query_start, state_change, wait_event_type, +wait_event, state, backend_xid, backend_xmin, backend_type, +CASE +WHEN query LIKE '%' +OR query LIKE 'autovacuum:%' +THEN query +ELSE +'' +END AS query +FROM pg_catalog.pg_stat_activity; +ELSIF EXISTS ( +SELECT 1 FROM pg_attribute WHERE +attrelid = +(SELECT oid FROM pg_class + WHERE relname = 'pg_stat_activity') +AND attname = 'wait_event_type') THEN +-- >= 9.6 +RETURN QUERY SELECT +datid, datname, pid, usesysid, usename, application_name, +client_addr, client_hostname, client_port, backend_start, +xact_start, query_start, state_change, wait_event_type, +wait_event, state, backend_xid, backend_xmin, +CASE +WHEN query LIKE '%' +OR query LIKE 'autovacuum:%' +THEN query +ELSE +'' +END AS query +FROM pg_catalog.pg_stat_activity; +ELSIF EXISTS ( +SELECT 1 FROM pg_attribute WHERE +attrelid = +(SELECT oid FROM pg_class + WHERE relname = 'pg_stat_activity') +AND attname = 'backend_xid') THEN +-- >= 9.4 +RETURN QUERY SELECT +datid, datname, pid, usesysid, usename, application_name, +client_addr, client_hostname, client_port, backend_start, +xact_start, query_start, state_change, waiting, state, +backend_xid, backend_xmin, +CASE +WHEN query LIKE '%' +OR query LIKE 'autovacuum:%' +THEN query +ELSE +'' +END AS query +FROM pg_catalog.pg_stat_activity; +ELSE +-- >= 9.2; anything older is unsupported +RETURN QUERY SELECT +datid, datname, pid, usesysid, usename, application_name, +client_addr, client_hostname, client_port, backend_start, +xact_start, query_start, state_change, waiting, state, +CASE +WHEN query LIKE '%' +OR query LIKE 'autovacuum:%' +THEN query +ELSE +'' +END AS query +FROM pg_catalog.pg_stat_activity; +END IF; +END; +$$; + +INSERT INTO LaunchpadDatabaseRevision VALUES (2209, 84, 1); ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/upgrade-python-debian into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/upgrade-python-debian into lp:launchpad. Commit message: Upgrade to python-debian 0.1.32. Requested reviews: William Grant (wgrant): code For more details, see: https://code.launchpad.net/~cjwatson/launchpad/upgrade-python-debian/+merge/345511 This also adds chardet (which python-debian now depends on) so that we don't need to depend on it in launchpad-dependencies. -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. === modified file 'constraints.txt' --- constraints.txt 2018-05-14 09:33:34 + +++ constraints.txt 2018-05-14 11:15:26 + @@ -236,6 +236,7 @@ celery==3.1.26.post2 cffi==1.11.2 Chameleon==2.11 +chardet==3.0.4 constantly==15.1.0 cryptography==2.1.4 cssselect==0.9.1 @@ -322,7 +323,7 @@ pyOpenSSL==17.5.0 pystache==0.5.3 python-dateutil==1.5 -python-debian==0.1.23 +python-debian==0.1.32 python-keystoneclient==0.7.1 python-memcached==1.58 python-mimeparse==0.1.4 ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/upgrade-python-debian into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/upgrade-python-debian/+merge/345511 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/code-pagetests-future-imports into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/code-pagetests-future-imports into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/code-pagetests-future-imports/+merge/345472 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/code-doctests-future-imports into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/code-doctests-future-imports into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/code-doctests-future-imports/+merge/345470 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/email-force-strong-auth into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/email-force-strong-auth into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/email-force-strong-auth/+merge/343790 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/no-tsearch2 into lp:launchpad
William Grant has proposed merging lp:~wgrant/launchpad/no-tsearch2 into
lp:launchpad.
Commit message:
Drop the tsearch2 extension and ts2 schema.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~wgrant/launchpad/no-tsearch2/+merge/345508
PostgreSQL 10 removes contrib/tsearch2.
>From the extension only ts2.tsvector survives, as
public.ts2_tsvector. It's an alias for pg_catalog.tsvector, but we
can't change existing column types without table rewrites or catalog
hackery.
ts2.ftiupdate() and the ts2.default text search config move into
public.
--
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~wgrant/launchpad/no-tsearch2 into lp:launchpad.
=== added file 'database/schema/patch-2209-84-0.sql'
--- database/schema/patch-2209-84-0.sql 1970-01-01 00:00:00 +
+++ database/schema/patch-2209-84-0.sql 2018-05-14 10:47:32 +
@@ -0,0 +1,72 @@
+-- Copyright 2018 Canonical Ltd. This software is licensed under the
+-- GNU Affero General Public License version 3 (see the file LICENSE).
+
+SET client_min_messages=ERROR;
+
+-- Drop the tsearch2 extension and ts2 schema.
+--
+-- PostgreSQL 10 removes contrib/tsearch2.
+--
+-- From the extension only ts2.tsvector survives, as
+-- public.ts2_tsvector. It's an alias for pg_catalog.tsvector, but we
+-- can't change existing column types without table rewrites or catalog
+-- hackery.
+--
+-- ts2.ftiupdate() and the ts2.default text search config move into
+-- public.
+
+ALTER EXTENSION tsearch2 DROP DOMAIN ts2.tsvector;
+DROP EXTENSION tsearch2;
+
+ALTER DOMAIN ts2.tsvector RENAME TO ts2_tsvector;
+ALTER DOMAIN ts2.ts2_tsvector SET SCHEMA public;
+
+ALTER TEXT SEARCH CONFIGURATION ts2.default SET SCHEMA public;
+
+ALTER FUNCTION ts2.ftiupdate() SET SCHEMA public;
+
+-- From launchpad-2209-00-0.sql, but without ts2 schema specifiers since it's
+-- all in pg_catalog now.
+CREATE OR REPLACE FUNCTION ftiupdate() RETURNS trigger
+LANGUAGE plpythonu
+AS $_$
+new = TD["new"]
+args = TD["args"][:]
+
+# Short circuit if none of the relevant columns have been
+# modified and fti is not being set to NULL (setting the fti
+# column to NULL is thus how we can force a rebuild of the fti
+# column).
+if TD["event"] == "UPDATE" and new["fti"] != None:
+old = TD["old"]
+relevant_modification = False
+for column_name in args[::2]:
+if new[column_name] != old[column_name]:
+relevant_modification = True
+break
+if not relevant_modification:
+return "OK"
+
+# Generate an SQL statement that turns the requested
+# column values into a weighted tsvector
+sql = []
+for i in range(0, len(args), 2):
+sql.append(
+"setweight(to_tsvector('default', coalesce("
+"substring(ltrim($%d) from 1 for 2500),'')),"
+"CAST($%d AS \"char\"))" % (i + 1, i + 2))
+args[i] = new[args[i]]
+
+sql = "SELECT %s AS fti" % "||".join(sql)
+
+# Execute and store in the fti column
+plan = plpy.prepare(sql, ["text", "char"] * (len(args)/2))
+new["fti"] = plpy.execute(plan, args, 1)[0]["fti"]
+
+# Tell PostgreSQL we have modified the data
+return "MODIFY"
+$_$;
+
+DROP SCHEMA ts2;
+
+INSERT INTO LaunchpadDatabaseRevision VALUES (2209, 84, 0);
=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2018-04-10 13:07:25 +
+++ database/schema/security.cfg 2018-05-14 10:47:32 +
@@ -45,6 +45,7 @@
public.fticache=
public.ftq(text) = EXECUTE
public._ftq(text) = EXECUTE
+public.ftiupdate() = EXECUTE
public.getlocalnodeid()= EXECUTE
public.gitrepository_denorm_access(integer)= EXECUTE
public.is_blacklisted_name(text, integer) = EXECUTE
___
Mailing list: https://launchpad.net/~launchpad-reviewers
Post to : launchpad-reviewers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~launchpad-reviewers
More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/ts-rank into lp:launchpad
Review: Approve -- https://code.launchpad.net/~wgrant/launchpad/ts-rank/+merge/345504 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/psycopg2-2.7 into lp:launchpad
The proposal to merge lp:~wgrant/launchpad/psycopg2-2.7 into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~wgrant/launchpad/psycopg2-2.7/+merge/345500 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/fix-database-setup-version-check into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/fix-database-setup-version-check into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/fix-database-setup-version-check/+merge/345471 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/ts-rank into lp:launchpad
The proposal to merge lp:~wgrant/launchpad/ts-rank into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~wgrant/launchpad/ts-rank/+merge/345504 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/buildd-virtualenv into lp:launchpad
The proposal to merge lp:~wgrant/launchpad/buildd-virtualenv into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~wgrant/launchpad/buildd-virtualenv/+merge/345501 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/ts-rank into lp:launchpad
William Grant has proposed merging lp:~wgrant/launchpad/ts-rank into
lp:launchpad.
Commit message:
Switch all ts2.rank callsites to pg_catalog.ts_rank.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~wgrant/launchpad/ts-rank/+merge/345504
ts2 is long-deprecated and going away, and pg_catalog.rank is a window function.
--
Your team Launchpad code reviewers is requested to review the proposed merge of
lp:~wgrant/launchpad/ts-rank into lp:launchpad.
=== modified file 'lib/lp/registry/model/distribution.py'
--- lib/lp/registry/model/distribution.py 2016-11-12 21:47:32 +
+++ lib/lp/registry/model/distribution.py 2018-05-14 09:30:49 +
@@ -993,7 +993,7 @@
find_spec = (
DistributionSourcePackageCache,
SourcePackageName,
-SQL('rank(fti, ftq(?)) AS rank', params=(text,)),
+SQL('ts_rank(fti, ftq(?)) AS rank', params=(text,)),
)
origin = [
DistributionSourcePackageCache,
=== modified file 'lib/lp/registry/model/distroseries.py'
--- lib/lp/registry/model/distroseries.py 2018-01-26 13:47:51 +
+++ lib/lp/registry/model/distroseries.py 2018-05-14 09:30:49 +
@@ -1234,7 +1234,7 @@
find_spec = (
DistroSeriesPackageCache,
BinaryPackageName,
-SQL('rank(fti, ftq(%s)) AS rank' % sqlvalues(text)))
+SQL('ts_rank(fti, ftq(%s)) AS rank' % sqlvalues(text)))
origin = [
DistroSeriesPackageCache,
Join(
=== modified file 'lib/lp/registry/model/pillar.py'
--- lib/lp/registry/model/pillar.py 2017-11-10 14:07:52 +
+++ lib/lp/registry/model/pillar.py 2018-05-14 09:30:49 +
@@ -214,7 +214,7 @@
# row should get the highest search rank (999).
# Each row in the PillarName table will join with only one
# of either the Product, Project, or Distribution tables,
-# so the coalesce() is necessary to find the rank() which
+# so the coalesce() is necessary to find the ts_rank() which
# is not null.
result.order_by(SQL('''
(CASE WHEN PillarName.name = lower(%(text)s)
@@ -222,9 +222,9 @@
OR lower(Project.title) = lower(%(text)s)
OR lower(Distribution.title) = lower(%(text)s)
THEN 999
-ELSE coalesce(rank(Product.fti, ftq(%(text)s)),
- rank(Project.fti, ftq(%(text)s)),
- rank(Distribution.fti, ftq(%(text)s)))
+ELSE coalesce(ts_rank(Product.fti, ftq(%(text)s)),
+ ts_rank(Project.fti, ftq(%(text)s)),
+ ts_rank(Distribution.fti, ftq(%(text)s)))
END) DESC, PillarName.name
''' % sqlvalues(text=text)))
# People shouldn't be calling this method with too big limits
=== modified file 'lib/lp/registry/vocabularies.py'
--- lib/lp/registry/vocabularies.py 2016-09-19 13:44:28 +
+++ lib/lp/registry/vocabularies.py 2018-05-14 09:30:49 +
@@ -321,7 +321,7 @@
getUtility(ILaunchBag).user), *vocab_filter)
order_by = SQL(
'(CASE name WHEN %s THEN 1 '
-' ELSE rank(fti, ftq(%s)) END) DESC, displayname, name'
+' ELSE ts_rank(fti, ftq(%s)) END) DESC, displayname, name'
% (fti_query, fti_query))
return IStore(Product).find(self._table, where_clause).order_by(
order_by).config(limit=100)
@@ -628,7 +628,7 @@
when person.name like lower(?) || '%%' then 0.6
when lower(person.displayname) like lower(?)
|| '%%' then 0.5
-else rank(fti, ftq(?))
+else ts_rank(fti, ftq(?))
end) as rank
FROM Person
WHERE Person.name LIKE lower(?) || '%%'
=== modified file 'lib/lp/services/database/doc/textsearching.txt'
--- lib/lp/services/database/doc/textsearching.txt 2016-11-14 11:45:46 +
+++ lib/lp/services/database/doc/textsearching.txt 2018-05-14 09:30:49 +
@@ -563,11 +563,11 @@
---
We have ranking information stored in the indexes, as specified in fti.py.
-The rank of a result is calculated using the tsearch2 rank() function.
+The rank of a result is calculated using the ts_rank() function.
>>> runsql(r"""
... SELECT
-... name, rank(fti, ftq('gnome')) AS rank
+... name, ts_rank(fti, ftq('gnome')) AS rank
... FROM product
... WHERE fti @@ ftq('gnome')
... ORDER BY rank DESC, name
@@ -581,7 +581,7 @@
You can also build complex multi table queries and mush all the
ranked results together. This query does a full text search on
the Bug and Message tables, as well as substring name searches
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/psycopg2-2.7 into lp:launchpad
William Grant has proposed merging lp:~wgrant/launchpad/psycopg2-2.7 into lp:launchpad. Commit message: Upgrade to psycopg2 2.7.4. Needed to build against PostgreSQL 10. Requested reviews: Launchpad code reviewers (launchpad-reviewers) For more details, see: https://code.launchpad.net/~wgrant/launchpad/psycopg2-2.7/+merge/345500 I had to flip the order of the bzr/git namespace validation functions so they didn't end up sending NULs to the DB which now crashes psycopg2. Previously the string would apparently be truncated by postgres, which probably means this upgrade causes other paths to crash with ValueErrors in the face of bad input, but better than what we had before. -- Your team Launchpad code reviewers is requested to review the proposed merge of lp:~wgrant/launchpad/psycopg2-2.7 into lp:launchpad. === modified file 'constraints.txt' --- constraints.txt 2018-05-12 14:16:37 + +++ constraints.txt 2018-05-14 08:59:58 + @@ -311,7 +311,7 @@ pgbouncer==0.0.8 prettytable==0.7.2 psutil==5.4.2 -psycopg2==2.6.1 +psycopg2==2.7.4 pyasn1==0.4.2 pyasn1-modules==0.2.1 pycparser==2.18 === modified file 'lib/lp/code/model/branchnamespace.py' --- lib/lp/code/model/branchnamespace.py 2017-05-23 12:42:38 + +++ lib/lp/code/model/branchnamespace.py 2018-05-14 08:59:58 + @@ -194,10 +194,6 @@ def validateBranchName(self, name): """See `IBranchNamespace`.""" -existing_branch = self.getByName(name) -if existing_branch is not None: -raise BranchExists(existing_branch) - # Not all code paths that lead to branch creation go via a # schema-validated form (e.g. pushing a new branch to codehosting), # so we validate the branch name here to give a nicer error message @@ -205,6 +201,10 @@ # constraint "valid_name"...'. IBranch['name'].validate(unicode(name)) +existing_branch = self.getByName(name) +if existing_branch is not None: +raise BranchExists(existing_branch) + def validateMove(self, branch, mover, name=None): """See `IBranchNamespace`.""" if name is None: === modified file 'lib/lp/code/model/gitnamespace.py' --- lib/lp/code/model/gitnamespace.py 2016-10-14 17:25:51 + +++ lib/lp/code/model/gitnamespace.py 2018-05-14 08:59:58 + @@ -138,16 +138,16 @@ def validateRepositoryName(self, name): """See `IGitNamespace`.""" -existing_repository = self.getByName(name) -if existing_repository is not None: -raise GitRepositoryExists(existing_repository) - # Not all code paths that lead to Git repository creation go via a # schema-validated form, so we validate the repository name here to # give a nicer error message than 'ERROR: new row for relation # "gitrepository" violates check constraint "valid_name"...'. IGitRepository['name'].validate(unicode(name)) +existing_repository = self.getByName(name) +if existing_repository is not None: +raise GitRepositoryExists(existing_repository) + def validateDefaultFlags(self, repository): """See `IGitNamespace`.""" repository_set = getUtility(IGitRepositorySet) ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/psycopg2-2.7 into lp:launchpad
Review: Approve -- https://code.launchpad.net/~wgrant/launchpad/psycopg2-2.7/+merge/345500 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/buildd-virtualenv into lp:launchpad
William Grant has proposed merging lp:~wgrant/launchpad/buildd-virtualenv into lp:launchpad. Commit message: Fix the test suite to run buildd-slave.tac in its virtualenv. Requested reviews: Launchpad code reviewers (launchpad-reviewers) For more details, see: https://code.launchpad.net/~wgrant/launchpad/buildd-virtualenv/+merge/345501 Broke when launchpad-developer-dependencies stopped pulling in python-twisted-core via python-lpbuildd. -- Your team Launchpad code reviewers is requested to review the proposed merge of lp:~wgrant/launchpad/buildd-virtualenv into lp:launchpad. === modified file 'lib/lp/buildmaster/tests/mock_slaves.py' --- lib/lp/buildmaster/tests/mock_slaves.py 2018-01-26 13:47:51 + +++ lib/lp/buildmaster/tests/mock_slaves.py 2018-05-14 09:02:33 + @@ -20,6 +20,7 @@ ] import os +import sys import types import xmlrpclib @@ -37,6 +38,7 @@ from lp.buildmaster.interactor import BuilderSlave from lp.buildmaster.interfaces.builder import CannotFetchFile from lp.services.config import config +from lp.services.daemons.tachandler import twistd_script from lp.services.webapp import urlappend from lp.testing.sampledata import I386_ARCHITECTURE_NAME @@ -274,19 +276,28 @@ return defer.Deferred() +class LPBuilddSlaveTestSetup(BuilddSlaveTestSetup): +"""A BuilddSlaveTestSetup that uses the LP virtualenv.""" + +def setUp(self): +super(LPBuilddSlaveTestSetup, self).setUp( +python_path=sys.executable, +twistd_script=twistd_script) + + class SlaveTestHelpers(fixtures.Fixture): @property def base_url(self): """The URL for the XML-RPC service set up by `BuilddSlaveTestSetup`.""" -return 'http://localhost:%d' % BuilddSlaveTestSetup().daemon_port +return 'http://localhost:%d' % LPBuilddSlaveTestSetup().daemon_port def getServerSlave(self): """Set up a test build slave server. :return: A `BuilddSlaveTestSetup` object. """ -tachandler = self.useFixture(BuilddSlaveTestSetup()) +tachandler = self.useFixture(LPBuilddSlaveTestSetup()) self.addDetail( 'xmlrpc-log-file', Content( ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/archive-sourcefiles-private into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/archive-sourcefiles-private into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/archive-sourcefiles-private/+merge/345442 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~wgrant/launchpad/buildd-virtualenv into lp:launchpad
Review: Approve -- https://code.launchpad.net/~wgrant/launchpad/buildd-virtualenv/+merge/345501 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
[Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/celery-3.1.26 into lp:launchpad
The proposal to merge lp:~cjwatson/launchpad/celery-3.1.26 into lp:launchpad has been updated. Status: Needs review => Merged For more details, see: https://code.launchpad.net/~cjwatson/launchpad/celery-3.1.26/+merge/345462 -- Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/code-pagetests-future-imports into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/code-pagetests-future-imports/+merge/345472 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/celery-3.1.26 into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/celery-3.1.26/+merge/345462 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
Re: [Launchpad-reviewers] [Merge] lp:~cjwatson/launchpad/code-doctests-future-imports into lp:launchpad
Review: Approve code -- https://code.launchpad.net/~cjwatson/launchpad/code-doctests-future-imports/+merge/345470 Your team Launchpad code reviewers is subscribed to branch lp:launchpad. ___ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
