== PostgreSQL Weekly News - December 28 2014 == PGCon 2015 (June 16-20) call for papers is out. Talk submission ends January 19, 2014. Details at http://www.pgcon.org/2015/papers.php
== PostgreSQL Product News == Bucardo 5.3.0, a multiple source replication system for Postgres, released: https://mail.endcrypt.com/pipermail/bucardo-announce/2014/000030.html == PostgreSQL Jobs for December == http://archives.postgresql.org/pgsql-jobs/2014-12/threads.php == PostgreSQL Local == FOSDEM PGDay (January 30, 2015) and the FOSDEM PostgreSQL Dev Room (January 31-February 1, 2015) http://fosdem2015.pgconf.eu/ Prague PostgreSQL Developer Day (P2D2) 2015 will be in Prague, Czech Republic February 11-12, 2015. http://www.p2d2.cz/ The Melbourne PostgreSQL meetup on February 18, 2015 will be hosting Gabriele Bartolini on PostgreSQL 9.4 for devops. Details below, and R, SVP. http://www.meetup.com/melpug/events/219082475/ pgDaySF 2015 will be held March 10, 2015 in Burlingame, California. http://sfpostgres.org/pgday-sf-2015-call-for-speakers-and-sponsors/ The CfP is open for Nordic PostgreSQL Day 2015, which will be held March 11, 2015 in Copenhagen, Denmark. http://2015.nordicpgday.org/cfp/ The CfP for PGConf US 2015 is open through December 17th, 2014 Notifications will go out on January 10, 2014. The event takes place March 25-27, 2015 in NYC. http://nyc.pgconf.us/2015/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to da...@fetter.org, German language to p...@pgug.de, Italian language to p...@itpug.org. Spanish language to p...@arpug.com.ar. == Applied Patches == Heikki Linnakangas pushed: - Use a pairing heap for the priority queue in kNN-GiST searches. This performs slightly better, uses less memory, and needs slightly less code in GiST, than the Red-Black tree previously used. Reviewed by Peter Geoghegan http://git.postgresql.org/pg/commitdiff/e7032610f76e6c8345496ae7bbdf49d3c40df30f - Turn much of the btree_gin macros into real functions. This makes the functions much nicer to read and edit, and also makes debugging easier. http://git.postgresql.org/pg/commitdiff/7f0dccaed64a8ed6f5db8ad43e7612202fbeeeaf - Move rbtree.c from src/backend/utils/misc to src/backend/lib. We have other general-purpose data structures in src/backend/lib, so it seems like a better home for the red-black tree as well. http://git.postgresql.org/pg/commitdiff/955557ddccb4065831af80b0966cbd02937dfb72 Andrew Dunstan pushed: - Further tidy up on json aggregate documentation http://git.postgresql.org/pg/commitdiff/2a3f2743f200dbcdd412a00ff69f30d3e34b662b - Fix documentation of argument type of json_agg and jsonb_agg. json_agg was originally designed to aggregate records. However, it soon became clear that it is useful for aggregating all kinds of values and that's what we have on 9.3 and 9.4, and in head for it and jsonb_agg. The documentation suggested otherwise, so this fixes it. http://git.postgresql.org/pg/commitdiff/b2d145bff02e5f278649aca0e6fb6142ea2f298d - Fix installcheck case for tap tests http://git.postgresql.org/pg/commitdiff/3f37b6c3166b2dc1956de8b0fb046ea6e7320d52 Peter Eisentraut pushed: - Change local_preload_libraries to PGC_USERSET. This allows it to be used with ALTER ROLE SET. Although the old setting of PGC_BACKEND prevented changes after session start, after discussion it was more useful to allow ALTER ROLE SET instead and just document that changes during a session have no effect. This is similar to how session_preload_libraries works already. An alternative would be to change things to allow PGC_BACKEND and PGC_SU_BACKEND settings to be changed by ALTER ROLE SET. But that might need further research (e.g., log_connections would probably not work). based on patch by Kyotaro Horiguchi http://git.postgresql.org/pg/commitdiff/584e35d17c00b69d598522d3cf321fbd9b7dbd03 Álvaro Herrera pushed: - get_object_address: separate domain constraints from table constraints. Apart from enabling comments on domain constraints, this enables a future project to replicate object dropping to remote servers: with the current mechanism there's no way to distinguish between the two types of constraints, so there's no way to know what to drop. Also added support for the domain constraint comments in psql's \dd and pg_dump. Catalog version bumped due to the change in ObjectType enum. http://git.postgresql.org/pg/commitdiff/7eca575d1c28f6eee2bf4564f3d458d10c4a8f47 - Use a bitmask to represent role attributes. The previous representation using a boolean column for each attribute would not scale as well as we want to add further attributes. Extra auxilliary functions are added to go along with this change, to make up for the lost convenience of access of the old representation. Catalog version bumped due to change in catalogs and the new functions. Author: Adam Brightwell, minor tweaks by Álvaro Reviewed by: Stephen Frost, Andres Freund, Álvaro Herrera http://git.postgresql.org/pg/commitdiff/1826987a46d079458007b7b6bbcbbd852353adbb - Add SQL-callable pg_get_object_address. This allows access to get_object_address from SQL, which is useful to obtain OID addressing information from data equivalent to that emitted by the parser. This is necessary infrastructure of a project to let replication systems propagate object dropping events to remote servers, where the schema might be different than the server originating the DROP. This patch also adds support for OBJECT_DEFAULT to get_object_address; that is, it is now possible to refer to a column's default value. Catalog version bumped due to the new function. Reviewed by Stephen Frost, Heikki Linnakangas, Robert Haas, Andres Freund, Abhijit Menon-Sen, Adam Brightwell. http://git.postgresql.org/pg/commitdiff/d7ee82e50f624221db76023c17137661fe69ec61 - Revert "Use a bitmask to represent role attributes". This reverts commit 1826987a46d079458007b7b6bbcbbd852353adbb. The overall design was deemed unacceptable, in discussion following the previous commit message; we might find some parts of it still salvageable, but I don't want to be on the hook for fixing it, so let's wait until we have a new patch. http://git.postgresql.org/pg/commitdiff/a609d96778c1714b9af916477b2c30891fbe578a - Grab heavyweight tuple lock only before sleeping. We were trying to acquire the lock even when we were subsequently not sleeping in some other transaction, which opens us up unnecessarily to deadlocks. In particular, this is troublesome if an update tries to lock an updated version of a tuple and finds itself doing EvalPlanQual update chain walking; more than two sessions doing this concurrently will find themselves sleeping on each other because the HW tuple lock acquisition in heap_lock_tuple called from EvalPlanQualFetch races with the same tuple lock being acquired in heap_update -- one of these sessions sleeps on the other one to finish while holding the tuple lock, and the other one sleeps on the tuple lock. Per trouble report from Andrew Sackville-West in http://www.postgresql.org/message-id/20140731233051.GN17765@andrew-ThinkPad-X230 His scenario can be simplified down to a relatively simple isolationtester spec file which I don't include in this commit; the reason is that the current isolationtester is not able to deal with more than one blocked session concurrently and it blocks instead of raising the expected deadlock. In the future, if we improve isolationtester, it would be good to include the spec file in the isolation schedule. I posted it in http://www.postgresql.org/message-id/20141212205254.gc1...@alvh.no-ip.org Hat tip to Mark Kirkwood, who helped diagnose the trouble. http://git.postgresql.org/pg/commitdiff/0e5680f4737a9c6aa94aa9e77543e5de60411322 - Remove event trigger from object_address test. It is causing trouble when run in parallel mode, because dropping the function other sessions are running concurrently causes them to fail due to inability to find the function. Per buildfarm, as noted by Tom Lane. http://git.postgresql.org/pg/commitdiff/289121a452e3c255e4fd63dbb0252261605c2eaf - Restrict name list len for domain constraints. This avoids an ugly-looking "cache lookup failure" message. Ugliness pointed out by Andres Freund. http://git.postgresql.org/pg/commitdiff/6630420fc9139c3a1ee051bc2f2ae8fa32e71c62 Tom Lane pushed: - Remove failing collation case from object_address regression test. Per buildfarm, this test case does not yield consistent results. I don't think it's useful enough to figure out a workaround, either. http://git.postgresql.org/pg/commitdiff/3e2275355954cabf89ba044fe8557ef347bf3fb1 - Add CST (China Standard Time) to our lists of timezone abbreviations. For some reason this seems to have been missed when the lists in src/timezone/tznames/ were first constructed. We can't put it in Default because of the conflict with US CST, but we should certainly list it among the alternative entries in Asia.txt. (I checked for other oversights, but all the other abbreviations that are in current use according to the IANA files seem to be accounted for.) Noted while responding to bug #12326. http://git.postgresql.org/pg/commitdiff/5b89473d870dc2a9fec0926c5afccf53042dbb0a - Temporarily revert "Move pg_lzcompress.c to src/common." This reverts commit 60838df922345b26a616e49ac9fab808a35d1f85. That change needs a bit more thought to be workable. In view of the potentially machine-dependent stuff that went in today, we need all of the buildfarm to be testing those other changes. http://git.postgresql.org/pg/commitdiff/966115c305c297cf4a1ec5ebbd0a55be46ff33ad - Remove duplicate assignment in new pg_get_object_address() function. Noted by Coverity. http://git.postgresql.org/pg/commitdiff/9a11df14498434bbb5e16dfa7aecbf71bf24d196 Bruce Momjian pushed: - pgbench: remove odd trailing period in init progress output http://git.postgresql.org/pg/commitdiff/83bcc70459c596ceb9a90e96f011d24b88d93a85 Fujii Masao pushed: - Remove unused fields from ReindexStmt. fe263d1 changed the REINDEX logic so that those fields are not used at all, but forgot to remove them. Sawada Masahiko http://git.postgresql.org/pg/commitdiff/3b6ca123b5665044557154385a6be5d1df76c5e5 - Remove duplicate include of slot.h. Back-patch to 9.4, where this problem was added. http://git.postgresql.org/pg/commitdiff/4a5593197b0ddec913fcd7758d61e782ab5c4d59 - Move pg_lzcompress.c to src/common. Exposing compression and decompression APIs of pglz makes possible its use by extensions and contrib modules. pglz_decompress contained a call to elog to emit an error message in case of corrupted data. This function is changed to return a status code to let its callers return an error instead. This commit is required for upcoming WAL compression feature so that the WAL reader facility can decompress the WAL data by using pglz_decompress. Michael Paquier http://git.postgresql.org/pg/commitdiff/60838df922345b26a616e49ac9fab808a35d1f85 Andres Freund pushed: - Suppress MSVC warning in typeStringToTypeName function. MSVC doesn't realize ereport(ERROR) doesn't return. David Rowley http://git.postgresql.org/pg/commitdiff/cd5ebe1edd6d9c19caba406a90f626afec5d3f0a - Convert the PGPROC->lwWaitLink list into a dlist instead of open coding it. Besides being shorter and much easier to read it changes the logic in LWLockRelease() to release all shared lockers when waking up any. This can yield some significant performance improvements - and the fairness isn't really much worse than before, as we always allowed new shared lockers to jump the queue. http://git.postgresql.org/pg/commitdiff/7882c3b0b95640e361f1533fe0f2d02e4e5d8610 - Add capability to suppress CONTEXT: messages to elog machinery. Hiding context messages usually is not a good idea - except for rather verbose debugging/development utensils like LOG_DEBUG. There the amount of repeated context messages just bloat the log without adding information. http://git.postgresql.org/pg/commitdiff/570bd2b3fd20d3f8896f5e6a8133d6aed385ac86 - Lockless StrategyGetBuffer clock sweep hot path. StrategyGetBuffer() has proven to be a bottleneck in a number of buffer acquisition heavy workloads. To some degree this has already been alleviated by 5d7962c6, but it still can be quite a heavy bottleneck. The problem is that in unfortunate usage patterns a single StrategyGetBuffer() call will have to look at a large number of buffers - in turn making it likely that the process will be put to sleep while still holding the spinlock. Replace most of the usage of the buffer_strategy_lock spinlock for the clock sweep by a atomic nextVictimBuffer variable. That variable, modulo NBuffers, is the current hand of the clock sweep. The buffer clock-sweep then only needs to acquire the spinlock after a wraparound. And even then only in the process that did the wrapping around. That alleviates nearly all the contention on the relevant spinlock, although significant contention on the cacheline can still exist. Reviewed-By: Robert Haas and Amit Kapila. Discussion: 20141010160020.gg6...@alap3.anarazel.de, 20141027133218.ga2...@awork2.anarazel.de http://git.postgresql.org/pg/commitdiff/d72731a70450b5e7084991b9caa15cb58a2820df - Blindly fix a dtrace probe in lwlock.c for a removed local variable. Per buildfarm member locust. http://git.postgresql.org/pg/commitdiff/740a4ec7f4adccd56eced452580ccec7157fc8d8 - Improve LWLock scalability. The old LWLock implementation had the problem that concurrent lock acquisitions required exclusively acquiring a spinlock. Often that could lead to acquirers waiting behind the spinlock, even if the actual LWLock was free. The new implementation doesn't acquire the spinlock when acquiring the lock itself. Instead the new atomic operations are used to atomically manipulate the state. Only the waitqueue, used solely in the slow path, is still protected by the spinlock. Check lwlock.c's header for an explanation about the used algorithm. For some common workloads on larger machines this can yield significant performance improvements. Particularly in read mostly workloads. Reviewed-By: Amit Kapila and Robert Haas. Author: Andres Freund. Discussion: 20130926225545.gb26...@awork2.anarazel.de http://git.postgresql.org/pg/commitdiff/ab5194e6f617a9a9e7aadb3dd1cee948a42d0755 Noah Misch pushed: - Have config_sspi_auth() permit IPv6 localhost connections. Windows versions later than Windows Server 2003 map "localhost" to ::1. Account for that in the generated pg_hba.conf, fixing another oversight in commit f6dc6dd5ba54d52c0733aaafc50da2fbaeabb8b0. Back-patch to 9.0, like that commit. David Rowley and Noah Misch http://git.postgresql.org/pg/commitdiff/8d9cb0bc483431780aef483c7c110ab39e465958 == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Michael Paquier sent in another revision of a patch to clarify some points about how numeric functions and operators work. Etsuro Fujita sent in a patch to fix the fact that ExplainModifyTarget does not always show the parent of the inheritance tree in multi-target-table cases. Heikki Linnakangas sent in another revision of a patch to advance local xmin more aggressively using the new pairing heap code. Robert Haas sent in another revision of a patch to implement parallel mode and parallel contexts. Petr (PJMODOS) Jelinek sent in another revision of a patch to implement TABLESAMPLE. Christoph Berg sent in a patch to correct the misapprehension that pg_upgrade requires "postmaster," an entity not present in all live versions of PostgreSQL, when "postgres" will do. Michael Paquier sent in two more revisions of a patch to move contrib binaries to bin. Álvaro Herrera sent in a flock of five patches intended to complete the feature which allows DROP to propagate via WAL. Etsuro Fujita sent in two more revisions of a patch to allow foreign tables to be part of table inheritance hierarchies. Heikki Linnakangas and Teodor Sigaev traded patches to support ranges with B-Tree GIN indexes. Heikki Linnakangas sent in another revision of a patch to add a compress method for SP-GiST. Andres Freund sent in another revision of a patch to add the capability to suppress CONTEXT: messages to elog machinery. Andres Freund sent in another revision of a patch to convert the PGPROC->lwWaitLink list into a dlist instead of open coding it, and one to implement wait-free LW_SHARED LWLock acquisition. Andres Freund sent in a patch to add a lockless StrategyGetBuffer clock sweep hotpath. Michael Paquier sent in a patch to install libpq.dll in bin directory on Windows / Cygwin. Andrew Dunstan sent in a patch to speed up the bin checks. Teodor Sigaev sent in another revision of a patch to speed up tidbitmap using a cache page. Jim Nasby sent in a patch to add a hack to test out several hashing strategies. Andreas Karlsson sent in another revision of a patch to use 128-bit integers for sum, avg and statistics aggregates. Bruce Momjian sent in a patch to fix an issue where misaligned BufferDescriptors is causing major performance problems on AMD. Jeff Davis sent in another revision of a patch to implement spill-to-disk for HashAggs. Alexander Shulgin sent in another revision of a patch to turn recovery.conf into GUCs. Kaigai Kouhei sent in another revision of a patch to add a CTID scan as an example of the custom scan capability. David Rowley and Noah Misch traded patches to update Windows with the changes that secured "make check." Abhijit Menon-Sen sent in a patch to use CPUID at startup to determine whether the SSE4.2 CRC instructions are available, and to use them instead of the slice-by-8 implementation if they are. Fujii Masao and Michael Paquier traded patches implementing compression of full-page writes. Abhijit Menon-Sen sent in a patch to implement a fast bloat measurement tool. Alexander Shulgin and Pavel Stehule traded patches to allow COPY to handle wrong rows without rolling back entirely. Alexey Vasiliev sent in a patch to add recovery_timeout option to control timeout of restore_command nonzero status code. Peter Geoghegan sent in another revision of a patch to implement INSERT ... ON CONFLICT {UPDATE | IGNORE}. Ian Lawrence Barwick sent in a patch to fix psql's tab completion of COMMENT ON not to repeat IS more than needed. Amit Kapila and Dilip Kumar traded patches to enable vacuumdb to use parallel cores. -- Sent via pgsql-announce mailing list (pgsql-announce@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-announce
