[HACKERS] Review of SQLDA support for ECPG

2009-10-04 Thread Noah Misch
I took a look at 2-pg85-sqlda-10-ctxdiff.patch. Starting from CVS HEAD of roughly 2009-10-03 05:00 UTC, prerequisite patches 1a-1h applied cleanly. 2-pg85-sqlda hit a trivial whitespace reject in ecpg.trailer along with a more substantive reject at ecpg.addons:407 (FetchStmtMOVEfetch_args).

Re: [HACKERS] Review of SQLDA support for ECPG

2009-10-07 Thread Noah Misch
On Mon, Oct 05, 2009 at 02:23:57PM +0200, Boszormenyi Zoltan wrote: Noah Misch ?rta: I will post a new patch for SQLDA and for all others that need updating. Thanks; that one does apply cleanly. The main test suite acquired no regressions, but I get failures in two tests of the ecpg

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-08 Thread Noah Misch
On Tue, Jul 07, 2009 at 09:31:14AM -0500, Kevin Grittner wrote: I don't remember any clear resolution to the wild variations in plan time mentioned here: http://archives.postgresql.org/pgsql-hackers/2009-06/msg00743.php I think it would be prudent to try to figure out why small changes

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-09 Thread Noah Misch
On Wed, Jul 08, 2009 at 05:23:16PM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: The expontential factor seems smaller for real queries. I have a query of sixteen joins that takes 71s to plan deterministically; it looks like this: SELECT 1 FROM fact JOIN dim0 ... JOIN dim6

Re: [HACKERS] *_collapse_limit, geqo_threshold

2009-07-09 Thread Noah Misch
On Wed, Jul 08, 2009 at 10:28:02AM -0500, Robert Haas wrote: On Jul 8, 2009, at 8:43 AM, Noah Misch n...@leadboat.com wrote: The expontential factor seems smaller for real queries. I have a query of sixteen joins that takes 71s to plan deterministically; it looks like this: SELECT 1 FROM

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2011-12-20 Thread Noah Misch
On Mon, Dec 19, 2011 at 11:13:57PM -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: I created a function that does this in a loop: HeapTuple t; CatalogCacheFlushCatalog(ProcedureRelationId); t = SearchSysCache1(PROCOID, ObjectIdGetDatum

Re: [HACKERS] RangeVarGetRelid()

2011-12-20 Thread Noah Misch
On Mon, Dec 19, 2011 at 11:52:54PM -0500, Robert Haas wrote: After staring at this for quite a while longer, it seemed to me that the logic for renaming a relation was similar enough to the logic for changing a schema that the two calbacks could reasonably be combined using a bit of

Re: [HACKERS] RangeVarGetRelid()

2011-12-21 Thread Noah Misch
On Wed, Dec 21, 2011 at 03:16:39PM -0500, Robert Haas wrote: On Tue, Dec 20, 2011 at 8:14 PM, Noah Misch n...@leadboat.com wrote: RangeVarCallbackForAlterRelation() does not preserve ALTER TABLE's refusal to operate on foreign tables. I should probably fix that, but I'm wondering if I

Re: [HACKERS] Collect frequency statistics for arrays

2011-12-27 Thread Noah Misch
On Tue, Dec 20, 2011 at 04:37:37PM +0400, Alexander Korotkov wrote: On Wed, Nov 16, 2011 at 1:43 AM, Nathan Boley npbo...@gmail.com wrote: FYI, I've added myself as the reviewer for the current commitfest. How is going review now? I will examine this patch within the week. -- Sent via

Re: [HACKERS] 16-bit page checksums for 9.2

2011-12-29 Thread Noah Misch
On Thu, Dec 29, 2011 at 11:08:43AM -0600, Kevin Grittner wrote: Heikki Linnakangas wrote: Simon Riggs wrote: OK, then we are talking at cross purposes. Double write buffers, in the way you explain them allow us to remove full page writes. They clearly don't do anything to check page

[HACKERS] get_fn_expr_argtype() vs. internal calls

2011-12-29 Thread Noah Misch
We document that a polymorphic C-language function may identify the concrete data type of each argument using calls to get_fn_expr_argtype(). That relies on FmgrInfo.fn_expr, which only the executor sets. Calls of internal origin, by way of {Direct,,Oid}FunctionCall*(), don't cons up an fn_expr,

[HACKERS] Add protransform for numeric, varbit, and temporal types

2011-12-31 Thread Noah Misch
Building on commit 8f9fe6edce358f7904e0db119416b4d1080a83aa, this adds protransform functions to the length coercions for numeric, varbit, timestamp, timestamptz, time, timetz and interval. This mostly serves to make more ALTER TABLE ALTER TYPE operations avoid a rewrite, including numeric(10,2)

Re: [HACKERS] pg_upgrade and relkind filtering

2011-12-31 Thread Noah Misch
On Mon, Dec 05, 2011 at 05:06:37PM -0500, Bruce Momjian wrote: Pg_upgrade has the following check to make sure the cluster is safe for upgrading: res = executeQueryOrDie(conn, SELECT n.nspname, c.relname, a.attname

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2012-01-02 Thread Noah Misch
On Mon, Jan 02, 2012 at 05:09:16PM +, Simon Riggs wrote: Attached patch makes SnapshotNow into an MVCC snapshot, initialised at the start of each scan iff SnapshotNow is passed as the scan's snapshot. It's fairly brief but seems to do the trick. That's a neat trick. However, if you start

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2012-01-02 Thread Noah Misch
On Mon, Jan 02, 2012 at 06:41:31PM +, Simon Riggs wrote: On Mon, Jan 2, 2012 at 6:06 PM, Noah Misch n...@leadboat.com wrote: On Mon, Jan 02, 2012 at 05:09:16PM +, Simon Riggs wrote: Attached patch makes SnapshotNow into an MVCC snapshot, initialised at the start of each scan iff

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2012-01-02 Thread Noah Misch
On Mon, Jan 02, 2012 at 04:33:28PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of lun ene 02 16:25:25 -0300 2012: On Mon, Jan 02, 2012 at 06:41:31PM +, Simon Riggs wrote: On Mon, Jan 2, 2012 at 6:06 PM, Noah Misch n...@leadboat.com wrote: On Mon, Jan 02, 2012

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2012-01-02 Thread Noah Misch
On Tue, Jan 03, 2012 at 01:18:41AM +, Simon Riggs wrote: Just for the record, yes we do run multiple catalog scans in some parts of the code. So I can see how we might trigger 4 nested scans, using cache replacement while scanning, so best assume more, with no guarantee of them being

Re: [HACKERS] Patch to allow users to kill their own queries

2012-01-03 Thread Noah Misch
On Tue, Dec 20, 2011 at 02:30:08PM +0100, Magnus Hagander wrote: That said - can someone who knows the translation stuff better than me comment on if this is actually going to be translatable, or if it violates too many translation rules? +pg_signal_backend(int pid, int sig, bool

Re: [HACKERS] Collect frequency statistics for arrays

2012-01-03 Thread Noah Misch
On Wed, Jan 04, 2012 at 12:09:16AM +0400, Alexander Korotkov wrote: Thanks for your great work on reviewing this patch. Now I'm trying to find memory corruption bug. Unfortunately it doesn't appears on my system. Can you check if this bug remains in attached version of patch. If so, please

[HACKERS] Avoid FK validations for no-rewrite ALTER TABLE ALTER TYPE

2012-01-04 Thread Noah Misch
Git master can already avoid rewriting the table for column type changes like varchar(10) - varchar(20). However, if the column in question is on either side of a FK relationship, we always revalidate the foreign key. Concretely, I wanted these no-rewrite type changes to also assume FK validity:

Re: [HACKERS] Avoid FK validations for no-rewrite ALTER TABLE ALTER TYPE

2012-01-04 Thread Noah Misch
I neglected to commit after revising the text of a few comments; use this version instead. Thanks. diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 3b52415..9eba8e8 100644 *** a/src/backend/commands/tablecmds.c --- b/src/backend/commands/tablecmds.c

Re: [HACKERS] Collect frequency statistics for arrays

2012-01-06 Thread Noah Misch
Corrections: On Thu, Dec 29, 2011 at 11:35:00AM -0500, Noah Misch wrote: On Wed, Nov 09, 2011 at 08:49:35PM +0400, Alexander Korotkov wrote: + *We set s to be the estimated frequency of the K'th element in a natural + *language's frequency table, where K is the target

[HACKERS] Second thoughts on CheckIndexCompatible() vs. operator families

2012-01-06 Thread Noah Misch
In 367bc426a1c22b9f6badb06cd41fc438fd034639, I introduced a CheckIndexCompatible() that approves btree and hash indexes having changed to a different operator class within the same operator family. To make that valid, I also tightened the operator family contracts for those access methods to

Re: [HACKERS] Disabled features on Hot Standby

2012-01-13 Thread Noah Misch
On Fri, Jan 13, 2012 at 12:08:05PM -0500, Robert Haas wrote: On Fri, Jan 13, 2012 at 11:13 AM, Simon Riggs si...@2ndquadrant.com wrote: Many WAL records have latestRemovedXid on them. We can use the same idea with XLOG_HEAP2_VISIBLE records, so we add a field to send the latest

Re: [HACKERS] Disabled features on Hot Standby

2012-01-13 Thread Noah Misch
On Sat, Jan 14, 2012 at 12:42:02AM -0500, Robert Haas wrote: On Fri, Jan 13, 2012 at 8:02 PM, Noah Misch n...@leadboat.com wrote: Simon spoke to the FPI side of the question. ?For heap pages, the XLOG_HEAP_NEWPAGE consumers are CLUSTER, VACUUM FULL and ALTER TABLE SET TABLESPACE

Re: [HACKERS] Disabled features on Hot Standby

2012-01-14 Thread Noah Misch
On Sat, Jan 14, 2012 at 08:08:29AM +, Simon Riggs wrote: On Sat, Jan 14, 2012 at 1:02 AM, Noah Misch n...@leadboat.com wrote: However, CLUSTER/VACUUM FULL already remove tuples still-visible to standby snapshots without provoking a recovery conflict. ?(Again only

Re: [HACKERS] Measuring relation free space

2012-01-14 Thread Noah Misch
On Sat, Jan 14, 2012 at 04:41:57AM -0500, Jaime Casanova wrote: pgstattuple and relation_free_space are very close in all the numbers except for 2 indexes pgbench_branches_pkey and pgbench_tellers_pkey; after a VACUUM FULL and a REINDEX (and the difference persistence) i checked

[HACKERS] psql filename completion: quoting

2012-01-14 Thread Noah Misch
Occasionally, I have a SQL file destined for psql's \i command whose name contains a space. Less often, I'll have a .csv destined for \copy with the same problem. psql's filename completion does not handle these well. It completes on the literal name, but the commands will only recognize quoted

[HACKERS] psql COPY vs. ON_ERROR_ROLLBACK, multi-command strings

2012-01-14 Thread Noah Misch
It has bothered me that psql's \copy ignores the ON_ERROR_ROLLBACK setting. Only SendQuery() takes note of ON_ERROR_ROLLBACK, and \copy, like all backslash commands, does not route through SendQuery(). Looking into this turned up several other weaknesses in psql's handling of COPY. For example,

Re: [HACKERS] Measuring relation free space

2012-01-16 Thread Noah Misch
On Sat, Jan 14, 2012 at 02:40:46PM -0500, Jaime Casanova wrote: On Sat, Jan 14, 2012 at 6:26 AM, Noah Misch n...@leadboat.com wrote: - pgstattuple() and relation_free_space() should emit the same number, even if ?that means improving pgstattuple() at the same time. yes, i just wanted

Re: [HACKERS] foreign key locks, 2nd attempt

2012-01-17 Thread Noah Misch
On Sun, Jan 15, 2012 at 01:49:54AM -0300, Alvaro Herrera wrote: - I'm not sure that the multixact truncation code is sane on checkpoints. It might be that I need to tweak more the pg_control info we keep about truncation. The whole truncation thing needs more testing, too. My largest

Re: [HACKERS] foreign key locks, 2nd attempt

2012-01-17 Thread Noah Misch
On Mon, Jan 16, 2012 at 04:52:36PM -0300, Alvaro Herrera wrote: Excerpts from Heikki Linnakangas's message of lun ene 16 16:17:42 -0300 2012: On 15.01.2012 06:49, Alvaro Herrera wrote: --- 164,178 #define HEAP_HASVARWIDTH0x0002/* has variable-width attribute(s) */

Re: [HACKERS] Collect frequency statistics for arrays

2012-01-17 Thread Noah Misch
On Tue, Jan 17, 2012 at 12:04:06PM +0400, Alexander Korotkov wrote: Thanks for your fixes to the patch. Them looks correct to me. I did some fixes in the patch. The proof of some concepts is still needed. I'm going to provide it in a few days. Your further fixes look good. Could you also

Re: [HACKERS] psql \timing vs failed statements

2012-01-17 Thread Noah Misch
On Tue, Jan 17, 2012 at 04:01:23PM +0100, Magnus Hagander wrote: Thus - if I were to change psql to output timing on failed queries as well, will anybody object? ;) +1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Measuring relation free space

2012-01-18 Thread Noah Misch
On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: On Mon, Jan 16, 2012 at 5:09 AM, Noah Misch n...@leadboat.com wrote: pgstattuple()'s decision to treat half-dead pages like deleted pages is better. ?That transient state can only end in the page's deletion. the only page

Re: [HACKERS] foreign key locks, 2nd attempt

2012-01-18 Thread Noah Misch
On Wed, Jan 18, 2012 at 05:18:31PM -0300, Alvaro Herrera wrote: Excerpts from Heikki Linnakangas's message of mar ene 17 03:21:28 -0300 2012: On 16.01.2012 21:52, Alvaro Herrera wrote: I was initially thinking that pg_multixact should return the empty set if requested members of a multi

[HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-19 Thread Noah Misch
Hi Mikko, First, for everyone else: this patch provides a more-compact binary output format for arrays. When the array contains no NULL elements and has a fixed-length element type, the new format saves four bytes per array element. We could do more. We could add support for arrays containing

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-19 Thread Noah Misch
On Thu, Jan 19, 2012 at 02:00:20PM -0500, Robert Haas wrote: On Thu, Jan 19, 2012 at 10:37 AM, Noah Misch n...@leadboat.com wrote: I agree with Merlin; the frontend/backend protocol is logically distinct from the binary send/recv formats of data types. ?For one key point, the latter

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Noah Misch
On Fri, Jan 20, 2012 at 07:03:22PM -0500, Jaime Casanova wrote: On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: ignoring all non-leaf pages still gives a considerable difference between pgstattuple

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-01-21 Thread Noah Misch
On Sat, Jan 14, 2012 at 08:18:48PM +0100, Marco Nenciarini wrote: This is our latest version of the patch. Gabriele, Gianni and I have discussed a lot and decided to send an initial patch which uses EACH REFERENCES instead of ARRAY REFERENCES. The reason behind this is that ARRAY REFERENCES

Re: [HACKERS] Avoid FK validations for no-rewrite ALTER TABLE ALTER TYPE

2012-01-21 Thread Noah Misch
On Sat, Jan 21, 2012 at 08:04:20PM -0800, Jeff Janes wrote: This is failing make check for me. *** /tmp/foo/src/test/regress/expected/alter_table.out Sat Jan 21 19:51:46 2012 --- /tmp/foo/src/test/regress/results/alter_table.out Sat Jan 21 19:54:18 2012 *** ***

Re: [HACKERS] Second thoughts on CheckIndexCompatible() vs. operator families

2012-01-21 Thread Noah Misch
New version that repairs a defective test case. diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index 712b0b0..1bf1de5 100644 *** a/src/backend/commands/indexcmds.c --- b/src/backend/commands/indexcmds.c *** *** 23,28 --- 23,29 #include

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-01-22 Thread Noah Misch
On Sun, Jan 22, 2012 at 09:06:49PM +, Simon Riggs wrote: On Sat, Jan 21, 2012 at 8:42 PM, Noah Misch n...@leadboat.com wrote: You currently forbid multi-column EACH FKs. ?I agree that we should allow only one array column per FK; with more, the set of required PK rows would

Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-22 Thread Noah Misch
On Sun, Jan 22, 2012 at 11:47:06PM +0200, Mikko Tiihonen wrote: On 01/20/2012 04:45 AM, Noah Misch wrote: On Thu, Jan 19, 2012 at 02:00:20PM -0500, Robert Haas wrote: Having said that, if we're to follow the precedent set by bytea_format, maybe we ought to just add binary_array_format={huge

Re: [HACKERS] Collect frequency statistics for arrays

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 01:21:20AM +0400, Alexander Korotkov wrote: Updated patch is attached. I've updated comment of mcelem_array_contained_selec with more detailed description of probability distribution assumption. Also, I found that rest behavious should be better described by Poisson

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 10:34:12AM -0500, Robert Haas wrote: On Mon, Jan 23, 2012 at 9:36 AM, Florian Weimer fwei...@bfk.de wrote: * Robert Haas: In this particular case, I knew that the change was coming and could push updated Java and Perl client libraries well before the server-side

Re: [HACKERS] Measuring relation free space

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 04:56:24PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of vie ene 20 22:33:30 -0300 2012: pgstattuple() figures the free_percent by adding up all space available to hold tuples and dividing that by the simple size of the relation. Non-leaf pages

Re: [HACKERS] Measuring relation free space

2012-01-25 Thread Noah Misch
On Tue, Jan 24, 2012 at 11:24:08AM -0500, Jaime Casanova wrote: On Mon, Jan 23, 2012 at 7:18 PM, Noah Misch n...@leadboat.com wrote: If someone feels like doing it, +1 for making pgstattuple() count non-leaf free space. actually i agreed that non-leaf pages are irrelevant... i just

Re: [HACKERS] Second thoughts on CheckIndexCompatible() vs. operator families

2012-01-25 Thread Noah Misch
On Wed, Jan 25, 2012 at 03:32:49PM -0500, Robert Haas wrote: On Sun, Jan 22, 2012 at 12:23 AM, Noah Misch n...@leadboat.com wrote: New version that repairs a defective test case. Committed. I don't find this to be particularly good style: Thanks. + for (i = 0; i old_natts ret; i

Re: [HACKERS] Avoid FK validations for no-rewrite ALTER TABLE ALTER TYPE

2012-01-25 Thread Noah Misch
On Wed, Jan 25, 2012 at 11:17:27AM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of dom ene 22 02:05:31 -0300 2012: Thanks. I've attached a new version fixing this problem. Looks good to me. Can you please clarify this bit? * Since we

Re: [HACKERS] Second thoughts on CheckIndexCompatible() vs. operator families

2012-01-26 Thread Noah Misch
On Wed, Jan 25, 2012 at 11:53:10PM -0500, Tom Lane wrote: Not only is that code spectacularly unreadable, but has nobody noticed that this commit broke the buildfarm? Thanks for reporting the problem. This arose because the new test case temporarily sets client_min_messages=DEBUG1. The

Re: [HACKERS] Avoid FK validations for no-rewrite ALTER TABLE ALTER TYPE

2012-01-26 Thread Noah Misch
On Wed, Jan 25, 2012 at 10:39:56PM -0500, Noah Misch wrote: In any event, the patch needed a rebase, so I've attached it rebased and with that comment edited to reference ri_GenerateQualCollation(), that being the most-relevant source for the assumption in question. Commit

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-26 Thread Noah Misch
On Wed, Jan 11, 2012 at 10:12:31PM +, Simon Riggs wrote: v7 This patch uses FPIs to guard against torn hint writes, even when the checksums are disabled. One could not simply condition them on the page_checksums setting, though. Suppose page_checksums=off and we're hinting a page

Re: [HACKERS] initdb and fsync

2012-01-27 Thread Noah Misch
On Fri, Jan 27, 2012 at 04:19:41PM -0800, Jeff Davis wrote: It looks like initdb doesn't fsync all the files it creates, e.g. the PG_VERSION file. While it's unlikely that it would cause any real data loss, it can be inconvenient in some testing scenarios involving VMs. Thoughts? Would a

Re: [HACKERS] Second thoughts on CheckIndexCompatible() vs. operator families

2012-01-28 Thread Noah Misch
On Thu, Jan 26, 2012 at 08:23:34AM -0500, Robert Haas wrote: On Thu, Jan 26, 2012 at 6:55 AM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 25, 2012 at 11:53:10PM -0500, Tom Lane wrote: Not only is that code spectacularly unreadable, but has nobody noticed that this commit broke

Re: [HACKERS] foreign key locks, 2nd attempt

2012-01-30 Thread Noah Misch
On Tue, Jan 24, 2012 at 03:47:16PM -0300, Alvaro Herrera wrote: The biggest item remaining is the point you raised about multixactid wraparound. This is closely related to multixact truncation and the way checkpoints are to be handled. If we think that MultiXactId wraparound is possible, and

Re: [HACKERS] show Heap Fetches in EXPLAIN for index-only scans

2012-02-02 Thread Noah Misch
On Wed, Jan 25, 2012 at 08:42:05PM -0500, Robert Haas wrote: Only the first pass of vacuum knows how to mark pages all-visible. After the update, the first pass of the first vacuum sees a dead tuple on the old page and truncates it to a dead line pointer. When it comes to the new page, it

[HACKERS] Re: Review of: explain / allow collecting row counts without timing info

2012-02-03 Thread Noah Misch
On Fri, Feb 03, 2012 at 11:12:33AM -0500, Robert Haas wrote: After looking at this a bit, I think we can make the interface a bit more convenient. I'd like to propose that we call the EXPLAIN option ROWS rather than TIMING, and that we provide the row-count information if *either* ROWS or

Re: [HACKERS] initdb and fsync

2012-02-04 Thread Noah Misch
On Sat, Feb 04, 2012 at 03:41:27PM -0800, Jeff Davis wrote: On Sat, 2012-01-28 at 13:18 -0500, Tom Lane wrote: Yeah. Personally I would be sad if initdb got noticeably slower, and I've never seen or heard of a failure that this would fix. I worked up a patch, and it looks like it does

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Noah Misch
On Sun, Feb 05, 2012 at 10:53:20AM -0800, Jeff Davis wrote: On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: If we add fsync calls to the initdb process, they should cover the entire data directory tree. This patch syncs files that initdb.c writes, but we ought to also sync files

Re: [HACKERS] Add protransform for numeric, varbit, and temporal types

2012-02-07 Thread Noah Misch
On Tue, Feb 07, 2012 at 12:43:11PM -0500, Robert Haas wrote: I've committed the numeric and varbit patches and will look at the temporal one next. Thanks. The comment you added to numeric_transform() has a few typos, constained - constrained and nodes - notes. I did notice one odd thing,

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-07 Thread Noah Misch
On Tue, Feb 07, 2012 at 08:58:59PM +, Simon Riggs wrote: On Thu, Jan 26, 2012 at 8:20 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 11, 2012 at 10:12:31PM +, Simon Riggs wrote: This patch uses FPIs to guard against torn hint writes, even when the checksums are disabled. ?One

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-08 Thread Noah Misch
On Wed, Feb 08, 2012 at 11:40:34AM +, Simon Riggs wrote: On Wed, Feb 8, 2012 at 3:24 AM, Noah Misch n...@leadboat.com wrote: On Tue, Feb 07, 2012 at 08:58:59PM +, Simon Riggs wrote: On Thu, Jan 26, 2012 at 8:20 PM, Noah Misch n...@leadboat.com wrote: This patch uses FPIs to guard

Re: [HACKERS] Add protransform for numeric, varbit, and temporal types

2012-02-09 Thread Noah Misch
On Wed, Feb 08, 2012 at 09:37:01AM -0500, Robert Haas wrote: On Tue, Feb 7, 2012 at 12:43 PM, Robert Haas robertmh...@gmail.com wrote: I've committed the numeric and varbit patches and will look at the temporal one next. Committed, after changing the OIDs so they don't conflict. Here's

Re: [HACKERS] Progress on fast path sorting, btree index creation time

2012-02-09 Thread Noah Misch
On Tue, Feb 07, 2012 at 09:38:39PM -0500, Robert Haas wrote: Second, there's a concern about binary bloat: duplicating lots of code with different comparators inlined generates, well, a lot of machine code. Of course, an 0.8% increase in the size of the resulting binary is very unlikely to

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-02-10 Thread Noah Misch
On Thu, Feb 09, 2012 at 11:11:16PM +0200, Marti Raudsepp wrote: I've always been a little wary of using the TRUNCATE command due to the warning in the documentation about it not being MVCC-safe: queries may silently give wrong results and it's hard to tell when they are affected. That got

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-02-10 Thread Noah Misch
On Fri, Feb 10, 2012 at 01:59:18PM -0500, Robert Haas wrote: On Fri, Feb 10, 2012 at 6:42 AM, Noah Misch n...@leadboat.com wrote: I like the design you have chosen. ?It would find applications beyond TRUNCATE, so your use of non-specific naming is sound. ?For example, older snapshots

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-21 Thread Noah Misch
On Mon, Feb 20, 2012 at 08:57:08AM -0500, Robert Haas wrote: On Mon, Feb 20, 2012 at 4:18 AM, Simon Riggs si...@2ndquadrant.com wrote: What straightforward implementation is that?? This *is* the straightforward one. God knows what else we'd break if we drop the lock, reacquire as an

Re: [HACKERS] Measuring relation free space

2012-02-21 Thread Noah Misch
On Tue, Feb 14, 2012 at 02:04:26AM -0500, Jaime Casanova wrote: On Wed, Jan 25, 2012 at 9:47 PM, Noah Misch n...@leadboat.com wrote: With all that done, run some quick benchmarks: see how SELECT free_percent FROM pgstattuple(rel) fares compared to SELECT relation_free_space(rel

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-21 Thread Noah Misch
On Sun, Feb 19, 2012 at 05:04:06PM -0500, Robert Haas wrote: On Sun, Feb 19, 2012 at 11:35 AM, Simon Riggs si...@2ndquadrant.com wrote: So, when the page has a checksum, PD_CHECKSUM2 is not set, and when it doesn't have a checksum, PD_CHECKSUM2 is not set? ?What good does that do? As

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-22 Thread Noah Misch
the books on this patch for the purpose of this CommitFest; I'm marking it Returned with Feedback. Thanks for your efforts thus far. On Mon, Jan 30, 2012 at 06:48:47PM -0500, Noah Misch wrote: On Tue, Jan 24, 2012 at 03:47:16PM -0300, Alvaro Herrera wrote: * Columns that are part of the key Noah

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-23 Thread Noah Misch
On Thu, Feb 23, 2012 at 02:08:28PM +0100, Jeroen Vermeulen wrote: On 2012-02-23 10:18, Simon Riggs wrote: However, review of such a large patch should not be simply pass or fail. We should be looking back at the original problem and ask ourselves whether some subset of the patch could solve a

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-23 Thread Noah Misch
On Thu, Feb 23, 2012 at 06:36:42PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of mi?? feb 22 14:00:07 -0300 2012: On Mon, Feb 13, 2012 at 07:16:58PM -0300, Alvaro Herrera wrote: On Mon, Jan 30, 2012 at 06:48:47PM -0500, Noah Misch wrote: On Tue, Jan 24, 2012

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-23 Thread Noah Misch
On Thu, Feb 23, 2012 at 12:43:11PM -0300, Alvaro Herrera wrote: Excerpts from Simon Riggs's message of jue feb 23 06:18:57 -0300 2012: On Wed, Feb 22, 2012 at 5:00 PM, Noah Misch n...@leadboat.com wrote: All in all, I think this is in pretty much final shape. ??Only pg_upgrade bits

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-02-24 Thread Noah Misch
Hi Marco, This version fixes everything I noted in my last review. Apart from corner cases I note, the patch is technically sound. I pose a few policy-type questions; comments from a broader audience would help those. On Mon, Feb 06, 2012 at 07:04:42PM +0100, Marco Nenciarini wrote: Please

Re: [HACKERS] leakproof

2012-02-24 Thread Noah Misch
On Wed, Feb 22, 2012 at 06:30:37PM -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 02/22/2012 04:29 PM, Marc Munro wrote: As the developer of veil I feel marginally qualified to bikeshed here: how about silent? A silent function being one that will not blab. I also

Re: [HACKERS] psql COPY vs. ON_ERROR_ROLLBACK, multi-command strings

2012-02-24 Thread Noah Misch
On Wed, Jan 25, 2012 at 06:25:46PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of s??b ene 14 12:40:02 -0300 2012: It has bothered me that psql's \copy ignores the ON_ERROR_ROLLBACK setting. Only SendQuery() takes note of ON_ERROR_ROLLBACK, and \copy, like all backslash

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-27 Thread Noah Misch
On Mon, Feb 27, 2012 at 02:13:32PM +0200, Heikki Linnakangas wrote: On 23.02.2012 18:01, Alvaro Herrera wrote: As far as complexity, yeah, it's a lot more complex now -- no question about that. How about assigning a new, real, transaction id, to represent the group of transaction ids. The

Re: [HACKERS] ECPG FETCH readahead

2012-03-02 Thread Noah Misch
On Thu, Dec 29, 2011 at 10:46:23AM +0100, Boszormenyi Zoltan wrote: 2011-11-16 20:51 keltez?ssel, Boszormenyi Zoltan ?rta: 2010-10-14 11:56 keltez?ssel, Boszormenyi Zoltan ?rta: On Thu, Jun 24, 2010 at 8:19 AM, Michael Meskes mes...@postgresql.org wrote: On Thu, Jun 24, 2010 at

Re: [HACKERS] index-only quals vs. security_barrier views

2012-03-02 Thread Noah Misch
On Thu, Feb 09, 2012 at 12:02:29PM -0500, Robert Haas wrote: When Heikki worked up his original index-only scan patches (which didn't end up looking much like what eventually got committed), he had the notion of an index-only qual. That is, given a query like this: select sum(1) from foo

Re: [HACKERS] COPY with hints, rebirth

2012-03-02 Thread Noah Misch
On Fri, Mar 02, 2012 at 08:46:45AM +, Simon Riggs wrote: On Thu, Mar 1, 2012 at 8:49 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It's still broken: [BEGIN;TRUNCATE;SAVEPOINT;COPY;ROLLBACK TO] So this approach isn't the one... The COPY FREEZE patch provides a

Re: [HACKERS] ECPG FETCH readahead

2012-03-05 Thread Noah Misch
On Sun, Mar 04, 2012 at 05:16:06PM +0100, Michael Meskes wrote: On Fri, Mar 02, 2012 at 11:41:05AM -0500, Noah Misch wrote: I suggest enabling the feature by default but drastically reducing the default readahead chunk size from 256 to, say, 5. That still reduces the FETCH round trip

Re: [HACKERS] ECPG FETCH readahead

2012-03-05 Thread Noah Misch
On Sun, Mar 04, 2012 at 04:33:32PM +0100, Boszormenyi Zoltan wrote: 2012-03-02 17:41 keltez?ssel, Noah Misch ?rta: On Thu, Dec 29, 2011 at 10:46:23AM +0100, Boszormenyi Zoltan wrote: I suggest enabling the feature by default but drastically reducing the default readahead chunk size from

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-03-05 Thread Noah Misch
On Mon, Feb 13, 2012 at 09:29:56AM -0500, Robert Haas wrote: On Fri, Feb 10, 2012 at 11:46 PM, Noah Misch n...@leadboat.com wrote: I've yet to see an MVCC anomaly that one can reproduce at REPEATABLE READ and not at READ COMMITTED. ?They tend to be narrow race conditions at READ

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-03-06 Thread Noah Misch
On Sun, Mar 04, 2012 at 01:02:57PM +, Simon Riggs wrote: More detailed thoughts show that the test in heap_beginscan_internal() is not right enough, i.e. wrong. We need a specific XidInMVCCSnapshot test on the relvalidxid, so it needs to be a specific xid, not an xmin because otherwise

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-03-06 Thread Noah Misch
On Mon, Mar 05, 2012 at 03:46:16PM -0500, Robert Haas wrote: On Mon, Mar 5, 2012 at 2:22 PM, Noah Misch n...@leadboat.com wrote: I can see this strategy applying to many relation-pertinent system catalogs. Do you foresee applications to non-relation catalogs? Well, in theory, we have

Re: [HACKERS] ECPG FETCH readahead

2012-03-06 Thread Noah Misch
On Tue, Mar 06, 2012 at 07:07:41AM +0100, Boszormenyi Zoltan wrote: 2012-03-05 19:56 keltez?ssel, Noah Misch ?rta: Or how about a new feature in the backend, so ECPG can do UPDATE/DELETE ... WHERE OFFSET N OF cursor and the offset of computed from the actual cursor position

Re: [HACKERS] RFC: Making TRUNCATE more MVCC-safe

2012-03-07 Thread Noah Misch
On Tue, Mar 06, 2012 at 08:36:05AM -0500, Robert Haas wrote: On Tue, Mar 6, 2012 at 5:43 AM, Noah Misch n...@leadboat.com wrote: Well, consider something like CLUSTER. ?It's perfectly OK for CLUSTER to operate on a table that has been truncated since CLUSTER's snapshot was taken

Re: [HACKERS] psql COPY vs. ON_ERROR_ROLLBACK, multi-command strings

2012-03-08 Thread Noah Misch
On Wed, Mar 07, 2012 at 04:57:12PM -0500, Robert Haas wrote: On Sat, Feb 25, 2012 at 12:57 AM, Noah Misch n...@leadboat.com wrote: Thanks. ?While testing a crashing function, I noticed that my above patch added some noise to psql output when the server crashes: [local] test=# select

Re: [HACKERS] sortsupport for text

2012-03-08 Thread Noah Misch
On Fri, Mar 02, 2012 at 03:45:38PM -0500, Robert Haas wrote: SELECT SUM(1) FROM (SELECT * FROM randomtext ORDER BY t) x; On unpatched master, this takes about 416 ms (plus or minus a few). With the attached patch, it takes about 389 ms (plus or minus a very few), a speedup of about 7%. I

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-08 Thread Noah Misch
On Wed, Mar 07, 2012 at 07:51:42PM -0500, Tom Lane wrote: Alexander Korotkov aekorot...@gmail.com writes: True. If (max count - min count + 1) is small, enumerating of frequencies is both more compact and more precise representation. Simultaneously, if (max count - min count + 1) is large,

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-08 Thread Noah Misch
On Thu, Mar 08, 2012 at 11:30:52AM -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Wed, Mar 07, 2012 at 07:51:42PM -0500, Tom Lane wrote: On reflection my idea above is wrong; for example assume that we have a column with 900 arrays of length 1 and 100 arrays of length 2

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Noah Misch
On Mon, Mar 12, 2012 at 01:28:11PM -0400, Robert Haas wrote: I spent some time thinking about this over the weekend, and I have an observation, and an idea. Here's the observation: I believe that locking a tuple whose xmin is uncommitted is always a noop, because if it's ever possible for a

Re: [HACKERS] Measuring relation free space

2012-03-12 Thread Noah Misch
On Fri, Mar 09, 2012 at 02:18:02AM -0500, Jaime Casanova wrote: On Wed, Feb 22, 2012 at 12:27 AM, Noah Misch n...@leadboat.com wrote: On Tue, Feb 14, 2012 at 02:04:26AM -0500, Jaime Casanova wrote: 1) pgstattuple-gin_spgist.patch This first patch adds gin and spgist support to pgstattuple

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Noah Misch
On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you that some worst case performance tests should be done. Could you please say what you think the worst cases would be, so those can be

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Noah Misch
On Wed, Mar 14, 2012 at 01:23:14PM -0400, Robert Haas wrote: On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch n...@leadboat.com wrote: More often than that; each 2-member mxid takes 4 bytes in an offsets file and 10 bytes in a members file. ?So, more like one fsync per ~580 mxids. ?Note

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Noah Misch
On Thu, Mar 15, 2012 at 08:37:36PM -0400, Robert Haas wrote: On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas robertmh...@gmail.com wrote: You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on mxid

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Noah Misch
On Thu, Mar 15, 2012 at 04:14:03PM -0700, Daniel Farina wrote: Parallel to pg_cancel_backend, it'd be nice to allow the user to just outright kill a backend that they own (politely, with a SIGTERM), aborting any transactions in progress, including the idle transaction, and closing the socket.

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

2012-03-16 Thread Noah Misch
[used followup EACH-foreign-key.v4b.patch.bz2 for review] On Wed, Mar 14, 2012 at 07:03:08PM +0100, Marco Nenciarini wrote: please find attached v4 of the EACH Foreign Key patch (formerly known also as Foreign Key Array). On Fri, Feb 24, 2012 at 09:01:35PM -0500, Noah Misch wrote: We can

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-17 Thread Noah Misch
On Fri, Mar 16, 2012 at 04:42:07PM -0700, Daniel Farina wrote: On Fri, Mar 16, 2012 at 3:42 PM, Noah Misch n...@leadboat.com wrote: On Thu, Mar 15, 2012 at 04:14:03PM -0700, Daniel Farina wrote: I imagine the problem is a race condition whereby a pid might be reused by another process owned

  1   2   3   4   5   6   7   8   9   10   >