Re: [HACKERS] json accessors

2012-12-05 Thread Merlin Moncure
On Wed, Dec 5, 2012 at 12:49 PM, Josh Berkus wrote: > >>> *) xmlpath/jsonpath do searching (and decomposition) but are very >>> clunky from sql perspective and probably absolutely nogo in terms if >>> GIST/GIN. postgres spiritually wants to do things via operators and >>> we should (if possible)

Re: [HACKERS] json accessors

2012-12-05 Thread Merlin Moncure
On Wed, Dec 5, 2012 at 11:14 AM, David E. Wheeler wrote: > On Nov 28, 2012, at 4:10 PM, Merlin Moncure wrote: > >>> Yes, it's iterative. And for deeply nested json it might be somewhat >>> inefficient, although the parser is pretty fast AFAICT. But it's a start

Re: [HACKERS] json accessors

2012-11-30 Thread Merlin Moncure
On Fri, Nov 30, 2012 at 9:02 AM, Andrew Dunstan wrote: > > On 11/30/2012 09:51 AM, Merlin Moncure wrote: >> >> >> Two questions: >> 1) is it possible for these to work without a polymorphic object >> passed through as hstore does (null::foo)? >> select

Re: [HACKERS] json accessors

2012-11-30 Thread Merlin Moncure
On Fri, Nov 30, 2012 at 8:38 AM, Andrew Dunstan wrote: > OK, so based on this discussion, I'm thinking of the following: ok, this is looking awesome -- couple naming suggestions (see inline): > * keep the original functions and operators. json_keys is still >required for the case where the

Re: [HACKERS] json accessors

2012-11-29 Thread Merlin Moncure
On Thu, Nov 29, 2012 at 4:14 PM, Andrew Dunstan wrote: > There are many things wrong with this. First, converting to hstore so you > can call populate_record is quite horrible and ugly and inefficient. And > it's dependent on having hstore loaded - you can't have an hstore_to_jon in > core because

Re: [HACKERS] json accessors

2012-11-29 Thread Merlin Moncure
On Thu, Nov 29, 2012 at 1:19 PM, Andrew Dunstan wrote: > > On 11/29/2012 01:06 PM, Merlin Moncure wrote: >> >> so, just hashing out your proposal and making sure it's reasonable >> analogous implementation of xpath. Sleeping on it, I say mostly, but >> no

Re: [HACKERS] json accessors

2012-11-29 Thread Merlin Moncure
On Thu, Nov 29, 2012 at 7:58 AM, Andrew Dunstan wrote: > On 11/28/2012 08:16 PM, Hannu Krosing wrote: >> You could even do a template-less row_from_json which returns a records >> with all fields converted to >> the JSON-encodable types and hope that the next conversions will be done >> by postgre

Re: [HACKERS] json accessors

2012-11-28 Thread Merlin Moncure
On Wed, Nov 28, 2012 at 2:44 PM, Andrew Dunstan wrote: > On 11/28/2012 02:08 PM, Merlin Moncure wrote: >> *) ISTM your keytext operators are a reasonable replacement for a >> hypothetical json_path. That said you're basically forcing json->sql >> mapping through a

Re: [HACKERS] json accessors

2012-11-28 Thread Merlin Moncure
On Wed, Nov 28, 2012 at 11:04 AM, Andrew Dunstan wrote: > > This is a proposal to create some basic functions to extract values from > json. The simple functions I envision would be: > > * json_object_keys(json) => setof text >returns the set of dequoted, unescaped keys of the object, >er

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Merlin Moncure
On Tue, Nov 27, 2012 at 4:41 PM, Tom Lane wrote: > Merlin Moncure writes: >> ... I think if you relaxed >> the function sigs of a few functions on this page >> (http://www.postgresql.org/docs/9.2/interactive/functions-string.html), >> most reported problems w

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Merlin Moncure
On Tue, Nov 27, 2012 at 4:09 PM, Peter Eisentraut wrote: > On 11/27/12 12:07 PM, Merlin Moncure wrote: >> Speaking of polymorphism, why not just implement lpad()'s first >> argument as 'anyelement'? > > One of the arguments made here was that lpad(not-text) *

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-11-27 Thread Merlin Moncure
On Tue, Nov 27, 2012 at 10:52 AM, Pavel Stehule wrote: > it is a basic problem - PostgreSQL has unique possibilities - > polymorphic parameters and almost all databases doesn't support > overloading Speaking of polymorphism, why not just implement lpad()'s first argument as 'anyelement'? ISTM th

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-26 Thread Merlin Moncure
On Mon, Nov 26, 2012 at 3:03 PM, Robert Haas wrote: > On Mon, Nov 26, 2012 at 3:29 PM, Jeff Davis wrote: >> Your intuition here is better than mine, but I am still missing >> something here. If we keep the buffer pinned, then there will be very >> few pin/unpin cycles here, so I don't see where

Re: [HACKERS] WIP json generation enhancements

2012-11-26 Thread Merlin Moncure
On Thu, Nov 22, 2012 at 4:54 AM, Dimitri Fontaine wrote: > Andrew Dunstan writes: >> Here is a WIP patch for enhancements to json generation. >> >> First, there is the much_requested json_agg, which will aggregate rows >> directly to json. So the following will now work: >> >> select json_agg

Re: [HACKERS] StrategyGetBuffer questions

2012-11-21 Thread Merlin Moncure
On Tue, Nov 20, 2012 at 4:50 PM, Jeff Janes wrote: > On Tue, Nov 20, 2012 at 1:26 PM, Merlin Moncure wrote: >> In this sprawling thread on scaling issues [1], the topic meandered >> into StrategyGetBuffer() -- in particular the clock sweep loop. I'm >> wondering: >

Re: [HACKERS] StrategyGetBuffer questions

2012-11-20 Thread Merlin Moncure
On Tue, Nov 20, 2012 at 4:50 PM, Jeff Janes wrote: > On Tue, Nov 20, 2012 at 1:26 PM, Merlin Moncure wrote: >> In this sprawling thread on scaling issues [1], the topic meandered >> into StrategyGetBuffer() -- in particular the clock sweep loop. I'm >> wondering: >

Re: [HACKERS] Do we need so many hint bits?

2012-11-20 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 10:37 PM, Simon Riggs wrote: > On 15 November 2012 22:21, Tom Lane wrote: > >>> Removing those 3 hints would give us 3 more flag bits (eventually, after >>> we are sure they aren't just leftover), and it would also reduce the >>> chance that a page is dirtied for no other

Re: [HACKERS] Materialized views WIP patch

2012-11-16 Thread Merlin Moncure
On Fri, Nov 16, 2012 at 7:13 AM, Dimitri Fontaine wrote: > Jeff Davis writes: >> The documentation says that a materialized view is basically a >> create-table-as-select except that it remembers the query. Would you say >> that there is a compelling use case for this alone, or is this a >> buildi

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 10:14 PM, Pavan Deolasee wrote: > Another approach could be to set those additional bits, but don't dirty the > page. So if the page is already dirty or gets dirty later on before its > eviction, those hint bits will get recorded. We can also try some other > variants like:

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Merlin Moncure
On Fri, Nov 16, 2012 at 8:58 AM, Andres Freund wrote: > On 2012-11-16 08:43:12 -0600, Merlin Moncure wrote: >> On Thu, Nov 15, 2012 at 9:21 PM, Tom Lane wrote: >> > Jeff Davis writes: >> >> It occurred to me recently that many of the hint bits aren't terribly

Re: [HACKERS] Do we need so many hint bits?

2012-11-16 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 9:21 PM, Tom Lane wrote: > Jeff Davis writes: >> It occurred to me recently that many of the hint bits aren't terribly >> important (at least it's not obvious to me). HEAP_XMIN_COMMITTED clearly >> has a purpose, and we'd expect it to be used many times following the >> in

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-16 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 6:42 PM, Jeff Davis wrote: > On Tue, 2012-11-06 at 17:55 -0600, Merlin Moncure wrote: >> So given that -- the patch simple adds an extra check when/where hint >> bit status is checked in the visibility routines (currently, only >> HeapTupleSatisfiesMVC

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-16 Thread Merlin Moncure
On Fri, Nov 16, 2012 at 4:32 AM, Amit Kapila wrote: > On Thursday, November 15, 2012 10:19 PM Merlin Moncure wrote: >> On Thu, Nov 15, 2012 at 10:25 AM, Amit Kapila >> wrote: > >> >> Sure, although in scans we are using ring buffer as well so in >> practica

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 10:25 AM, Amit Kapila wrote: >> IMNSHO. deferring non-critical i/o from foreground process to >> background process is generally good. > > Yes, in regard of deferring you are right. > However in this case may be when foreground process has to mark page dirty > due to hint-b

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 4:39 AM, Amit Kapila wrote: >>In each visibility function (except HeapTupleSatisfiesVacuum() ), an >> addition check has been added to check if the commit status of Xmin or Xmax >> of a tuple can be >retrieved from the cache. > > > > 1. From your explanation and code,

Re: [HACKERS] WIP checksums patch

2012-11-14 Thread Merlin Moncure
On Tue, Oct 9, 2012 at 1:51 AM, Amit Kapila wrote: > On Monday, October 01, 2012 11:11 PM Jeff Davis wrote: >> On Mon, 2012-10-01 at 18:14 +0100, Simon Riggs wrote: >> > You are missing large parts of the previous thread, giving various >> > opinions on what the UI should look like for enabling ch

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Merlin Moncure
On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane wrote: >> Ok. It won't help all that much on 9.0, though. > > Well, it won't help GIST much, but the actually-reported-from-the-field > case is in btree, and it does fix that. > > It occurs to me that if we're sufficiently scared of this case, we could > p

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Mon, Nov 12, 2012 at 1:21 PM, Simon Riggs wrote: > On 10 September 2012 17:50, Tom Lane wrote: > >> The point of the proposal that I am making is to have a simple, >> low-maintenance solution for people who need a single-application >> database. A compromise somewhere in the middle isn't like

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila wrote: > On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: > On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: > On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: > On Friday, September 07, 2012 11:19 PM Tom Lane wrote: > Heikki Li

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Merlin Moncure
On Fri, Nov 9, 2012 at 8:22 AM, Kevin Grittner wrote: > Robert Haas wrote: > >> What I've been wondering since this last came up is whether we >> could use some variant of the SIREAD locks Kevin introduced for SSI >> to handle this case - essentially have the transaction doing the >> TRUNCATE make

Re: [HACKERS] Extend libpq to support mixed text and binary results

2012-11-07 Thread Merlin Moncure
On Wed, Nov 7, 2012 at 7:08 AM, Albe Laurenz wrote: > The Frontend/Backend protocol's "bind" message allows to > specify mixed text and binary result formats, but there > is not support in the C API for that. > > I'd like to add support for that to libpq for these reasons: > > - I think that every

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-07 Thread Merlin Moncure
On Wed, Nov 7, 2012 at 6:01 AM, Amit Kapila wrote: >> >> Following the sig is a first cut at a patch (written by Atri) that >> >> attempts to mitigate hint bit i/o penalty when many pages worth of >> >> tuples are sequentially written out with the same transaction id. >> >> There have been other a

[HACKERS] WIP patch for hint bit i/o mitigation

2012-11-06 Thread Merlin Moncure
Following the sig is a first cut at a patch (written by Atri) that attempts to mitigate hint bit i/o penalty when many pages worth of tuples are sequentially written out with the same transaction id. There have been other attempts to deal with this problem that fit niche cases (especially those tha

Re: [HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-11-05 Thread Merlin Moncure
On Mon, Nov 5, 2012 at 11:33 AM, Robert Haas wrote: > On Tue, Oct 30, 2012 at 8:37 AM, crocket wrote: >> MySQL permits a connection to access multiple databases. >> But Postgresql restricts a connection to one database. >> I think postgresql database connection is somewhat limited. >> >> Is it an

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-05 Thread Merlin Moncure
On Fri, Nov 2, 2012 at 8:00 PM, Noah Misch wrote: >> hm, several times over the last couple of months (both on postgres 9.1 >> and 9.2), i've seen a similar crash, but on linux. It hits the log >> like this: >> >> Execution halted (~ 200x) >> Error: segfault from C stack overflow >> Execution h

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-02 Thread Merlin Moncure
On Fri, Nov 2, 2012 at 1:25 PM, Matthew Gerber wrote: > Hello, > > I am encountering an error on my Postgres installation for Windows Server > 64-bit. The error was posted here a couple months ago; however, no solution > was found on the pgsql-bugs list, so I am reposting it to pgsql-hackers in >

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Merlin Moncure
On Mon, Oct 29, 2012 at 11:26 AM, Pavel Stehule wrote: > Hello > > > 2012/10/29 Hannu Krosing : >> Hi >> >> Is there a way to test for a variable being unassigned in pl/pgsql ? >> >> I'm writing an audit trigger where I'd like to save full before and >> after images into audit log and I really do

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-29 Thread Merlin Moncure
On Mon, Oct 29, 2012 at 8:31 AM, Amit Kapila wrote: > SYNTAX: > > ALTER SYSTEM SET configuration_parameter = value COMMENT 'value'; > > Comments/Suggestions about the value of this feature and Implementation > Idea? How is this better than simply updating them through the pg_settings view (in SQ

Re: [HACKERS] Deprecating RULES

2012-10-22 Thread Merlin Moncure
On Fri, Oct 19, 2012 at 2:55 PM, Robert Haas wrote: > On Fri, Oct 19, 2012 at 10:29 AM, Andrew Dunstan wrote: >> As you can see, in the case of rewrite it takes us back 7 1/2 years. I know >> this is a *very* rough measure, but it still tends to indicate to me that >> the maintenance burden isn't

Re: [HACKERS] [PATCH] Enforce that INSERT...RETURNING preserves the order of multi rows

2012-10-17 Thread Merlin Moncure
On Wed, Oct 17, 2012 at 9:29 AM, Peter Geoghegan wrote: > On 17 October 2012 14:53, Merlin Moncure wrote: >> Is that defined in the standard? > > RETURNING isn't even defined in the standard. Right: Point being, assumptions based on implementation ordering are generally

Re: [HACKERS] [PATCH] Enforce that INSERT...RETURNING preserves the order of multi rows

2012-10-17 Thread Merlin Moncure
On Wed, Oct 17, 2012 at 7:38 AM, P. Christeas wrote: > It has been a fact that the RETURNING clause on an INSERT will return > multiple rows with the same order as multiple VALUES have been fed. Is that defined in the standard? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] Improving the performance of psql tab completion

2012-10-10 Thread Merlin Moncure
On Wed, Oct 10, 2012 at 8:45 AM, Tom Lane wrote: > Merlin Moncure writes: >> ...but isn't pg_table_is_visible overkill for tab completion? > > How much does this help? > > update pg_proc set procost = 10 where proname = 'pg_table_is_visible'; hm, it fixes

[HACKERS] Improving the performance of psql tab completion

2012-10-10 Thread Merlin Moncure
Hackers, I have a database with 94059 entries in pg_class. Things are mostly working fine but psql tab completion is frustratingly slow (around 2.5 seconds on this box). I poked around in psql a bit and saw that the main culprit was the table visibility condition check. Here's a typical query (t

Re: [HACKERS] Regarding identifying a foreign scan

2012-10-08 Thread Merlin Moncure
On Sun, Oct 7, 2012 at 10:29 AM, Tom Lane wrote: > Atri Sharma writes: >> Does that mean that using (some) global storage is the cause of the problem? > > If you're using global storage for state that needs to be replicated > per-scan, yes, probably. But it's hard to be sure when you're being >

Re: [HACKERS] FDW for PostgreSQL

2012-10-03 Thread Merlin Moncure
On Fri, Sep 14, 2012 at 9:25 AM, Shigeru HANADA wrote: > Hi all, > > I'd like to propose FDW for PostgreSQL as a contrib module again. > Attached patch is updated version of the patch proposed in 9.2 > development cycle. very nice. > - binary transfer (only against servers with same PG major v

Re: [HACKERS] CTE optimization fence on the todo list?

2012-10-01 Thread Merlin Moncure
On Wed, Sep 19, 2012 at 3:05 PM, Daniel Browning wrote: > Another good reason to reject it might be because the only way to disable > the CTE fence is to disable it by default. If that were the case, then I > would imagine that it would break backwards compatibility, especially in the > case of wr

Re: [HACKERS] data to json enhancements

2012-09-27 Thread Merlin Moncure
On Thu, Sep 27, 2012 at 8:22 AM, Robert Haas wrote: > On Wed, Sep 26, 2012 at 1:46 PM, Tom Lane wrote: >> Also, on reflection I'm not sure about commandeering cast-to-json for >> this --- aren't we really casting to "json member" or something like >> that? The distinction between a container and

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-20 Thread Merlin Moncure
On Thu, Sep 20, 2012 at 9:24 AM, Kevin Grittner wrote: > Merlin Moncure wrote: > >> Hm, I bet it's possible (although probably not easy) to deduce >> volatility from the function body...maybe through the validator. >> If you could do that (perhaps warning in cases

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-20 Thread Merlin Moncure
On Wed, Sep 19, 2012 at 2:39 PM, Kevin Grittner wrote: > Tom Lane wrote: >> Robert Haas writes: >>> It still seems like awfully weird behavior. >> >> Why? The WHERE condition relates only to the output of the _stats >> subquery, so why shouldn't it be evaluated there, rather than >> after the j

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Merlin Moncure
On Wed, Sep 19, 2012 at 9:30 AM, Tom Lane wrote: > florian.schoppm...@emc.com (Florian Schoppmann) writes: >> In PostgreSQL 9.1 and 9.2 (possibly also in earlier versions), the query > >> --8<-- >> WITH source AS ( >> SELECT i FROM generate_series(1,10) AS i >> ) >> SELECT >> i >> FROM >>

Re: [HACKERS] Enum binary access

2012-09-10 Thread Merlin Moncure
On Mon, Sep 10, 2012 at 8:42 AM, Petr Chmelar wrote: > Hi there, > > we tried to create the libpqtypes enum binary send but it doesn't work: > > // register types > PGregisterType user_def[] = { {"seqtype", enum_put, enum_get} }; > PQregisterTypes(connector->getConn(), PQT_USERDEFINED, user_def, 1

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Merlin Moncure
On Thu, Sep 6, 2012 at 12:56 PM, Jeff Davis wrote: > On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: >> In general I think the selling point for such a feature would be "no >> administrative hassles", and I believe that has to go not only for the >> end-user experience but also for the applicat

Re: [HACKERS] sha1, sha2 functions into core?

2012-08-21 Thread Merlin Moncure
On Mon, Aug 20, 2012 at 5:54 PM, Greg Sabino Mullane wrote: >> 3) use a purposefully slow hashing function like bcrypt. >> >> but I disagree: I don't like any scheme that encourages use of low >> entropy passwords. > > Perhaps off-topic, but how to do you figure that? Yeah -- bcrypt's main claim

Re: [HACKERS] Planner avoidance of index only scans for partial indexes

2012-08-17 Thread Merlin Moncure
On Thu, Aug 16, 2012 at 5:20 PM, Josh Berkus wrote: > I'm mainly looking for numbers for the 9.2 release. Like, "up to 14X > speedup on data warehousing queries". OK: I work in the multi family industry and we have a large database which we are organizing into an on-demand OLAP style data delive

[HACKERS] Planner avoidance of index only scans for partial indexes

2012-08-15 Thread Merlin Moncure
If you create an index like this: create index on foo(a,b,v) where d = some_constant; there is no way to get an IOS on the index: you have to supply a the partial index exclusionary value to get the value of the index and that fools the IOS chooser because it doesn't see the value in the explicit

Re: [HACKERS] sha1, sha2 functions into core?

2012-08-15 Thread Merlin Moncure
On Wed, Aug 15, 2012 at 10:22 AM, Joe Conway wrote: > On 08/15/2012 06:48 AM, Tom Lane wrote: >>> On Wed, Aug 15, 2012 at 6:11 AM, Bruce Momjian wrote: Is there a TODO here? >> >> If anybody's concerned about the security of our password storage, >> they'd be much better off working on impro

Re: [HACKERS] WIP patch for LATERAL subqueries

2012-08-08 Thread Merlin Moncure
On Tue, Aug 7, 2012 at 6:08 PM, Tom Lane wrote: > I wrote: >> What I'd like to do next, barring objections, is to band-aid the places >> where the planner could crash on a LATERAL query (probably just make it >> throw FEATURE_NOT_SUPPORTED errors), write some documentation, and >> then commit what

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-31 Thread Merlin Moncure
On Mon, Jul 30, 2012 at 10:26 PM, Jan Wieck wrote: > On 7/30/2012 10:31 PM, Leon Smith wrote: >> >> This is not necessarily true, on multiple levels. I mean, some of >> the programs I write are highly concurrent, and this form of batching >> would have almost no risk of stalling the network b

Re: [HACKERS] strange evaluation Window function and SRF functions?

2012-07-30 Thread Merlin Moncure
On Mon, Jul 30, 2012 at 11:47 AM, Pavel Stehule wrote: > > > 2012/7/30 Thom Brown >> >> On 30 July 2012 17:19, Pavel Stehule wrote: >>> >>> Hello >>> >>> I seen nice trick based on window function >>> http://stackoverflow.com/questions/11700930/how-can-i-trim-a-text-array-in-postgresql >>> >>> b

Re: [HACKERS] Covering Indexes

2012-07-27 Thread Merlin Moncure
On Fri, Jul 27, 2012 at 12:24 PM, Jeff Davis wrote: > On Thu, 2012-07-26 at 12:13 -0400, Bruce Momjian wrote: >> So, do we want a TODO item about adding columns to a unique index that >> will not be used for uniqueness checks? > > -1 from me, at least in its current form. > > At it's heart, this i

Re: [HACKERS] Covering Indexes

2012-07-26 Thread Merlin Moncure
On Thu, Jul 26, 2012 at 12:17 PM, Robert Haas wrote: >> I think so. The case where you want a wide multiple column primary >> key to be extended to cover that one extra commonly grabbed value is >> not super common but entirely plausible. With the existing >> infrastructure to get the advantages

Re: [HACKERS] Covering Indexes

2012-07-26 Thread Merlin Moncure
On Thu, Jul 26, 2012 at 11:13 AM, Bruce Momjian wrote: > On Tue, Jul 17, 2012 at 06:00:37PM +0100, Simon Riggs wrote: >> > Either way the data in "c" and "d" are IN THE INDEX otherwise in neither >> > case could the data values be returned while strictly querying the index. >> > >> > So the questi

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tuesday, July 24, 2012, Marko Kreen wrote: > On Wed, Jul 25, 2012 at 1:29 AM, Merlin Moncure wrote: >> On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen wrote: >>> So if we give only PQgetResult() in 9.2, I do not see that we >>> are locked out from any interesting opt

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 5:29 PM, Merlin Moncure wrote: > so #2 seems like the lowest common > denominator (it would permanently preclude #3 and would require #4 to > introduce two new functions instead of just one). #1 of course would > bolt on to #2. oops, got #1 and #2 bac

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen wrote: > So if we give only PQgetResult() in 9.2, I do not see that we > are locked out from any interesting optimizations. Well, you are locked out of having PQgetResult reuse the conn's result since that would then introduce potentially breaking chan

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 3:52 PM, Marko Kreen wrote: > * Is there better API than PQsetSingleRowMode()? New PQsend* > functions is my alternative. I would prefer to have PQsetSingleRowMode() over new flavor of PQsend. To consolidate my argument above: since you're throwing PQgetResult in the m

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 2:37 PM, Tom Lane wrote: > Merlin Moncure writes: >> Either way, it looks like there's plausible paths to optimizing >> repeated result fetch without having expose an alternate data-fetching >> API and that the proposed implementation doesn&#

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 1:49 PM, Merlin Moncure wrote: > On Tue, Jul 24, 2012 at 1:33 PM, Merlin Moncure wrote: >> The 'source' result (or source data that would be copied into the >> destination result) would be stored in the PGconn, right? So, the idea >> is

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 1:33 PM, Merlin Moncure wrote: > The 'source' result (or source data that would be copied into the > destination result) would be stored in the PGconn, right? So, the idea > is that when you set up single row mode the connection generates a > templat

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:57 AM, Marko Kreen wrote: > On Tue, Jul 24, 2012 at 7:52 PM, Merlin Moncure wrote: >> But, the faster rowbuf method is a generally incompatible way of >> dealing with data vs current libpq -- this is bad. If it's truly >> impossible to

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:39 AM, Marko Kreen wrote: > On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure wrote: >> On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen wrote: >>>> I'm arguing that *all* data getting must continue to do so through the >>>> result o

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen wrote: >> I'm arguing that *all* data getting must continue to do so through the >> result object, and bypassing the result to get at data is breaking the >> result abstraction in the libpq api. I bet you can still maintain >> data access through resu

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 10:49 AM, Tom Lane wrote: > Merlin Moncure writes: >> I'm arguing that *all* data getting must continue to do so through the >> result object, and bypassing the result to get at data is breaking the >> result abstraction in the libpq api. > &

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Mon, Jul 23, 2012 at 5:07 PM, Marko Kreen wrote: >> Does PQgetRowData() break the ability to call PQgetvalue() against the >> result as well as other functions like PQgetisnull()? If so, I >> object. > > I don't get what are you objecting to. The PQgetRowData() > is called instead of PQgetRes

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-23 Thread Merlin Moncure
On Mon, Jul 23, 2012 at 2:05 PM, Marko Kreen wrote: > > Here is a simple test program that takes a SELECT > query, reads it and outputs a COPY-formatted stream > to standard output, to simulate some activity. > > It operates on 3 modes, specified by commant-line switches: > > -f Load full result

Re: [HACKERS] row literal problem

2012-07-20 Thread Merlin Moncure
On Fri, Jul 20, 2012 at 1:31 AM, Tom Lane wrote: > I wrote: >> I think the way to solve this is to do whatever it takes to get access >> to the subplan targetlist. We could then do something a bit cleaner >> than what the named-rowtype code is currently doing: if there are >> resjunk columns in t

Re: [HACKERS] row literal problem

2012-07-18 Thread Merlin Moncure
On Wed, Jul 18, 2012 at 3:56 PM, Merlin Moncure wrote: > hm, it's the 'group by' -- for example if you add group by > pg_namespace.oid, group by pg_namespace.oid || 'abc', you can invent > columns that come back into the rowtype. here's a cut down example:

Re: [HACKERS] row literal problem

2012-07-18 Thread Merlin Moncure
On Wed, Jul 18, 2012 at 3:42 PM, Andrew Dunstan wrote: > > On 07/18/2012 03:30 PM, Tom Lane wrote: >> >> Andrew Dunstan writes: >>> >>> On 07/18/2012 03:18 PM, Merlin Moncure wrote: >>>> >>>> there are no null fields, right? if the la

Re: [HACKERS] row literal problem

2012-07-18 Thread Merlin Moncure
On Wed, Jul 18, 2012 at 1:58 PM, Andrew Dunstan wrote: > I'm chasing up an issue from a client who has this problem (in 9.1): > > with q as > ( > some query here > ) > select q.* from q > > yields: > >job_scope | checked_col > -

Re: [HACKERS] Schema version management

2012-07-11 Thread Merlin Moncure
On Tue, Jul 10, 2012 at 5:24 PM, Merlin Moncure wrote: > On Sun, May 20, 2012 at 2:41 PM, Joel Jacobson wrote: >> Hi, >> >> I just read a very interesting post about "schema version management". >> >> Quote: "You could set it up so that every de

Re: [HACKERS] Schema version management

2012-07-10 Thread Merlin Moncure
On Sun, May 20, 2012 at 2:41 PM, Joel Jacobson wrote: > Hi, > > I just read a very interesting post about "schema version management". > > Quote: "You could set it up so that every developer gets their own > test database, sets up the schema there, takes a dump, and checks that > in. There are goi

Re: [HACKERS] Posix Shared Mem patch

2012-06-29 Thread Merlin Moncure
On Fri, Jun 29, 2012 at 2:52 PM, Andres Freund wrote: > Hi All, > > In a *very* quick patch I tested using huge pages/MAP_HUGETLB for the mmap'ed > memory. > That gives around 9.5% performance benefit in a read-only pgbench run (-n -S - > j 64 -c 64 -T 10 -M prepared, scale 200, 6GB s_b, 8 cores,

Re: [HACKERS] Update on the spinlock->pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-06-29 Thread Merlin Moncure
On Fri, Jun 29, 2012 at 12:11 PM, Andres Freund wrote: > On Friday, June 29, 2012 07:07:11 PM Nils Goroll wrote: >> > Also, 20 transactions per connection is not enough of a run to make >> > any evaluation on. >> >> As you can see I've repeated the tests 10 times. I've tested slight >> variations

Re: [HACKERS] proof concept - access to session variables on client side

2012-06-26 Thread Merlin Moncure
On Tue, Jun 26, 2012 at 3:05 AM, Magnus Hagander wrote: > But with a small change to psql they could, without the need for a > whole new type of variable. For example, psql could set all those > variable as "psql.", which could then be accessed > from the DO PL code just fine. That's a really nea

Re: [HACKERS] why roll-your-own s_lock? / improving scalability

2012-06-26 Thread Merlin Moncure
On Tue, Jun 26, 2012 at 12:02 PM, Nils Goroll wrote: > Hi, > > I am currently trying to understand what looks like really bad scalability of > 9.1.3 on a 64core 512GB RAM system: the system runs OK when at 30% usr, but > only > marginal amounts of additional load seem to push it to 70% and the ap

Re: [HACKERS] libpq compression

2012-06-25 Thread Merlin Moncure
On Mon, Jun 25, 2012 at 3:38 PM, Euler Taveira wrote: > On 25-06-2012 16:45, Florian Pflug wrote: >> Agreed. If we extend the protocol to support compression, and not rely >> on SSL, then let's pick one of these LZ77-style compressors, and let's >> integrate it into our tree. >> > If we have an op

Re: [HACKERS] pgsql_fdw in contrib

2012-06-19 Thread Merlin Moncure
On Tue, Jun 19, 2012 at 10:15 AM, Kohei KaiGai wrote: > Let me push the pgsql_fdw in core from different perspective. > > Right now, FDW is a feature that will take many enhancement in > the near future like join-pushdown, writable APIs and so on. > If we would not have a FDW extension in core tha

Re: [HACKERS] pgsql_fdw in contrib

2012-06-19 Thread Merlin Moncure
On Mon, Jun 18, 2012 at 11:01 PM, Robert Haas wrote: > On Mon, Jun 18, 2012 at 12:24 PM, Merlin Moncure wrote: >> I can't help but wonder (having been down the contrib/core/extension >> road myself) if it isn't better to improve the facilities to register >> and

Re: [HACKERS] pgsql_fdw in contrib

2012-06-18 Thread Merlin Moncure
On Thu, Jun 14, 2012 at 7:29 AM, Shigeru HANADA wrote: > I'd like to propose pgsql_fdw, FDW for PostgreSQL, as a contrib module > in core, again.  This patch is basically rebased version of the patches > proposed in 9.2 development cycle, and contains some additional changes > such as concern abou

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-06-15 Thread Merlin Moncure
On Fri, Jun 15, 2012 at 1:21 PM, Marko Kreen wrote: > The row-processor API is now in 9.2, but it solves only the > "different-row-storage" problem, but not the "one-row-at-a-time" > problem, as libpq is still in control until all rows are received. > > This means libpq cannet still be used to imp

Re: [HACKERS] measuring spinning

2012-06-15 Thread Merlin Moncure
On Thu, Jun 14, 2012 at 4:39 PM, Robert Haas wrote: > On Wed, Jan 11, 2012 at 8:48 PM, Robert Haas wrote: >> I've had cause, a few times this development cycle, to want to measure >> the amount of spinning on each lwlock in the system.  To that end, >> I've found the attached patch useful.  Note

Re: [HACKERS] libpq compression

2012-06-15 Thread Merlin Moncure
On Fri, Jun 15, 2012 at 5:48 AM, Florian Pflug wrote: > On Jun15, 2012, at 12:09 , Magnus Hagander wrote: >> On Fri, Jun 15, 2012 at 5:52 PM, Florian Pflug wrote: >>> On Jun15, 2012, at 07:50 , Magnus Hagander wrote: Second, we also have things like the JDBC driver and the .Net driver t

Re: [HACKERS] libpq compression

2012-06-14 Thread Merlin Moncure
On Thu, Jun 14, 2012 at 1:43 PM, Tom Lane wrote: > So I've got very little patience with the idea of "let's put in some > hooks and then great things will happen".  It would be far better all > around if we supported exactly one, well-chosen, method.  But really > I still don't see a reason not to

Re: [HACKERS] libpq compression

2012-06-14 Thread Merlin Moncure
On Thu, Jun 14, 2012 at 9:57 AM, Phil Sorber wrote: > It doesn't sound like there is a lot of support for this idea, but I > think it would be nice to get something like lz4 > (http://code.google.com/p/lz4/) or snappy > (http://code.google.com/p/snappy/) support. Both are BSD-ish licensed. > It co

Re: [HACKERS] Is cachedFetchXidStatus provably valid?

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 3:55 PM, Tom Lane wrote: > Merlin Moncure writes: >> It's probably an academic concern, but what happens if a backend saves >> off cachedFetchXidStatus and then sleeps for a very long time.  During >> that time an xid wraparound happens an

Re: [HACKERS] temporal support patch

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 4:10 PM, Miroslav Šimulčík wrote: > I have working patch for postgresql version 9.0.4, but it needs refactoring > before i can submit it, because some parts don't > meet formatting requirements yet. And yes, changes are large, so it will be > better to discuss design first

[HACKERS] Is cachedFetchXidStatus provably valid?

2012-06-13 Thread Merlin Moncure
It's probably an academic concern, but what happens if a backend saves off cachedFetchXidStatus and then sleeps for a very long time. During that time an xid wraparound happens and the backend wakes up and happens to read another unhinted tuple with the same xid and a different commit status. Thi

Re: [HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 9:40 AM, Andres Freund wrote: > Hi Merlin, > > On Wednesday, June 13, 2012 04:21:12 PM Merlin Moncure wrote: >> On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund > wrote: >> > +synchronized catalog at the decoding site. That adds some complex

Re: [HACKERS] hint bit i/o reduction

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 9:02 AM, Amit Kapila wrote: >> Yes, but only in the unhinted case -- in oltp workloads tuples get >> hinted fairly quickly so I doubt this would be a huge impact.  Hinted >> scans will work exactly as they do now.  In the unhinted case for OLTP >> a  few tests are added but

Re: [HACKERS] [PATCH 16/16] current version of the design document

2012-06-13 Thread Merlin Moncure
On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund wrote: > +synchronized catalog at the decoding site. That adds some complexity to use > +cases like replicating into a different database or cross-version > +replication. For those it is relatively straight-forward to develop a proxy > pg > +instance

<    3   4   5   6   7   8   9   10   11   12   >