Re: [HACKERS] ANALYZE sampling is too good

2013-12-12 Thread Josh Berkus
high, the expected % of observed values should be "discounted" somehow. That is, with total random distribution you have a 1:1 ratio between observed frequency of a value and assumed frequency. However, with highly grouped values, you might have a 2:1 ratio. Again, more math (backed by

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Josh Berkus
different "block sizes" on different FSes before we arrive at a reasonable size, and even then I'll bet we have to offer a GUC. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 12:40 PM, Tom Lane wrote: > Josh Berkus writes: >> And, for that matter, accepting this patch by no means blocks doing >> something more sophisticated in the future. > > Yeah. I think the only real argument against it is "do we really need > yet anoth

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
means blocks doing something more sophisticated in the future. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
improvement over what we have now. +1 There is an inherent tuning and troubleshooting challenge in anything involving a feedback loop. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread Josh Berkus
regexes to filter out the noise error messages from the really, really important ones. Speaking as someone who does trainings for new DBAs, the part where I do "what to look for in the logs" requires over an hour and still doesn't cover everything. And doesn't internationalize.

Re: [HACKERS] ANALYZE sampling is too good

2013-12-10 Thread Josh Berkus
ou're confident that the blocks you sampled are representative of the population. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] plpgsql_check_function - rebase for 9.3

2013-12-10 Thread Josh Berkus
useful. I can't think of one, offhand. And +1 for NOT adding a new knob. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ANALYZE sampling is too good

2013-12-10 Thread Josh Berkus
iscussed, we need math though. Does anyone have an ACM subscription and time to do a search? Someone must. We can buy one with community funds, but no reason to do so if we don't have to. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers maili

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Josh Berkus
On 12/10/2013 10:48 AM, Andres Freund wrote: > On 2013-12-10 10:44:30 -0800, Josh Berkus wrote: >> On 12/10/2013 10:39 AM, Andres Freund wrote: >>> Hi, >>> >>> On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: >>>> We've just run across a case

Re: [HACKERS] coredump of 9.3.2

2013-12-10 Thread Josh Berkus
#x27;m not suprised that it doesn't work ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Josh Berkus
On 12/10/2013 10:39 AM, Andres Freund wrote: > Hi, > > On 2013-12-10 10:38:32 -0800, Josh Berkus wrote: >> We've just run across a case of this exact issue on 9.2.4. I thought it >> was supposed to be 9.3-only? > > Could you please describe "this exact

Re: [HACKERS] Errors on missing pg_subtrans/ files with 9.3

2013-12-10 Thread Josh Berkus
Andres, all: We've just run across a case of this exact issue on 9.2.4. I thought it was supposed to be 9.3-only? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] ANALYZE sampling is too good

2013-12-09 Thread Josh Berkus
3000. 3. Other algos exist. The tend to be wrong in other directions. 4. Nobody has done an analysis of whether it's worse, on average, to estimate low vs. high for ndistinct. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] ANALYZE sampling is too good

2013-12-09 Thread Josh Berkus
ata from table pages while they're in memory anyway for other reasons. You do seem kind of hostile to the idea of full-page-sampling, going pretty far beyond the "I'd need to see the math". Why? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] ANALYZE sampling is too good

2013-12-08 Thread Josh Berkus
could search for this; there are several published papers, with math and pseudocode. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dblink performance regression

2013-12-07 Thread Josh Berkus
All, I tested out Joe's original patch, and it does eliminate the 8% performance regression. Will try the new one. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] ANALYZE sampling is too good

2013-12-07 Thread Josh Berkus
umes that all columns are randomly distributed. If we could recoginze common distribution patterns, then not only could we have better query estimates, those would require keeping *fewer* stats, since all you need for a normal distribution are the end points and the variance. -- Josh Berkus Postg

Re: [HACKERS] WITHIN GROUP patch

2013-12-06 Thread Josh Berkus
xamples of the two syntax alternatives we're talking about here? I've lost track. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ANALYZE sampling is too good

2013-12-05 Thread Josh Berkus
SQL use block-based sampling, but of course, I don't know which specific algo they use. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Josh Berkus
On 12/05/2013 12:41 PM, Robert Haas wrote: > On Thu, Dec 5, 2013 at 12:54 PM, Josh Berkus wrote: >> Actually, I've been able to do 35K TPS on commodity hardware on Ubuntu >> 10.04. I have yet to go about 15K on any Ubuntu running a 3.X Kernel. >> The CPU scheduling o

Re: [HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Josh Berkus
quot;OMG, there's something wrong with the server", that is, a flag for messages which only occur when PostgreSQL encounters a bug, data corrpution, or platform error. Right now, I have to suss those out by regex. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com --

Re: [HACKERS] Re: [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-05 Thread Josh Berkus
things are all over the place, and get worse with each new feature added. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Josh Berkus
On 12/05/2013 05:48 AM, Stephen Frost wrote: > * Peter Geoghegan (p...@heroku.com) wrote: >> On Wed, Dec 4, 2013 at 11:07 AM, Josh Berkus wrote: >>> But you know what? 2.6, overall, still performs better than any kernel >>> in the 3.X series, at least for Postgres.

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Josh Berkus
bably be using persistent RAM. And the geeks who manage that have already said that MMAP is a bad interface for persistent RAM. They haven't defined a good one, though. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-04 Thread Josh Berkus
WN when I'm travelling. > Seriously, though, one of the best things to do would be to make a point of > picking up a kernel around -rc3 (right around now, say, for 3.13) and > running a few benchmarks on it. If you report a performance regression at > that stage, it will get attention

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-04 Thread Josh Berkus
earn about new changes to IO or memory performance in the Linux Kernel without reading all of the traffic on LKML and all Linux commit messages and filtering them myself. If there were a better way to look for this information, Linux would be more likely to get feedback in a timely fashion. A

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Josh Berkus
I'd rather do the testing thing as a community thing, though, which means raising non-profit money and having an open bid process for the person to do the work. I think we cold raise more money that way, and are likely to get a better end result. -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Josh Berkus
omated performace tests will require paying someone like Greg S., Mark or me for 6 solid months to develop them, since worthwhile open source performance test platforms currently don't exist. That money has never been available; maybe I should do a kickstarter. -- Josh Berkus PostgreSQL Expert

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Josh Berkus
the IO stack; they really don't believe that anyone who cares about performance should be using it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Why we are going to have to go DirectIO

2013-12-03 Thread Josh Berkus
with the potential to destroy PostgreSQL performance without warning, leaving us scrambling for workarounds. Too bad the alternatives are so unpopular. I don't know where we'll get the resources to implement our own storage, but it's looking like we don't have a choice. -

Re: [HACKERS] Visual Studio 2013 build

2013-12-02 Thread Josh Berkus
there are no documented limitations that make any problems > foreseeable. > I will add it to the CommitFest 2014-01 so that there is time for > testing and tweaking. Wow, thanks for doing this! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hacke

Re: [HACKERS] Draft release notes for 9.3.2

2013-12-02 Thread Josh Berkus
ce in this case? If so, what? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pre-commit triggers

2013-11-26 Thread Josh Berkus
On 11/26/2013 09:45 AM, Tom Lane wrote: > Josh Berkus writes: >> On 11/24/2013 06:42 AM, Simon Riggs wrote: >>> I think we should be thinking harder about how to implement >>> ASSERTIONs, possibly calling them ASSERTION TRIGGERs not pre-commit >>> write eve

Re: [HACKERS] why semicolon after begin is not allowed in postgresql?

2013-11-26 Thread Josh Berkus
, or why Perl requires them, or why Java has all those squiggly brackets. PostgreSQL is very accepting of new procedural languages; you could always create your own, with whatever syntax you want. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] pre-commit triggers

2013-11-26 Thread Josh Berkus
On 11/24/2013 06:42 AM, Simon Riggs wrote: > I think we should be thinking harder about how to implement > ASSERTIONs, possibly calling them ASSERTION TRIGGERs not pre-commit > write event triggers. I don't know that anyone is working on this, though, or even plans to. -- Josh Ber

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-26 Thread Josh Berkus
ile step forwards. That being said, the other requirement I am concerned about is being able to support the syntax of this feature in commonly used ORMs. That is, can I write a fairly small Django or Rails extension which does upsert using this patch? Fortunately, I think I can ... -- Josh B

Re: [HACKERS] Can we trust fsync?

2013-11-22 Thread Josh Berkus
al website doesn't inspire much confidence. > > Well, it was his idea, and quite a good one. I guess we could > reimplement this in C if someone wants to do the legwork. Yeah, too bad Brad didn't post a license for it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexpert

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-21 Thread Josh Berkus
in, if there's no activity on the master, you can't trigger the bug? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-21 Thread Josh Berkus
the replica, in case it already has corruption. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] additional json functionality

2013-11-21 Thread Josh Berkus
rmats: BSON, MessagePack ( > http://msgpack.org/), protobuf, hierarchical H-Store is coming along. > Adding another one would create confusion, and a situation similar to this: > http://xkcd.com/927/ Apparently you missed that JSONB is just Hstore2 with different in/out functions? > Po

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
e an old base backup around and all WAL files, > you can start from that. > > Does that answer your questsions? Yeah, thanks. If you have any ideas for how we'd write code to scan for this kind of corruption, please post them. -- Josh Berkus PostgreSQL Experts Inc. http://pgexpert

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
On 11/20/2013 10:30 AM, Josh Berkus wrote: > Andrews, Kevin: Andres, that is. > > Presumably a replica created while all traffic was halted on the master > would be clean, correct? This bug can only be triggered if there's > heavy write load on the master, right? > Al

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
Andrews, Kevin: Presumably a replica created while all traffic was halted on the master would be clean, correct? This bug can only be triggered if there's heavy write load on the master, right? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers ma

Re: [HACKERS] Storage formats for JSON WAS: additional json functionality

2013-11-20 Thread Josh Berkus
y-update-in-place seems like a clear win for heirarchical data storage, but practically it's not. Of course, an investigation into this by someone with much more knowledge of low-level storage than me (most of this list) is welcome. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] additional json functionality

2013-11-20 Thread Josh Berkus
e? Why go to a lot of trouble to extend our home-grown > format if we've got a standard format to plug into? See hstore2 patch from Teodor. That's what we're talking about, not hstore1, which as you point out is non-heirarchical. -- Josh Berkus PostgreSQL Experts Inc. http://pg

Re: [HACKERS] -d option for pg_isready is broken

2013-11-20 Thread Josh Berkus
ns instead of needing to go through the shell (as I currently do with pg_isready). -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-19 Thread Josh Berkus
ge some testing on our side. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pre-commit triggers

2013-11-19 Thread Josh Berkus
ge and complex accounting infrastructure to track which tables were modified and how. This is the sort of thing the SQL committee covered in ASSERTIONS, but of course they never specified any method for implementation. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- S

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-19 Thread Josh Berkus
t onesided, > isn't it? Nope, just that you should be worried about it, and maybe shouting for help. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-19 Thread Josh Berkus
need to halt all work on replication until we have such a thing. I can't see how you expect to complete streaming logical replication without a replication testing framework. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-19 Thread Josh Berkus
w that? We certainly do know that we won't catch these kinds of bugs *without* testing. Heck, even MySQL AB had automated testing for replication. They largely ignored the results, but they had it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mai

Re: [HACKERS] -d option for pg_isready is broken

2013-11-19 Thread Josh Berkus
ng the exact same code as psql? Why does pg_isready have its own code for this? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-19 Thread Josh Berkus
On 11/19/2013 09:58 AM, Andres Freund wrote: > On 2013-11-19 09:51:28 -0800, Josh Berkus wrote: >> Maybe it's worth it now to devise some automated replication testing? > > It'd be a good idea, but I am not sure where to get resources for it > from. Well, servers

Re: [HACKERS] More legacy code: pg_ctl

2013-11-19 Thread Josh Berkus
On 11/19/2013 08:29 AM, Robert Haas wrote: > On Mon, Nov 18, 2013 at 8:20 PM, Josh Berkus wrote: >> Oh, and one more: >> >> c) that "stop" defaults to "smart" mode, instead of "fast" mode. > > And that "smart" mode is called

Re: [HACKERS] additional json functionality

2013-11-19 Thread Josh Berkus
no fragments) * whitespace not significant * reordering of keys * duplicate keys prohibited If people want other manglings of JSON, they can use TEXT fields and custom parsers written in PL/v8, the same way I do. -- Josh Berkus PostgreSQL Experts Inc. http://pgexp

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-19 Thread Josh Berkus
4eedc3f0f347a856eea8590730769125964597 > > Yes, the combination of those is guilty. > > Man, this is (to a good part my) bad. > >> But that'd mean nobody noticed it during 9.3's beta... Ah, so this affected 9.3.0 as well? Maybe it's worth it now to devise

Re: [HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
On 11/18/2013 05:09 PM, Josh Berkus wrote: > Folks, > > Speaking of legacy code with bad default behaviors: pg_ctl. The current > utility is designed to fathfully reproduce the rather hackish shell > script we originally had for postgres startup. This results in a > cou

[HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
1999-era shell script. I don't know if the answer is to rename the utility like we're discussing with pg_dump/pg_backup, or something else. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Josh Berkus
es they were for the same client (9.0.14 and 9.3.1), one for a > different client (9.2.5). To emphasize a salient point: we have not previously seen data corruption with these exact symptoms prior to the recent patch release. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com --

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Josh Berkus
expect MVCC bugs, which is going to be an almost zero > population. +1 None of our users would willingly choose worse performance over the 0.1% possibility of needing to analyze a transaction failure. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hacke

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
that version doesn't have to be the default if users don't specify. Note that this raises the issue of "first alternate data type ambiguity" again for overloading builtin functions. We really need that method of "prefering" a specific version of the function ... -- Josh

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
On 11/18/2013 06:13 AM, Peter Eisentraut wrote: > On 11/15/13, 6:15 PM, Josh Berkus wrote: >> Thing is, I'm not particularly concerned about *Merlin's* specific use >> case, which there are ways around. What I am concerned about is that we >> may have users who ha

Re: [HACKERS] additional json functionality

2013-11-17 Thread Josh Berkus
g to do with Java. Besides, we might get sued by these people: http://www.jstor.org/ ;-) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] additional json functionality

2013-11-16 Thread Josh Berkus
NB is as close as we can get to "JSON", and it gives people the idea that this is the successor type to JSON. So +1 from me as well on "JSONB". -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] additional json functionality

2013-11-15 Thread Josh Berkus
t with their "standard". That's a reason why we won't use the name BSON, either, since it's a trademark of 10gen. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] additional json functionality

2013-11-15 Thread Josh Berkus
the new version 10.0. That's why naming old JSON as "json_text" won't work; it'll be a hardened roadblock to upgrading. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] additional json functionality

2013-11-15 Thread Josh Berkus
written yet. I believe this was a danger we recognized when we added the JSON type, including the possibility that a future binary type might need to be a separate type due to compatibility issues. The only sad thing is the naming; it would be better for the new type to carry the JSON name in

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-15 Thread Josh Berkus
nd if/when we find some use for them re add them Wait, which Josh? Not me ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] additional json functionality

2013-11-15 Thread Josh Berkus
7;s not at all workable. Users would be completely unable to predict or understand the JSON type and how it acts. That's not just violating POLS; that's bashing POLS' head in with a shovel. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

[HACKERS] -d option for pg_isready is broken

2013-11-13 Thread Josh Berkus
7;t do anything I suggest simply removing it from the utility. I'll note that the -U option doesn't appear to do anything relevant either, but at least it doesn't error unnecessarily: handyrep@john:~/handyrep$ pg_isready -h john -p 5432 -U no_such_user john:5432 - ac

Re: [HACKERS] additional json functionality

2013-11-13 Thread Josh Berkus
of XML. Now, your use case does show us that there's a case to be made for still having text JSON even after we have binary JSON. There's a strong simplicity argument against that, though ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] additional json functionality

2013-11-13 Thread Josh Berkus
ROM Where it becomes even more useful is when you want the json label to be the result of a calculated expression: SELECT build_json_object ( department, sum() ) Yes, you could do this with a two-array version as well; it's just not more intuitive, and in cases where you have dozen

Re: [HACKERS] pg_dump and pg_dumpall in real life

2013-11-11 Thread Josh Berkus
On 11/11/2013 03:06 PM, David Johnston wrote: > Josh Berkus wrote >> Well, then we just need pg_restore to handle the "role already exists" >> error message gracefully. That's all. Or a "CREATE ROLE IF NOT EXISTS" >> statement, and use that for r

Re: [HACKERS] pg_dump and pg_dumpall in real life

2013-11-11 Thread Josh Berkus
ff like this. > As for 'what we need', I'd think someone with the time and energy to > write the patch and work with the community to implement it.. +1 -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Changing pg_dump default file format

2013-11-08 Thread Josh Berkus
and folks *keep* reporting to pgsql-security. Even though the 8.1 driver had been officially deprecated for 2 years before the security issue was discovered. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Josh Berkus
All, I'm amused at how this has become a "let's pile on everything which has ever been missing in pg_dump into one thread". Agree with Tom that if we're going to create a new program name, we should fix the pg_dumpall issue as well. -- Josh Berkus PostgreSQL Experts

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Josh Berkus
to send to STDOUT. If we wanted to change the default format, we would need to make "filename" a required parameter. If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. -

Re: [HACKERS] [v9.4] row level security

2013-11-06 Thread Josh Berkus
All, Just a comment: I'm really glad to see the serious work on this. If RLS doesn't make it into 9.4, it'll be because the problems of RLS are fundamentally unsolvable, not because we didn't give it our best. Great work, all! -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] List of "binary-compatible" data types

2013-11-05 Thread Josh Berkus
s" from "need to rewrite values" could be useful for a lot of type conversions. I'd also love some way of doing a no-rewrite conversion between timestamp and timestamptz, based on the assumption that the original values are UTC time. That's one I encounter a l

Re: [HACKERS] List of "binary-compatible" data types

2013-11-05 Thread Josh Berkus
e surprised how painful that is. Of course, if we get binary JSON in 9.4 (Oleg?), then a binary conversion will be required, so maybe it's a moot point. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] List of "binary-compatible" data types

2013-11-05 Thread Josh Berkus
xt --> JSON require a physical rewrite? We have to validate it, sure, but we don't need to rewrite it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] List of "binary-compatible" data types

2013-11-04 Thread Josh Berkus
On 11/04/2013 05:21 PM, Josh Berkus wrote: > Thom, > > >> SELECT >> castsource::regtype::text, >> array_agg(casttarget::regtype order by casttarget::regtype::text) >> casttargets >> FROM pg_cast >> WHERE castmethod = 'b' >> GR

Re: [HACKERS] List of "binary-compatible" data types

2013-11-04 Thread Josh Berkus
Thom, > SELECT > castsource::regtype::text, > array_agg(casttarget::regtype order by casttarget::regtype::text) > casttargets > FROM pg_cast > WHERE castmethod = 'b' > GROUP BY 1 > ORDER BY 1; Are we actually covering 100% of these for ALTER COLUMN now?

[HACKERS] List of "binary-compatible" data types

2013-11-04 Thread Josh Berkus
the new type or an unconstrained domain over the new type, a table rewrite is not needed ..." Which is nice, but nowhere do we present users with a set of binary-compatible data types, even among the built-in types. I'd happily write this up, if I knew what the binary-compatible data types

Re: [HACKERS] CLUSTER FREEZE

2013-10-25 Thread Josh Berkus
s still bad policy for the project. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CLUSTER FREEZE

2013-10-24 Thread Josh Berkus
On 10/24/2013 04:55 PM, Robert Haas wrote: > On Thu, Oct 24, 2013 at 1:09 PM, Josh Berkus wrote: >> On 10/23/2013 09:58 PM, Amit Kapila wrote: >>> I wonder why anyone would like to freeze during CLUSTER command when >>> they already have separate way (VACUUM FREEZE) to

Re: [HACKERS] Location for external scripts for Extensions?

2013-10-24 Thread Josh Berkus
On 10/24/2013 02:36 PM, Peter Eisentraut wrote: > On 10/22/13, 2:27 PM, Josh Berkus wrote: >> pg_partman has several external (python) scripts which help the >> extension, located in /extras/ in its source. The problem currently is >> that if you install pg_partman via pgxn

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Josh Berkus
oint it's too late to enable it. So if there's any way we can avoid having a GUC for this, I'm for it. And if we do have a GUC, failback should be enabled by default. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Josh Berkus
On 10/24/2013 11:12 AM, Heikki Linnakangas wrote: > On 24.10.2013 20:39, Josh Berkus wrote: >> On 10/24/2013 04:15 AM, Pavan Deolasee wrote: >>> If we do what you are suggesting, it seems like a single line patch >>> to me. >>> In XLogSaveBufferForHint

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Josh Berkus
ional GUC? I'm opposed to the idea of having a GUC to enable failback. When would anyone using replication ever want to disable that? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-10-24 Thread Josh Berkus
if we had portable atomic addition, so that we could spare the > spinlock ... Oh, sure, just take the *easy* way out. ;-) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] CLUSTER FREEZE

2013-10-24 Thread Josh Berkus
I'm rewriting the table anyway, let's freeze it". Otherwise, you have to write the same pages twice, if both CLUSTER and FREEZE are required. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-10-23 Thread Josh Berkus
would be nice, and IMHO the only way we can really resolve all of these stats concerns. Any approach we take using histograms etc. is going to leave out some stat someone needs. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

[HACKERS] Autotuning and RAM survey results

2013-10-23 Thread Josh Berkus
dd an example doc of recalibrating for small RAM, but I don't think that's that hard. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commitfest II CLosed

2013-10-23 Thread Josh Berkus
rategies. It's not an effective discussion pattern when dealing with new code contributors either, or even some old ones. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Location for external scripts for Extensions?

2013-10-23 Thread Josh Berkus
Dimitri, > Josh Berkus writes: >> pg_partman has several external (python) scripts which help the >> extension, located in /extras/ in its source. The problem currently is >> that if you install pg_partman via pgxn or package, you don't get those >> scri

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-10-23 Thread Josh Berkus
useful. That is, I can answer the question in many cases of: "Does this query have a high average because of outliers, or because it's consisently slow?" by looking at the STDDEV. And FWIW, for sites where we monitor pg_stat_statements, we reset daily or weekly. Otherwise, the stats ha

Re: [HACKERS] Commitfest II CLosed

2013-10-22 Thread Josh Berkus
On 10/21/2013 11:59 AM, Tom Lane wrote: > Josh Berkus writes: >> Either you're proposing a solution, supporting someone else's solution, >> or you're saying the problem isn't important. There is no fourth >> alternative. > > Nonsense. Pointing out

[HACKERS] Location for external scripts for Extensions?

2013-10-22 Thread Josh Berkus
here should these go? PGSHARE/pg_partman/? Or somewhere else? I'm looking to set a canonical location for extensions with external scripts. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

<    5   6   7   8   9   10   11   12   13   14   >