Re: [HACKERS] Desirable pgbench features?

2016-03-30 Thread Josh berkus
lid, :lbal \vlog balancelog :lid, :lbal It would create a file called: 2247.balancelog.varlog and/or append a line: 2016-03-30 21:37:33.899, 511, 2150 This would allow CSV logging of all sorts of user custom information, including de-facto response times. -- -- Josh Berkus Red Hat OSAS (any

[HACKERS] Please correct/improve wiki page about abbreviated keys bug

2016-03-29 Thread Josh berkus
) -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] JPUG wants to have a copyright notice on the translated doc

2016-03-08 Thread Josh berkus
original copyright notice. You can *add* whatever you want. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New competition from Microsoft?

2016-03-07 Thread Josh berkus
On 03/07/2016 01:43 PM, Josh berkus wrote: All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our better features, like R support. Sorry

[HACKERS] New competition from Microsoft?

2016-03-07 Thread Josh berkus
All, http://blogs.microsoft.com/?p=67248 Once SQL Server is available on Linux, we're going to see more people using it as an alternative to PostgreSQL. Especially since they're picking up a lot of our better features, like R support. -- -- Josh Berkus Red Hat OSAS (any opinions are my own

Re: [HACKERS] How can we expand PostgreSQL ecosystem?

2016-03-07 Thread Josh berkus
.linuxfoundation.org/ > * Which software/services in what category should we address preferentially? > What software would many users desire to be interoperable when migrating > from commercial databases? > What is the effective way to absorb user requests for this? Is it > enou

Re: [HACKERS] The plan for FDW-based sharding

2016-03-02 Thread Josh berkus
u can just work on the individual FDW features, which *everyone* thinks are a good idea, and when most of them are done, FDW-based scaleout will be such an obvious solution that nobody will argue with it. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers ma

Re: [HACKERS] about google summer of code 2016

2016-02-19 Thread Josh berkus
to mentor this project with Oleg. +1 -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Josh berkus
system handle it seems good enough. What I like about this is that if I want to expose it to a non-superuser, I can just do a GRANT instead of needing to write a security definer view. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Josh berkus
timeline) which is exposed ONLY in pg_controldata. That leaves no reasonable way to expose this information in an API. (and yes, we have a bigger issue with stuff which is only in pg_log, but one thing at a time) -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql

Re: [HACKERS] Declarative partitioning

2016-02-15 Thread Josh berkus
not going to use CE for the new partitioning long-term, are we? This is just the first version, right? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

[HACKERS] Code of Conduct plan

2016-01-26 Thread Josh Berkus
members are unhappy with the amount of list traffic devoted to the subject of CoCs. As such, if you have comments on the plan above, please email the core team instead of replying on-list, or wait for the committee and address comments to them. --Josh Berkus PostgreSQL Core Team -- Sent via p

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-07 Thread Josh Berkus
t of people's imperfections. Yeah, we could also get rid of this conversation: "Here's a patch for X, which is on the TODO list" "Oh, we've obsolesced that, that was added to the TODO before we had Y" ... by auto-closing TODO items at a certain age. -- Josh Berkus Red Hat O

Re: [HACKERS] Another little thing about psql wrapped expanded output

2015-12-02 Thread Josh Berkus
xt on one row is kinda painful, and not at all useful. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Fwd: [HACKERS] Another little thing about psql wrapped expanded output

2015-12-02 Thread Josh Berkus
e separating the rows. This matters a lot if > only a few of the column values are very wide: everywhere else, there's > gonna be lots of whitespace. What pager lets me scroll right infinitely? Because I wanna install that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

[HACKERS] Beta2 Next week

2015-11-05 Thread Josh Berkus
Folks, We are going to try to get Beta2 wrapped on Monday for a release next week. So if you're currently working on a 9.5 Open Item, getting it checked in tommorrow or this weekend would be great. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] Request: pg_cancel_backend variant that handles 'idle in transaction' sessions

2015-11-04 Thread Josh Berkus
e a transaction is normally very > fast and in a corner case it becomes extremely slow and disruptive to > the rest of the system. In those cases, having a timeout for it is > valuable. I could see a use for both, having written scripts which do both. -- Josh Berkus PostgreSQL Expert

Re: [HACKERS] Patent warning about the Greenplum source code

2015-11-02 Thread Josh Berkus
ial useful outcome. How about we terminate it now? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ALTER SYSTEM vs symlink

2015-11-02 Thread Josh Berkus
users avoid the "mistake" I just made. > > Frankly, that behavior strikes me as a good idea. There is no situation, > IMV, where it's sane to try to put a symlink there. So, just a doc patch then? Since we have both include files and config_dir, I really don't understand why anyone symli

Re: [HACKERS] Patent warning about the Greenplum source code

2015-11-01 Thread Josh Berkus
>> viewing Citus Data source code has the same problems as Greenplum. > > Actually, it might only be their closed source software that contains > patents, i.e. not pg_shard. I will check and report back when I can > unless someone else reports here first. I will ask Citus Data for an

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2015-10-30 Thread Josh Berkus
n complexity. +1 -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"

2015-10-29 Thread Josh Berkus
ike refactoring --- in particular, we'd > usually not consider refactoring as fit material for back-patching. > > "Refactoring" seems rather a narrow definition of what might show up > in such a category, btw. Maybe "Code Beautification" would be a > suitable title?

Re: [HACKERS] Is there any ordering to the values in guc.c?

2015-10-28 Thread Josh Berkus
o the end of the list? > > The initial commit grouped them logically, and it went downhill from > there. :) Yeah, we're overdue for another overhaul of GUC ordering. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2015-10-27 Thread Josh Berkus
veral URLs should be easier to understand, easier to > test, easier to code, and easier to keep from blowing up badly. > > > Setting aside all other concerns, have a +1 from me on that too. I'm good with this. +1 -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rework the way multixact truncations work

2015-10-27 Thread Josh Berkus
act? > Well, multixacts are a lot larger than the other SLRUs, I think that > makes some sort of difference. And by "a lot larger" we're talking like 50X to 100X. I regularly see pg_multixact directories larger than 1GB. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] plpython is broken for recursive use

2015-10-15 Thread Josh Berkus
tate of the globals dict as a feature? That is, make use of the fact that you can store session-persistent data in it? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-10-13 Thread Josh Berkus
n'. I think having two different syntaxes is a bad idea. I'd rather have a wholly proprietary configuration markup than deal with two alternate ones. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] Release of CVEs

2015-10-11 Thread Josh Berkus
more prompt. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bugs and bug tracking

2015-10-09 Thread Josh Berkus
in the queue. I have yet to see a bug tracker which does this particular task well, so please don't emulate existing art. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] bugs and bug tracking

2015-10-07 Thread Josh Berkus
On 10/07/2015 10:25 AM, Alvaro Herrera wrote: > Hmm, I guess we could have the bug form add > To: n...@bugs.postgresql.org > CC: pgsql-b...@postgresql.org > as headers, which should work for most people (since we reply-all), Josh > Berkus being the exception. Well, this will jus

Re: [HACKERS] bugs and bug tracking

2015-10-07 Thread Josh Berkus
On 10/07/2015 11:05 AM, Alvaro Herrera wrote: > Josh Berkus wrote: >> On 10/07/2015 10:25 AM, Alvaro Herrera wrote: >>> Hmm, I guess we could have the bug form add >>> To: n...@bugs.postgresql.org >>> CC: pgsql-b...@postgresql.org >>> as headers, whic

Re: [HACKERS] bugs and bug tracking

2015-10-06 Thread Josh Berkus
On 10/06/2015 12:03 PM, Bruce Momjian wrote: > On Tue, Oct 6, 2015 at 03:33:20PM -0300, Alvaro Herrera wrote: >> Joshua D. Drake wrote: >>> On 10/06/2015 10:57 AM, Josh Berkus wrote: >>>> On 10/06/2015 10:17 AM, Bruce Momjian wrote: >> >>>> Speak

Re: [HACKERS] bugs and bug tracking

2015-10-06 Thread Josh Berkus
onvert to "stale" status. Until we build up a team of volunteers for bug triage, we might have to do that. Speaking of which ... this project is rich in skilled users who are involved in the community but don't code. Bug triage is exactly the kind of thing very part-time community suppor

Re: [HACKERS] No Issue Tracker - Say it Ain't So!]

2015-10-04 Thread Josh Berkus
rity an afterthought, which makes my teeth itch, but I think layers > of security would make it much less likely to be actually adopted. I think there needs to be some kind of administrative access which allows, for example, an issue to be closed so that it can't be reopened. Anyway, I'm not

Re: [HACKERS] Idea for improving buildfarm robustness

2015-10-03 Thread Josh Berkus
luding it in the releases, speak up ... Wait, we're backpatching this? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-10-01 Thread Josh Berkus
specific error message the user is getting, which is a minority of the time. So in addition to what Haas mentions, I think we want to be able to link the release notes to the original issues for our hypothetical bug tracker. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-01 Thread Josh Berkus
d, >> for example, log messages, function names, variables, etc. > > I'd be inclined to keep calling it the visibility map (vm) even if it > also contains freeze information. > -1 to rename. Visibility Map is a perfectly good name. -- Josh Berkus PostgreSQL Experts Inc. htt

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-30 Thread Josh Berkus
> explain clearly how things would improve. Also, consider low impact > solutions first (for example what low tech method makes bug > identification to release easier?) and move into a big tooling change > only after discarding them. Well, if you have suggestions that don't involve

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-30 Thread Josh Berkus
Github Issues. If it's more than that, you're going to have to explain. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-30 Thread Josh Berkus
On 09/30/2015 03:27 PM, Tom Lane wrote: > Josh Berkus <j...@agliodbs.com> writes: >> On 09/30/2015 03:10 PM, Tom Lane wrote: >>> I'd be feeling a lot more positive about this whole thread if any people >>> had stepped up and said "yes, *I* will put in a l

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-30 Thread Josh Berkus
this thread is a waste of time, just as the several similar ones > before it have been. Hmmm? Frost volunteered to stand up debbugs. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-30 Thread Josh Berkus
p: rm -rf /pgdata/* cp -p -r /pgdata2/* /pgdata/ ... as expected, this did NOT cause postgres to shut down. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.post

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 12:47 PM, Tom Lane wrote: > Josh Berkus <j...@agliodbs.com> writes: >> In general, having the postmaster survive deletion of PGDATA is >> suboptimal. In rare cases of having it survive installation of a new >> PGDATA (via PITR restore, for example)

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Josh Berkus
all of the above requires a bug *tracker*, that is, a tool which tracks the bug activity which was happening anyway, just makes it more visible. Rather than an Issue Resolution System, which would be intended to remake our workflow. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
s suboptimal. In rare cases of having it survive installation of a new PGDATA (via PITR restore, for example), I've even seen the zombie postmaster corrupt the data files. So if you want this change to be useful beyond the buildfarm, it should check every few minutes, and you'd SIGQUIT. -- Josh Berku

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
ld die, rather than a small list that cause us > not to. But as long as we're reasonably confident that we're seeing an > error that means somebody deleted pg_control, I think abandoning ship > is just fine. Give me source with the change, and I'll put it on a cheap, low-bandwith AWS in

Re: [HACKERS] 9.3.9 and pg_multixact corruption

2015-09-28 Thread Josh Berkus
doesn't have a bug and our stuff is > blowing up, then we have a bug and should fix it. I suppose there > could be some grey area but hopefully not too much. Or it's PILBChAK. I know Sun-CC used to warn that -O3 was unsuitable for most programs because it could change behavior. -- Josh

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Josh Berkus
y would we bother? The infra team seems to be good with debbugs, and several committers seem to like it, why not go with it? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-25 Thread Josh Berkus
he only time this info is required is for people that provide a Support > service based upon PostgreSQL, yet are not themselves sufficiently > involved to know what bugs have been reported and are as yet unfixed. I > expect such people are extremely interested in getting other people to

Re: [HACKERS] multivariate statistics / patch v7

2015-09-24 Thread Josh Berkus
a attributes as well, yes? I have a destruction test case for correlated column stats, so I'd like to test your patch on it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-24 Thread Josh Berkus
; https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=util-linux;dist=unstable > > A specific bug: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=786804 I adore "Toggle useless messages" as a feature. ;-) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-24 Thread Josh Berkus
ce, we won't go breaking everyone's links when we change the domain name. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-24 Thread Josh Berkus
age says "Fixes bug #1234" > and the state automatically goes to FIXED. I don't know debbugs, but I know that it would be possible to program RT to do all of the above, except add the item to the commitfest. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgs

Re: [HACKERS] Decimal64 and Decimal128

2015-09-24 Thread Josh Berkus
le. > Yes. Please just build an external extension and submit it to PGXN. Thanks! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-23 Thread Josh Berkus
trivial code fixes might be a side benefit, but isn't a good enough reason to have one. Obviously, everything said about "who's going to maintain this" is completely valid. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-23 Thread Josh Berkus
ould be willing to help here. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-23 Thread Josh Berkus
d Jira to be superior to OSS BT systems, and inferior in several ways (like that you can't have more than one person assigned to a bug). And email integration for Jira is nonexistant. When we discussed this 8 years ago, Debian said debbugs wasn't ready for anyone else to use. Has that changed? -- Jo

Re: [HACKERS] Rework the way multixact truncations work

2015-09-21 Thread Josh Berkus
ltixact truncation at all? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Streaming Replication clusters and load balancing

2015-09-21 Thread Josh Berkus
My solution will depend on patroni's included HTTP access, though, so I'm not sure it will work for you. Anyway, this isn't a topic for pgsql-hackers mailing list, so reply offlist if you want to discuss further. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hac

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-09-07 Thread Josh Berkus
m version: 0 -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Horizontal scalability/sharding

2015-09-03 Thread Josh Berkus
a feature, too, is nice. (* there are actually other ways to come close to simultaneity, but they are much more complicated) -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Freeze avoidance of very large table.

2015-09-03 Thread Josh Berkus
by toplevel make install; I'm not suggesting that the >>> extension is installed automatically. For that, you still need a >>> superuser to run CREATE EXTENSION. >>> >> >> +! for this > > OK, what does "+!" mean? (I know it is probably a shif

Re: [HACKERS] Horizontal scalability/sharding

2015-09-02 Thread Josh Berkus
On 09/01/2015 04:14 PM, Petr Jelinek wrote: > On 2015-09-02 00:09, Josh Berkus wrote: >> On 09/01/2015 02:29 PM, Tomas Vondra wrote: >>> So while you may be right in single-DC deployments, with multi-DC >>> deployments the situation is quite different - not only tha

Re: [HACKERS] Horizontal scalability/sharding

2015-09-02 Thread Josh Berkus
On 09/02/2015 11:41 AM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 1:57 PM, Josh Berkus <j...@agliodbs.com> wrote: >> Even if it's only on paper, any new sharding design needs to address >> these questions: >> >> 1. How do we ensure no/minimal data is lost i

Re: [HACKERS] Horizontal scalability/sharding

2015-09-02 Thread Josh Berkus
On 09/02/2015 12:30 PM, Robert Haas wrote: > On Wed, Sep 2, 2015 at 3:03 PM, Josh Berkus <j...@agliodbs.com> wrote: >>> 4. Therefore, I think that we should instead use logical replication, >>> which might be either synchronous or asynchronous. When you modi

Re: [HACKERS] Allow a per-tablespace effective_io_concurrency setting

2015-09-02 Thread Josh Berkus
But we actually say this in the docs: My experience with performance tuning is that values above 3 have no real effect on how queries are executed. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-09-02 Thread Josh Berkus
or the pg_controldata output it processes the > pg_control file directly, and for pg_config it relies on compile-time > CPPFLAGS. > > I think trying to duplicate the exact strings isn't too nice an > interface. Well, for pg_controldata, no, but what else would you do for pg_config? -- Josh B

Re: [HACKERS] Horizontal scalability/sharding

2015-09-01 Thread Josh Berkus
ee a strong place for binary replication and BDR for multi-region redundancy; you really don't want that to be part of the sharding system if you're aiming for write scalability. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] Horizontal scalability/sharding

2015-09-01 Thread Josh Berkus
On 09/01/2015 02:39 AM, Bruce Momjian wrote: > On Mon, Aug 31, 2015 at 01:16:21PM -0700, Josh Berkus wrote: >> I'm also going to pontificate that, for a future solution, we should not >> focus on write *IO*, but rather on CPU and RAM. The reason for this >> thinking is

Re: [HACKERS] Horizontal scalability/sharding

2015-09-01 Thread Josh Berkus
On 09/01/2015 10:17 AM, Robert Haas wrote: > On Tue, Sep 1, 2015 at 1:06 PM, Josh Berkus <j...@agliodbs.com> wrote: >> Any sharding solution worth bothering with will solve some or all of the >> above by extending our ability to process requests across multiple >> node

Re: [HACKERS] Horizontal scalability/sharding

2015-09-01 Thread Josh Berkus
On 09/01/2015 02:29 PM, Tomas Vondra wrote: > Hi, > > On 09/01/2015 09:19 PM, Josh Berkus wrote: >> Other way around, that is, having replication standbys as the only >> method of redundancy requires either high data loss or high latency >> for all writes. > > I

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Josh Berkus
illed user; that is, requiring a special C sharding function for this seems fine to me. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Josh Berkus
On 08/31/2015 02:47 PM, Robert Haas wrote: > On Mon, Aug 31, 2015 at 4:16 PM, Josh Berkus <j...@agliodbs.com> wrote: >> First, let me put out there that I think the horizontal scaling project >> which has buy-in from the community and we're working on is infinitely >>

Re: [HACKERS] Planned release for PostgreSQL 9.5

2015-08-25 Thread Josh Berkus
/wiki/PostgreSQL_9.5_Open_Items When that list gets down to a handful of non-critical items, we'll be in beta. Help wanted. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Declarative partitioning

2015-08-23 Thread Josh Berkus
On 08/21/2015 08:34 PM, Jim Nasby wrote: On 8/18/15 12:31 PM, Josh Berkus wrote: Also this would be useful for range partitions: CREATE PARTITION ON parent_table USING ( start_value ); ... where start_value is the start range of the new partition. Again, easier for users to get correct

Re: [HACKERS] Declarative partitioning

2015-08-20 Thread Josh Berkus
be in the first cut even if they require an access exclusive lock. Cheers, David. I don't see a way for them to *ever* not require an access exclusive lock. We could eventually implement: DETACH PARTITION CONCURRENTLY ... but that's the only way I can see around it. -- Josh Berkus

Re: [HACKERS] jsonb array-style subscripting

2015-08-20 Thread Josh Berkus
On 08/20/2015 12:24 PM, Jim Nasby wrote: On 8/17/15 4:25 PM, Josh Berkus wrote: On 08/17/2015 02:18 PM, Jim Nasby wrote: On 8/17/15 3:33 PM, Josh Berkus wrote: Again, how do we handle missing keys? Just return NULL? or ERROR? I'd prefer the former, but there will be arguments the other

Re: [HACKERS] Declarative partitioning

2015-08-19 Thread Josh Berkus
ON (columns) INCREMENT BY (INTERVAL '1 month' ) START WITH value; Oh, I like that syntax! How would it work if there were multiple columns? Maybe we don't want to allow that for this form? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-19 Thread Josh Berkus
many users deliberately use CTEs as an optimization barrier in order to force the planner. Given that, we need some kind of option to force the old behavior; either SQL syntax or a GUC option. Otherwise this will cause a bunch of backwards-compatibility breakage. Ideas? -- Josh Berkus PostgreSQL

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-19 Thread Josh Berkus
On 08/19/2015 01:32 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: On 08/18/2015 04:40 PM, Qingqing Zhou wrote: Attached please find the WIP patch and also the ANALYZE results. Notes: the patch may not directly apply to head as some network issue here so my Linux box can't talk

Re: [HACKERS] Declarative partitioning

2015-08-19 Thread Josh Berkus
On 08/19/2015 01:18 PM, Thom Brown wrote: On 19 August 2015 at 21:10, Josh Berkus j...@agliodbs.com mailto:j...@agliodbs.com wrote: On 08/19/2015 04:59 AM, Simon Riggs wrote: I like the idea of a regular partitioning step because it is how you design such tables - lets use

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread Josh Berkus
that partition. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] More WITH

2015-08-17 Thread Josh Berkus
EXPLAIN [ANALYZE] Would be tricky. We don't currently have any way to wrap an EXPLAIN in any larger statement, do we? Would be very useful for automated query analysis, though. SHOW Not very useful, easy to work around (pg_settings). -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] jsonb array-style subscripting

2015-08-17 Thread Josh Berkus
On 08/17/2015 02:18 PM, Jim Nasby wrote: On 8/17/15 3:33 PM, Josh Berkus wrote: Again, how do we handle missing keys? Just return NULL? or ERROR? I'd prefer the former, but there will be arguments the other way. I've been wondering if we should add some kind of strict JSON. My big

Re: [HACKERS] jsonb array-style subscripting

2015-08-17 Thread Josh Berkus
of in this implementation? array/key ambiguity is going to be painful. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: SCRAM authentication

2015-08-13 Thread Josh Berkus
. That is, giving DBAs the ability to see and log who's using what kind of verifier, and what account has what verifier(s) available, will make more of a difference. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] WIP: SCRAM authentication

2015-08-12 Thread Josh Berkus
difference in migrations is for PGAAS hosting. But the folks from Heroku and AWS have been notably silent on this; lemme ping them. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] WIP: SCRAM authentication

2015-08-11 Thread Josh Berkus
On 08/11/2015 07:28 AM, Robert Haas wrote: There may be a good answer to this question, but I don't think I've seen it spelled out clearly. Please see my follow-up post about making by-login-role migration easier for users. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] WIP: SCRAM authentication

2015-08-11 Thread Josh Berkus
On 08/11/2015 09:35 AM, Robert Haas wrote: On Tue, Aug 11, 2015 at 12:29 PM, Josh Berkus j...@agliodbs.com wrote: On 08/11/2015 07:28 AM, Robert Haas wrote: There may be a good answer to this question, but I don't think I've seen it spelled out clearly. Please see my follow-up post about

Re: [HACKERS] WIP: SCRAM authentication

2015-08-11 Thread Josh Berkus
On 08/11/2015 10:06 AM, Robert Haas wrote: On Tue, Aug 11, 2015 at 12:49 PM, Josh Berkus j...@agliodbs.com wrote: That makes sense if drivers go that way. I'm concerned that some drivers will have a different call for a SCRAM connection than for an MD5 one; we'd want to exert our project

Re: [HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-08-10 Thread Josh Berkus
about forced wraparound. But that's a different argument. BTW, has it occured to anyone that implementing XID epoch headers is going to mean messing with multixact logs again? Just thought I'd open a papercut and pour some lemon juice on it. -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-08-10 Thread Josh Berkus
vacuum update the visibility map in the same way vacuum freeze does? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: SCRAM authentication

2015-08-10 Thread Josh Berkus
to the parent role; 3. move all applications to using SCRAM and the new role; 4. disable logins on the old, parent role. It's currently (2) which is painfully difficult, and could be made less so via the two features recommended above. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] WIP: SCRAM authentication

2015-08-09 Thread Josh Berkus
potential of taking the password field and multiplexing it in some way, which is significant. There is a definite risk of making this too complicated and we'll need to contrast that against ease-of-migration, because complicated mechanisms tend to be less secure due to user error. -- Josh Berkus

Re: [HACKERS] Test code is worth the space

2015-08-08 Thread Josh Berkus
be run during development less frequently. I thought about doing the extended set as a satellite project, but that may not be workable. There already is, isn't there? All of those named sets of regression tests which aren't run by default. -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Josh Berkus
too much implied authorization to me, and liable to be a big foot-gun. --Josh Berkus -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Josh Berkus
On 08/08/2015 03:21 PM, Michael Paquier wrote: On Sun, Aug 9, 2015 at 6:51 AM, Josh Berkus j...@agliodbs.com wrote: 1. we drop the parameter password_encryption 2. we add the parameter password_storage, which takes a list: - plain : plain text - md5 : current md5 hashes - scram

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-07 Thread Josh Berkus
Petr, Just user-tested SYSTEM_ROWS and SYSTEM_TIME. They work as expected. Useful! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-06 Thread Josh Berkus
for someone to write in the future. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-06 Thread Josh Berkus
On 08/06/2015 01:14 PM, Josh Berkus wrote: On 08/06/2015 01:10 PM, Simon Riggs wrote: Given, user-stated probability of accessing a block of P and N total blocks, there are a few ways to implement block sampling. 1. Test P for each block individually. This gives a range of possible results

<    1   2   3   4   5   6   7   8   9   10   >