Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 03:25, Amit Kapila wrote: > On Wed, Sep 2, 2015 at 11:35 AM, Etsuro Fujita >> >> The UPDATE/DELETE pushdown, which I've proposed, would ensure the sane >> behaviour for inherited UPDATEs/DELETEs, as existing non-pushed-down >> UPDATE/DELETE does, because inheritance_planner guaran

Re: [HACKERS] Multi-tenancy with RLS

On Fri, Aug 14, 2015 at 12:00 PM, Haribabu Kommi wrote: > > Here I attached the proof concept patch. Here I attached an updated patch by adding policies to the most of the system catalog tables, except the following. AggregateRelationId AccessMethodRelationId AccessMethodOperatorRelationId Acces

Re: [HACKERS] Horizontal scalability/sharding

On Wed, Sep 2, 2015 at 11:35 AM, Etsuro Fujita wrote: > On 2015/09/02 14:28, Amit Langote wrote: > >> On 2015-09-02 PM 01:28, Amit Kapila wrote: >> >>> On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas wrote: > I'm not averse to making the "connect to the remote nodes" part of > this

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 6:13 AM, Jim Nasby wrote: > On 9/1/15 11:59 PM, Michael Paquier wrote: >> >> On Wed, Sep 2, 2015 at 12:59 PM, Jim Nasby wrote: >>> >>> On 9/1/15 8:42 PM, Michael Paquier wrote: >>> The crash is triggered by having an exception raised in this particular >>> call >>> stack. Si

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/1/15 11:59 PM, Michael Paquier wrote: On Wed, Sep 2, 2015 at 12:59 PM, Jim Nasby wrote: On 9/1/15 8:42 PM, Michael Paquier wrote: The crash is triggered by having an exception raised in this particular call stack. Since there's no syntax error in master/0.2.1, there's no assert failure eith

Re: [HACKERS] Horizontal scalability/sharding

On 2015/09/02 14:28, Amit Langote wrote: On 2015-09-02 PM 01:28, Amit Kapila wrote: On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas wrote: I'm not averse to making the "connect to the remote nodes" part of this solution use something other than the FDW infrastructure at some point in time if someb

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

On 07/25/2015 03:38 AM, dinesh kumar wrote: > > > On Fri, Jul 24, 2015 at 10:22 AM, Robert Haas > wrote: > > On Thu, Jul 23, 2015 at 8:15 PM, dinesh kumar > mailto:dineshkuma...@gmail.com>> wrote: > > On Thu, Jul 23, 2015 at 9:21 AM, Robert Haas > m

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 PM 01:28, Amit Kapila wrote: >> On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas wrote: >>> >>> I'm not averse to making the "connect to the remote nodes" part of >>> this solution use something other than the FDW infrastructure at some >>> point in time if somebody's prepared to build so

Re: [HACKERS] perlcritic

On Tue, Sep 01, 2015 at 11:26:27AM -0400, Robert Haas wrote: > On Tue, Sep 1, 2015 at 9:58 AM, Andrew Dunstan wrote: > > On 08/31/2015 11:57 PM, Peter Eisentraut wrote: > >> We now have 80+ Perl files in our tree, and it's growing. Some of those > >> files were originally written for Perl 4, and

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 12:59 PM, Jim Nasby wrote: > On 9/1/15 8:42 PM, Michael Paquier wrote: > The crash is triggered by having an exception raised in this particular call > stack. Since there's no syntax error in master/0.2.1, there's no assert > failure either. Were you running asserts? I thoug

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/1/15 8:42 PM, Michael Paquier wrote: test_factory is a jungle to me. Perhaps you could just extract a self-contained test case? It does not matter if the file is long as long as the problem can be easily reproduced. Sorry, more info on what's happening here. The idea behind test_factory i

Re: [HACKERS] security labels on databases are bad for dump & restore

On Mon, Aug 31, 2015 at 05:46:08PM -0400, Bruce Momjian wrote: > On Tue, Jul 28, 2015 at 04:23:36PM -0300, Alvaro Herrera wrote: > > Josh Berkus wrote: > > > On 07/28/2015 11:58 AM, Robert Haas wrote: > > > > I'd be strongly in favour of teaching GRANT, SECURITY LABEL, COMMENT > > > >> ON DATABASE,

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 4:25 PM, Bruce Momjian wrote: > > > The document opens a big question --- when queries can't be processed in > a traditional top/down fashion, Citus has the goal of sending groups of > results up the the coordinator, reordering them, then sending them back > to the shards fo

Re: [HACKERS] Horizontal scalability/sharding

> On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas wrote: > > > > On Tue, Sep 1, 2015 at 4:15 AM, Andres Freund wrote: > > > On 2015-08-31 20:54:51 -0400, Bruce Momjian wrote: > > >> Uh, we already have a list of things we need to add to FDWs to make them > > >> work, and Citus Data has provided a doc

Re: [HACKERS] [PATCH] SQL function to report log message

2015-09-02 0:13 GMT+02:00 Jim Nasby : > On 9/1/15 12:47 AM, Pavel Stehule wrote: > >> >> Wouldn't it be better to create an ENUM of error levels instead of >> inventing more parsing code? >> >> >> Do you think SQL ENUM? I little bit afraid about possible problems with >> pg_upgrade. >> >>

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/1/15 8:42 PM, Michael Paquier wrote: On Wed, Sep 2, 2015 at 9:39 AM, Michael Paquier wrote: On Wed, Sep 2, 2015 at 7:23 AM, Jim Nasby wrote: Well nuts, pretty sure that means the error isn't reproducing for you. :/ Do you maybe have unusual config options or postgresql.conf options? Yep

Re: [HACKERS] synchronous_commit = apply

On 1 September 2015 at 20:25, Thomas Munro wrote: > Hi > > Do you think it's reasonable to want to COMMIT a particular transaction on a > master node, and then immediately run a read-only query on a hot standby > node that is guaranteed to see that transaction? > well, that is important to make l

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 9:39 AM, Michael Paquier wrote: > On Wed, Sep 2, 2015 at 7:23 AM, Jim Nasby wrote: >> Well nuts, pretty sure that means the error isn't reproducing for you. :/ Do >> you maybe have unusual config options or postgresql.conf options? > > Yep. And I have found one reason why it

Re: [HACKERS] upgrade failure from 9.5 to head

On Wed, Jul 29, 2015 at 11:01:40AM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2015-07-29 10:38:19 -0400, Tom Lane wrote: > >> Now as far as dummy_seclabel is concerned, the easy answer is "we don't > >> care". But on reflection, doesn't this mean that the entire > >> implementation of

[HACKERS] synchronous_commit = apply

Hi Do you think it's reasonable to want to COMMIT a particular transaction on a master node, and then immediately run a read-only query on a hot standby node that is guaranteed to see that transaction? A friend of mine who works with a different RDBMS technology that can do that asked me how to a

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On Wed, Sep 2, 2015 at 7:23 AM, Jim Nasby wrote: > Well nuts, pretty sure that means the error isn't reproducing for you. :/ Do > you maybe have unusual config options or postgresql.conf options? Yep. And I have found one reason why it was not working, I have been using --extra-version with a cust

Re: [HACKERS] Shouldn't we document "don't use a mountpoint as $PGDATA"?

On Tue, Sep 1, 2015 at 08:20:47PM -0400, Bruce Momjian wrote: > On Tue, Jul 28, 2015 at 02:19:00PM -0400, Andrew Dunstan wrote: > > > > On 07/28/2015 02:01 PM, Tom Lane wrote: > > >I had a discussion with some folks at Red Hat about this: > > >https://bugzilla.redhat.com/show_bug.cgi?id=1247477 >

Re: [HACKERS] Shouldn't we document "don't use a mountpoint as $PGDATA"?

On Tue, Jul 28, 2015 at 02:19:00PM -0400, Andrew Dunstan wrote: > > On 07/28/2015 02:01 PM, Tom Lane wrote: > >I had a discussion with some folks at Red Hat about this: > >https://bugzilla.redhat.com/show_bug.cgi?id=1247477 > > > >I had the idea that we had documented somewhere that the data direc

Re: [HACKERS] Unicode mapping scripts cleanup

> I discovered that some of the source files that one is supposed to > download don't exist anymore or are labeled obsolete. Also, running the > scripts produces slight differences in the output. So apparently, the > CJK to Unicode mappings are still evolving and should be updated > occasionally.

Re: [HACKERS] Anyone working on the TOAST items on the TODO list?

On Sat, Jul 25, 2015 at 04:06:28PM -0400, Tom Lane wrote: > Bill Moran writes: > > On Sat, 25 Jul 2015 11:39:15 -0700 > > Josh Berkus wrote: > >> Nope. In fact, even the one which was 90% complete (replacing zlib with > >> lz4) completely dropped off the radar. > > > Interesting ... I wasn't lo

Re: [HACKERS] Unicode mapping scripts cleanup

> On Tue, Sep 1, 2015 at 5:13 AM, Peter Eisentraut wrote: >> So apparently, the >> CJK to Unicode mappings are still evolving and should be updated >> occasionally. Next steps would be to commit some or all of these >> differences after additional verification, and then update the scripts >> to

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-02 00:09, Josh Berkus wrote: > On 09/01/2015 02:29 PM, Tomas Vondra wrote: >> For example assume that there are just two shards in two separate data >> centers, connected by a link with limited bandwidth. Now, let's assume >> you always keep a local replica for failover. So you have A1+A

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On 2015-08-04 23:37:08 +0300, Ildus Kurbangaliev wrote: > diff --git a/src/backend/access/transam/clog.c > b/src/backend/access/transam/clog.c > index 3a58f1e..10c25cf 100644 > --- a/src/backend/access/transam/clog.c > +++ b/src/backend/access/transam/clog.c > @@ -457,7 +457,8 @@ CLOGShmemInit(voi

Re: [HACKERS] WIP: About CMake v2

Greg Stark wrote: > It is tempting and I've been wanting to evalangize this approach ever > since read http://aegis.sourceforge.net/auug97.pdf but I've never > actually had practical experience with it and iirc it's always this > scenario of wanting to compile submodules multiple times that blocke

Re: [HACKERS] Fwd: Core dump with nested CREATE TEMP TABLE

On 9/1/15 1:08 AM, Michael Paquier wrote: On Sun, Aug 30, 2015 at 1:06 AM, Jim Nasby wrote: Steps to reproduce: Download https://github.com/BlueTreble/test_factory/archive/crash.zip Unzip, cd into directory pgxn install pgtap (or just make test) FWIW, make test fails: ! ERROR: 42703: column "

[HACKERS] remove wal_level archive

So we've had several rounds of discussions about simplifying replication configuration in general and the wal_level setting in particular. [0][1] Let's get something going. While we have not reached a complete consensus yet, a few things have become clear: - We would like to have fewer or easier

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 10:17 AM, Robert Haas wrote: > On Tue, Sep 1, 2015 at 1:06 PM, Josh Berkus wrote: >> You're assuming that our primary bottleneck for writes is IO. It's not >> at present for most users, and it certainly won't be in the future. You >> need to move your thinking on systems

Re: [HACKERS] [PATCH] SQL function to report log message

On 9/1/15 12:47 AM, Pavel Stehule wrote: Wouldn't it be better to create an ENUM of error levels instead of inventing more parsing code? Do you think SQL ENUM? I little bit afraid about possible problems with pg_upgrade. It is not simple question - the ENUM can be interesting from cus

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 12:40:40PM -0400, Robert Haas wrote: > On Tue, Sep 1, 2015 at 6:55 AM, Bruce Momjian wrote: > > I assumed these queries were going to be solved by sending as digested > > data as possible to the coordinator, and having the coordinator complete > > any remaining processing.

Re: [HACKERS] WIP: About CMake v2

On Tue, Sep 1, 2015 at 2:41 PM, Robert Haas wrote: > Maybe we should merge all of the makefiles for subdirectories of > src/backend into a single makefile. The major disadvantage would be > that you couldn't rebuild a subdirectory any more by typing make -C > src/backend/executor or whatever. An

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 02:29 PM, Tomas Vondra wrote: > Hi, > > On 09/01/2015 09:19 PM, Josh Berkus wrote: >> Other way around, that is, having replication standbys as the only >> method of redundancy requires either high data loss or high latency >> for all writes. > > I haven't said that. I said that we

Re: [HACKERS] Horizontal scalability/sharding

On Sep 1, 2015, at 1:47 PM, Robert Haas wrote: > Admittedly, there are some problems with snapshots here: if you don't > do anything special about snapshots, then what you have here will be > "eventually consistent" behavior. But that might be suitable for some > environments, such as very loose

Re: [HACKERS] Unicode mapping scripts cleanup

On Tue, Sep 1, 2015 at 5:13 AM, Peter Eisentraut wrote: > So apparently, the > CJK to Unicode mappings are still evolving and should be updated > occasionally. Next steps would be to commit some or all of these > differences after additional verification, and then update the scripts > to use wh

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 08:18:38AM -0700, Joshua Drake wrote: > On 09/01/2015 02:48 AM, Bruce Momjian wrote: > >On Tue, Sep 1, 2015 at 09:30:41AM +0530, Pavan Deolasee wrote: > > >There is no question that using XC/XL will get us to a usable solution > >faster, but see my recent post to Josh Berk

Re: [HACKERS] Horizontal scalability/sharding

Hi, On 09/01/2015 09:19 PM, Josh Berkus wrote: On 09/01/2015 11:36 AM, Tomas Vondra wrote: We want multiple copies of shards created by the sharding system itself. Having a separate, and completely orthagonal, redundancy system to the sharding system is overly burdensome on the DBA and makes lo

Re: [HACKERS] SimpleTee flush

Kevin Grittner wrote: > I find the TestLib.pm framework to be more friendly with the > attached tweak to SimpleTee.pm. Pushed. Please ignore the typo in the commit message -- I was looking right at the code saying "autoflush", but what came off my fingers (from muscle memory?) was "autocommit".

Re: [HACKERS] HINTing on UPDATE foo SET foo.bar = ..;

On 2015-09-01 23:19, I wrote: Hopefully nobody minds if I slip this to the commit fest that started today? The attached patch should address all the comments from the 9.5 cycle. Apparently the CF app will. Meh. Whatever. Ignore then, I guess. .m -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] Turning off HOT/Cleanup sometimes

Hi Simon, On 2015-04-20 19:21:24 +0100, Simon Riggs wrote: > Since we have many votes in favour of change in this area I'll post a new > version and look for an early review/commit for next release. If I see correctly there's been no new patch version since, right? The patch is included in the cu

Re: [HACKERS] HINTing on UPDATE foo SET foo.bar = ..;

Hi, Hopefully nobody minds if I slip this to the commit fest that started today? The attached patch should address all the comments from the 9.5 cycle. .m *** a/src/backend/parser/analyze.c --- b/src/backend/parser/analyze.c *** *** 2107,2118 transformUpdateTargetList(Parse

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 3:19 PM, Josh Berkus wrote: > Mind you, I see a strong place for binary replication and BDR for > multi-region redundancy; you really don't want that to be part of the > sharding system if you're aiming for write scalability. I mostly agree, but keep in mind that you don't

Re: [HACKERS] 9.4 broken on alpha

On Tue, Sep 1, 2015 at 4:00 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Sep 1, 2015 at 2:54 PM, Andres Freund wrote: >>> On 2015-09-01 14:40:36 -0400, Robert Haas wrote: The best argument for continuing to support Alpha is probably that Linux does. > >>> Not sure why that's an

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

On Mon, Aug 31, 2015 at 8:01 AM, Ashutosh Bapat wrote: >> Thanks. It needs testing though to see if it really works as >> intended. Can you look into that? > > PFA the patch containing your code changes + test module. See if that meets > your expectations. Thanks. I don't think this test modul

Re: [HACKERS] 9.4 broken on alpha

On 09/01/2015 01:18 PM, Peter Geoghegan wrote: On Tue, Sep 1, 2015 at 1:00 PM, Tom Lane wrote: I think we've probably beat this to death. Nobody here believes that it's sane to try to support Alpha without access to hardware, and no offer of hardware has been forthcoming. If one were to mater

Re: [HACKERS] 9.4 broken on alpha

On Tue, Sep 1, 2015 at 1:00 PM, Tom Lane wrote: > I think we've probably beat this to death. Nobody here believes that > it's sane to try to support Alpha without access to hardware, and no > offer of hardware has been forthcoming. If one were to materialize, > we could usefully have a debate ab

Re: [HACKERS] 9.4 broken on alpha

Robert Haas writes: > On Tue, Sep 1, 2015 at 2:54 PM, Andres Freund wrote: >> On 2015-09-01 14:40:36 -0400, Robert Haas wrote: >>> The best argument for continuing to support Alpha is probably that >>> Linux does. >> Not sure why that's an argument? I mean linux supports architectures >> without

Re: [HACKERS] 9.4 broken on alpha

On Tue, Sep 1, 2015 at 2:54 PM, Andres Freund wrote: > On 2015-09-01 14:40:36 -0400, Robert Haas wrote: >> I doubt there is a big problem with supporting Alpha apart from >> lock-free algorithms. > > Note that we've had lock-free algorithms for years. E.g. the changecount > stuff in pgstat.c. Hmm

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 11:36 AM, Tomas Vondra wrote: >> We want multiple copies of shards created by the sharding system itself. >> Having a separate, and completely orthagonal, redundancy system to the >> sharding system is overly burdensome on the DBA and makes low-data-loss >> HA impossible. > > IMHO

Re: [HACKERS] Horizontal scalability/sharding

Hi, On 09/01/2015 08:22 PM, Andres Freund wrote: On 2015-09-01 14:11:21 -0400, Robert Haas wrote: On Tue, Sep 1, 2015 at 2:04 PM, Tomas Vondra wrote: Memory bandwidth, for example. It's quite difficult to spot, because the intuition is that memory is fast, but thanks to improvements in storag

Re: [HACKERS] 9.4 broken on alpha

Robert Haas writes: > The best argument for continuing to support Alpha is probably that > Linux does. I don't know how they do that. My sneaking suspicion is that they don't very well. In particular, unless I misunderstand things fundamentally, the coherency issues would be invisible without a

Re: [HACKERS] 9.4 broken on alpha

On 2015-09-01 14:40:36 -0400, Robert Haas wrote: > I doubt there is a big problem with supporting Alpha apart from > lock-free algorithms. Note that we've had lock-free algorithms for years. E.g. the changecount stuff in pgstat.c. > The best argument for continuing to support Alpha is probably th

Re: [HACKERS] 9.4 broken on alpha

On Sun, Aug 30, 2015 at 4:42 PM, Tom Lane wrote: > David Fetter writes: >> At a minimum, we should de-support every platform on which literally >> no new deployments will ever happen. >> I'm looking specifically at you, HPUX, and I could make a pretty good >> case for the idea that we can relegat

Re: [HACKERS] Horizontal scalability/sharding

Hi, On 08/31/2015 10:16 PM, Josh Berkus wrote: It's also important to recognize that there are three major use-cases for write-scalable clustering: * OLTP: small-medium cluster, absolute ACID consistency, bottlnecked on small writes per second * DW: small-large cluster, ACID optional, bot

Re: [HACKERS] [PROPOSAL] Effective storage of duplicates in B-tree index.

On Mon, Aug 31, 2015 at 12:41 AM, Anastasia Lubennikova wrote: > Now new B-tree index tuple must be inserted for each table row that we > index. > It can possibly cause page split. Because of MVCC even unique index could > contain duplicates. > Storing duplicates in posting list/tree helps to avoi

Re: [HACKERS] Horizontal scalability/sharding

On 2015-09-01 14:11:21 -0400, Robert Haas wrote: > On Tue, Sep 1, 2015 at 2:04 PM, Tomas Vondra > wrote: > > Memory bandwidth, for example. It's quite difficult to spot, because the > > intuition is that memory is fast, but thanks to improvements in storage (and > > stagnation in RAM bandwidth), t

Re: [HACKERS] Adding since-version tags to the docs?

On Mon, Aug 31, 2015 at 4:48 PM, Tom Lane wrote: > Right now, you might well care about whether a feature arrived in 9.3 vs > 9.4, for instance; but it's highly unlikely that you care whether a > feature arrived in 7.1 or 7.2. The problem with this proposal is that > it will add far more bloat of

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On 2015-09-01 14:07:19 -0400, Robert Haas wrote: > But I think it's quite wrong to assume that the infrastructure for > this is available and usable everywhere, because in my experience, > that's far from the case. Especially when the alternative is a rather short patch implementing an otherwise w

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 2:04 PM, Tomas Vondra wrote: > Memory bandwidth, for example. It's quite difficult to spot, because the > intuition is that memory is fast, but thanks to improvements in storage (and > stagnation in RAM bandwidth), this is becoming a significant issue. I'd appreciate any ti

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Tue, Sep 1, 2015 at 1:50 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Wed, Aug 19, 2015 at 9:41 AM, Tom Lane wrote: >> > That sort-of ties into what seems to me the main objection to this >> > proposal, namely that there is already a way to do this sort of thing: >> > DNS-based load ba

Re: [HACKERS] Horizontal scalability/sharding

Hi, On 09/01/2015 07:17 PM, Robert Haas wrote: On Tue, Sep 1, 2015 at 1:06 PM, Josh Berkus wrote: You're assuming that our primary bottleneck for writes is IO. It's not at present for most users, and it certainly won't be in the future. You need to move your thinking on systems resources into

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Tue, Sep 1, 2015 at 7:50 PM, Alvaro Herrera wrote: > Robert Haas wrote: > > On Wed, Aug 19, 2015 at 9:41 AM, Tom Lane wrote: > > > That sort-of ties into what seems to me the main objection to this > > > proposal, namely that there is already a way to do this sort of thing: > > > DNS-based lo

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Aug 19, 2015 at 11:06 PM, Amit Kapila wrote: > Always try with the first server specified in connection string and if that > is not available try with second and so on. I think for the case of > failover, > the design shouldn't be much complicated and it is a standard thing provided > by

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

Robert Haas wrote: > On Wed, Aug 19, 2015 at 9:41 AM, Tom Lane wrote: > > That sort-of ties into what seems to me the main objection to this > > proposal, namely that there is already a way to do this sort of thing: > > DNS-based load balancing. All the clients think they connect to > > db.mycomp

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

On Wed, Aug 19, 2015 at 9:41 AM, Tom Lane wrote: > That sort-of ties into what seems to me the main objection to this > proposal, namely that there is already a way to do this sort of thing: > DNS-based load balancing. All the clients think they connect to > db.mycompany.com, but which server the

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 10:17 AM, Robert Haas wrote: > On Tue, Sep 1, 2015 at 1:06 PM, Josh Berkus wrote: >> Any sharding solution worth bothering with will solve some or all of the >> above by extending our ability to process requests across multiple >> nodes. Any solution which does not is merely an acad

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 10:06 AM, Josh Berkus wrote: On 09/01/2015 02:39 AM, Bruce Momjian wrote: On Mon, Aug 31, 2015 at 01:16:21PM -0700, Josh Berkus wrote: Our real future bottlenecks are: * ability to handle more than a few hundred connections This, 1000 times this. No a connection pooler doesn'

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 1:06 PM, Josh Berkus wrote: > You're assuming that our primary bottleneck for writes is IO. It's not > at present for most users, and it certainly won't be in the future. You > need to move your thinking on systems resources into the 21st century, > instead of solving the

Re: [HACKERS] perlcritic

On Tue, Sep 1, 2015 at 11:58 AM, Mike Blackwell wrote: > David wrote: >> I believe there are ways to get perlcritic to keep quiet about things >> we don't find relevant. Maybe that's a better way to use it. > > There are indeed. A .perlcriticrc file can suppress (or add) either > individual rule

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 02:39 AM, Bruce Momjian wrote: > On Mon, Aug 31, 2015 at 01:16:21PM -0700, Josh Berkus wrote: >> I'm also going to pontificate that, for a future solution, we should not >> focus on write *IO*, but rather on CPU and RAM. The reason for this >> thinking is that, with the latest improve

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-01 17:20 GMT+02:00 Shulgin, Oleksandr : > I'm not familiar with the shared memory handling, but could we not >>> allocate just enough shared memory to fit the data we're going to write >>> instead of the fixed 8k? It's not that we cannot know the length of the >>> resulting plan text in

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 7:01 AM, Mason S wrote: > For efficient internodes joins with row shipping, FDWs may also not be easy > to do. Maybe it is possible if we optionally pass in lists of other nodes > and information about how they are partitioned so data knows where to get > shipped. > > A chal

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 6:55 AM, Bruce Momjian wrote: > I assumed these queries were going to be solved by sending as digested > data as possible to the coordinator, and having the coordinator complete > any remaining processing. I think we are going to need to decide if > such "sending data back

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 09:08 AM, Robert Haas wrote: On Tue, Sep 1, 2015 at 12:00 AM, Pavan Deolasee From my point of view, and EnterpriseDB's point of view, anything that doesn't go into the core PostgreSQL distribution isn't really getting us where we need to be. If there's code in XL that would be

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 4:15 AM, Andres Freund wrote: > On 2015-08-31 20:54:51 -0400, Bruce Momjian wrote: >> Uh, we already have a list of things we need to add to FDWs to make them >> work, and Citus Data has provided a document of more things that are >> needed, https://goo.gl/vJWF85. I am not

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 1, 2015 at 12:00 AM, Pavan Deolasee wrote: > My worry is that if we start implementing them again from scratch, it will > take a few years before we get them in a usable state. What XC/XL lacked is > probably a Robert Haas or a Tom Lane who could look at the work and suggest > major ed

Re: [HACKERS] WIP: About CMake v2

On Tuesday 01 September 2015 11:46:05 you wrote: > I would actually suggest that the cmake conversion would be better off > to ignore src/tools/msvc altogether to begin with. Build postgres by MSVC is important task for me. But Linux first of course. > A separate cmake build system would certain

Re: [HACKERS] perlcritic

​David wrote: ​ > I believe there are ways to get perlcritic to keep quiet about things > we don't find relevant. Maybe that's a better way to use it. > ​There are indeed. A .perlcriticrc file can suppress (or add) either individual rules or groups of rules. I use one to ignore the ones I dis

[HACKERS]

> I don't think it would be horrifically hard to change the way toast OIDs > are assigned (I'm thinking we'd basically switch to creating a sequence > for every toast table), but I don't think anyone's ever tried to push > toast hard enough to hit this kind of limit. I'd be interested in promoting

Re: [HACKERS] Horizontal scalability/sharding

On Tue, Sep 01, 2015 at 10:15:27AM +0200, Andres Freund wrote: > On 2015-08-31 20:54:51 -0400, Bruce Momjian wrote: > > Uh, we already have a list of things we need to add to FDWs to make them > > work, and Citus Data has provided a document of more things that are > > needed, https://goo.gl/vJWF85

Re: [HACKERS] WIP: Access method extendability

In general pattern of generic WAL usage is following. 1) Start using generic WAL: specify relation M-m, what about extensions which wants to use WAL but WAL record doesn't connected to any relation? For example, transaction manager or kind of FDW. GenericXLogStart(index); 2) Register buff

Re: [HACKERS] WIP: About CMake v2

Andres Freund writes: > On 2015-09-01 10:32:39 -0400, Noah Misch wrote: >> A monolithic patch replacing the GNU make build system with a CMake build >> system sounds far too hard to write and review; we should expect to maintain >> those two build systems in parallel for awhile. > I don't buy thi

Re: [HACKERS] [PROPOSAL] Effective storage of duplicates in B-tree index.

On 09/01/2015 11:31 AM, Alexander Korotkov wrote: ... Yes, In general GIN is a btree with effective duplicates handling + support of splitting single datums into multiple keys. This proposal is mostly porting duplicates handling from GIN to btree. Sure, there are differences - GIN indexes

Re: [HACKERS] perlcritic

On Tue, Sep 01, 2015 at 11:26:27AM -0400, Robert Haas wrote: > On Tue, Sep 1, 2015 at 9:58 AM, Andrew Dunstan wrote: > > On 08/31/2015 11:57 PM, Peter Eisentraut wrote: > >> We now have 80+ Perl files in our tree, and it's growing. Some > >> of those files were originally written for Perl 4, and

Re: [HACKERS] perlcritic

On Tue, Sep 1, 2015 at 9:58 AM, Andrew Dunstan wrote: > On 08/31/2015 11:57 PM, Peter Eisentraut wrote: >> We now have 80+ Perl files in our tree, and it's growing. Some of those >> files were originally written for Perl 4, and the coding styles and >> quality are quite, uh, divergent. So I figu

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 02:58 AM, Mark Kirkwood wrote: On 01/09/15 21:41, Bruce Momjian wrote: I'm thinking that partitioning and sharding are two different things: Partitioning is about reducing the amount of table data accessed and also perhaps easing admin activities (archiving/removing old stuff et

Re: [HACKERS] On-demand running query plans using auto_explain and signals

> > I'm not familiar with the shared memory handling, but could we not >> allocate just enough shared memory to fit the data we're going to write >> instead of the fixed 8k? It's not that we cannot know the length of the >> resulting plan text in advance. >> > > the shared memory cannot be reused

Re: [HACKERS] Horizontal scalability/sharding

On 09/01/2015 02:48 AM, Bruce Momjian wrote: On Tue, Sep 1, 2015 at 09:30:41AM +0530, Pavan Deolasee wrote: There is no question that using XC/XL will get us to a usable solution faster, but see my recent post to Josh Berkus --- the additional code will be so burdensome that I doubt it would

Re: [HACKERS] icc vs. gcc-style asm blocks ... maybe the twain can meet?

On Tue, Sep 1, 2015 at 10:08 AM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Aug 30, 2015 at 7:16 PM, Tom Lane wrote: >>> Hmm ... wait a second. The main collection of asm blocks in s_lock.h >>> believes that Intel's compiler will take gcc-style asm without any help: > >> The 2005 block se

Re: [HACKERS] Potential GIN vacuum bug

On Sun, Aug 30, 2015 at 6:57 PM, Tom Lane wrote: >> But we would still have to deal with the >> fact that unconditional acquire attempt by the backends will cause a vacuum >> to cancel itself, which is undesirable. > > Good point. > >> If we define a new namespace for >> this lock (like the relati

Re: [HACKERS] WIP: About CMake v2

On 2015-09-01 10:32:39 -0400, Noah Misch wrote: > A monolithic patch replacing the GNU make build system with a CMake build > system sounds far too hard to write and review; we should expect to maintain > those two build systems in parallel for awhile. I don't buy this. I don't think we're going t

Re: [HACKERS] WIP: About CMake v2

On Tue, Sep 01, 2015 at 12:49:55AM -0400, Tom Lane wrote: > Noah Misch writes: > > On Fri, Aug 28, 2015 at 01:28:49PM -0300, Alvaro Herrera wrote: > >> If it allows us to get rid of our custom MSVC scripts, it's a huge > >> benefit, for sure -- that has been a huge pain in the neck since day > >>

Re: [HACKERS] icc vs. gcc-style asm blocks ... maybe the twain can meet?

Robert Haas writes: > On Sun, Aug 30, 2015 at 7:16 PM, Tom Lane wrote: >> Hmm ... wait a second. The main collection of asm blocks in s_lock.h >> believes that Intel's compiler will take gcc-style asm without any help: > The 2005 block seems to have been the result of this thread: > http://www.

Re: [HACKERS] WIP: About CMake v2

On 2015-09-01 10:05:44 -0400, Tom Lane wrote: > Where I do *not* want to end up is maintaining *three* build systems. > So I'll definitely resist any proposals to commit a partly-done cmake > conversion (which I fear might seem attractive at some point). Enthusiastically seconded. -- Sent via p

Re: [HACKERS] WIP: About CMake v2

Robert Haas writes: > I guess I'm a bit skeptical about the idea of porting to a new build > system. There's a good chance of replacing the problems we know about > with new problems that are no less serious, but merely unknown to us. > But I'm not going to stand here and hold my breath if everyo

Re: [HACKERS] perlcritic

On 08/31/2015 11:57 PM, Peter Eisentraut wrote: We now have 80+ Perl files in our tree, and it's growing. Some of those files were originally written for Perl 4, and the coding styles and quality are quite, uh, divergent. So I figured it's time to clean up that code a bit. I ran perlcritic o

Re: [HACKERS] icc vs. gcc-style asm blocks ... maybe the twain can meet?

On Sun, Aug 30, 2015 at 7:16 PM, Tom Lane wrote: > I wrote: >> I came across some websites suggesting that icc will handle gcc-style >> asm blocks as long as you give it the -fasm-blocks command line option. >> It would be awfully nice to get rid of the __INTEL_COMPILER special >> cases in s_lock.

  1   2   >