Re: [HACKERS] Column mis-spelling hints vs case folding

2015-04-07 Thread Greg Stark
On 7 Apr 2015 09:37, "Robert Haas" wrote: > > On Sun, Apr 5, 2015 at 12:16 PM, Tom Lane wrote: > > A newbie error that we see *constantly* is misunderstanding identifier > > case-folding rules. > > I think this hint might be a lot more useful if its comparison mechanism > > were case-insensitive

Re: [HACKERS] Precedence of NOT LIKE, NOT BETWEEN, etc

2015-04-08 Thread Greg Stark
On Tue, Feb 24, 2015 at 5:04 PM, Tom Lane wrote: > Also, it strikes me that we could significantly reduce, maybe even fully > eliminate, the funny behaviors around the existing base_yylex() > substitutions if we made them use the same idea, ie replace the leading > token with something special but

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-04-08 Thread Greg Stark
I'm not sure what the best way to handle the hand-off from patch contribution to reviewer/committer. If I start tweaking things then you send in a new version it's actually more work to resolve the conflicts. I think at this point it's easiest if I just take it from here. I'm puzzled about the cha

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2015-04-14 Thread Greg Stark
I've been meaning to write this since PGConf and now isn't a great time since I'm on my phone but I think it's time. The way the clock sweep algorithm is meant to be thought about is that it's an approximate lru. Each usage count corresponds to an ntile of the lru. So we don't know which buffer is

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2015-04-15 Thread Greg Stark
On Wed, Apr 15, 2015 at 5:26 AM, Robert Haas wrote: >> The way our cache works we promote when a buffer is accessed but we only >> demote when a buffer is flushed. We flush a lot less often than we touch >> buffers so it's not surprising that the cache ends up full of buffers that >> are all in th

Re: [HACKERS] PATCH: default_index_tablespace

2015-04-16 Thread Greg Stark
On 15 Apr 2015 19:12, "Tom Lane" wrote: > > I'm afraid this idea is a nonstarter, because it will break existing > applications, and in particular existing pg_dump output files, which > expect to be able to determine an index's tablespace by setting > "default_tablespace". (It is *not* adequate t

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Greg Stark
On 15 Apr 2015 15:43, "Simon Riggs" wrote: > > It all depends upon who is being selfish. Why is a user "selfish" for > not wanting to clean every single block they scan, when the people > that made the mess do nothing and go faster 10 minutes from now? > Randomly and massively penalising large SEL

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-22 Thread Greg Stark
On Mon, Apr 20, 2015 at 8:48 PM, Bruce Momjian wrote: > >> But if the entire table is very hot, I think that that is just another of way >> of saying that autovacuum is horribly misconfigured. I think the purpose of > > Well, we have to assume there are many misconfigured configurations --- > aut

Re: [HACKERS] Hokey wrong versions of libpq in apt.postgresql.org

2014-08-20 Thread Greg Stark
On Sat, Aug 9, 2014 at 2:23 AM, Tom Lane wrote: > If the Debian guidelines think that only SO major version need > be considered, they're wrong, at least for the way we've been treating > that. The Debian approach is that you should have precisely one installed copy of a library for each soname.

Re: [HACKERS] [patch] pg_copy - a command for reliable WAL archiving

2014-08-20 Thread Greg Stark
On Tue, Aug 19, 2014 at 10:42 PM, Alvaro Herrera wrote: > Is there a way to create a link to a file which only exists as an open > file descriptor? If there was, you could create a temp file, open an > fd, then delete the file. That would remove the issue with files being > leaked due to failur

Re: [HACKERS] [patch] pg_copy - a command for reliable WAL archiving

2014-08-20 Thread Greg Stark
c.f.: O_TMPFILE (since Linux 3.11) Create an unnamed temporary file. The pathname argument specifies a directory; an unnamed inode will be created in that directory's filesystem. Anything written to the resulting file will be lost wh

Re: [HACKERS] [patch] pg_copy - a command for reliable WAL archiving

2014-08-20 Thread Greg Stark
On Wed, Aug 20, 2014 at 2:27 PM, Alvaro Herrera wrote: > Hmm, the real trick here is linkat(... "/proc/self/foobar"), not the > O_TMPFILE: you can have an open file descriptor to an "invisible" file > simply by creating a normal file and unlinking it. I looked at linkat() > yesterday but the idea

Re: [HACKERS] WIP Patch for GROUPING SETS phase 1

2014-08-22 Thread Greg Stark
On Fri, Aug 22, 2014 at 7:02 PM, Tom Lane wrote: > So the proposal you are pushing is going > to result in seriously teeing off some fraction of our userbase; > and the argument why that would be acceptable seems to boil down to > "I think there are few enough of them that we don't have to care" >

Re: [HACKERS] WIP Patch for GROUPING SETS phase 1

2014-08-22 Thread Greg Stark
On Fri, Aug 22, 2014 at 10:37 PM, Stephen Frost wrote: > Agreed- and how many of those have *every extension available* loaded... Actually that was also in the talk.a few slides later. 0.7% -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-08-23 Thread Greg Stark
On Sat, Aug 23, 2014 at 9:38 PM, Tom Lane wrote: > > Ah. Okay, but then what's wrong with the original proposal of "use ceil() > instead of floor()"? Basically I think the idea of treating fractions > less than one differently from fractions greater than one is bogus; nobody > will ever find tha

Re: [HACKERS] Proposal for CSN based snapshots

2014-08-26 Thread Greg Stark
On Tue, Aug 26, 2014 at 11:45 AM, Heikki Linnakangas wrote: >> It appears that this patch weakens the idea of hint bits. Even if >> HEAP_XMIN_COMMITTED is set, it still needs to find out if it's in the >> snapshot. >> >> That's fast if the xid is less than snap->xmin, but otherwise it needs >> to

Re: [HACKERS] Proposal for CSN based snapshots

2014-08-26 Thread Greg Stark
On Tue, Aug 26, 2014 at 8:32 PM, Jeff Davis wrote: > We are mixing two kinds of data: user data and visibility information. > Each is changed under different circumstances and has different > characteristics, and I'm beginning to think they shouldn't be mixed at > all. > > What if we just devised

[HACKERS] Re: Why data of timestamptz does not store value of timezone passed to it?

2014-08-29 Thread Greg Stark
On Fri, Aug 29, 2014 at 4:03 PM, Kevin Grittner wrote: > It was actually rather disappointing to hear that we had a > conforming implementation and changed away from it circa the 7.2 > release; and even more disturbing to hear that decision is still > being defended on the grounds that there's no

[HACKERS] Re: Why data of timestamptz does not store value of timezone passed to it?

2014-08-29 Thread Greg Stark
On Fri, Aug 29, 2014 at 4:19 PM, David G Johnston wrote: > And forcing people to change their data types to migrate to PostgreSQL is > undesirable IF our type is usefully equivalent to others in the majority of > situations - though I don't know if that is actually the case. You know... I wonder

Re: [HACKERS] On partitioning

2014-08-29 Thread Greg Stark
On Fri, Aug 29, 2014 at 4:56 PM, Alvaro Herrera wrote: > For scan plans, we need to prepare Append lists which are used to scan > for tuples in a partitioned relation. We can setup fake constraint > expressions based on the partitioning expressions, which let the planner > discard unnecessary par

Re: [HACKERS] On partitioning

2014-09-01 Thread Greg Stark
On Sun, Aug 31, 2014 at 9:03 PM, Tom Lane wrote: > Aside from costing planning time, most likely that would forever prevent > us from pushing some types of intelligence about partitioning into the > executor. How would it affect this calculus if there were partitioned indexes which were created o

Re: [HACKERS] On partitioning

2014-09-01 Thread Greg Stark
On Mon, Sep 1, 2014 at 4:59 PM, Tom Lane wrote: > The "push into executor" idea I was alluding to is that we might invent > plan constructs like a ModifyTable node that applies to a whole > inheritance^H^H^Hpartitioning tree and leaves the tuple routing to be > done at runtime. You're not going t

Re: [HACKERS] psql \watch versus \timing

2014-09-03 Thread Greg Stark
On Wed, Sep 3, 2014 at 12:48 PM, Michael Paquier wrote: > OK, then as all the comments are basically addressed, here is an > updated patch correcting the comment problems mentioned by Heikki. I just tried this and found it doesn't cooperate well with AUTOCOMMIT = 'off' and ON_ERROR_ROLLBACK = 'on

Re: [HACKERS] pg_upgrade and epoch

2014-09-05 Thread Greg Stark
On Wed, Sep 3, 2014 at 3:59 AM, Bruce Momjian wrote: > I have developed the attached patch which causes pg_upgrade to preserve > the transaction epoch. I plan to apply this for PG 9.5. I would say this is a simple bug and should be back patched to 9.4 and 9.3. We're only going to continue to get

Re: [HACKERS] pg_upgrade and epoch

2014-09-09 Thread Greg Stark
On Tue, Sep 9, 2014 at 4:05 PM, Bruce Momjian wrote: >> > Yes, I did think about that, but it seems like a behavior change. >> > However, it is tempting to avoid future bug reports about this. >> >> When this came up in March, Tom and I agreed that this wasn't something >> we wanted to slip into 9

Re: [HACKERS] Collations and Replication; Next Steps

2014-09-17 Thread Greg Stark
On Tue, Sep 16, 2014 at 11:41 PM, Peter Geoghegan wrote: > The timezone case you highlight here seems quite distinct from what > Matthew is talking about, because in point of fact the on-disk > representation is merely *interpreted* with reference to the timezone > database. So, you could have an

Re: [HACKERS] Collations and Replication; Next Steps

2014-09-17 Thread Greg Stark
On Wed, Sep 17, 2014 at 3:47 PM, Tatsuo Ishii wrote: > I don't think we cannot achieve that because even MySQL accomplishes:-) We've always considered it an advantage that we're consistent with the collations in the rest of the system. Generally speaking the fact that Postgres integrates with the

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Greg Stark
Fwiw I agree with TL2. The simplest, least surprising behaviour to explain to users is to say we round to nearest and if that means we rounded to zero (or another special value) we throw an error. We could list the minimum value in pg_settings and maybe in documentation. -- greg

Re: [HACKERS] Optimizer on sort aggregate

2014-10-18 Thread Greg Stark
On Sat, Oct 18, 2014 at 3:10 AM, Peter Geoghegan wrote: > So the first eight bytes of the first string is 0x131F1F1B221E, > and the second 0x131F1F1B220C. The last byte is different. That's interesting but I think it's mostly a quirk of your example. Afaics the difference is only that the

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-19 Thread Greg Stark
On Thu, Oct 16, 2014 at 8:00 PM, Peter Geoghegan wrote: > Basically, it's difficult to make this work for technical reasons > precisely because what I have here isn't join-like. Can I easily > disallow "OLD.*" in a RETURNING clause (recall that we only project > inserted tuples, as always)? Even i

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-10-20 Thread Greg Stark
On Mon, Oct 20, 2014 at 2:57 AM, Jim Nasby wrote: > Currently, a non-freeze vacuum will punt on any page it can't get a cleanup > lock on, with no retry. Presumably this should be a rare occurrence, but I > think it's bad that we just assume that and won't warn the user if something > bad is going

Re: [HACKERS] How ugly would this be? (ALTER DATABASE)

2014-10-25 Thread Greg Stark
On 24 Oct 2014 20:28, "Robert Haas" wrote: > > You could perhaps try to create a command that would move a schema > between two databases in the same cluster. It's fraught with > practical difficulties because a single backend can't be connected to > both databases at the same time, so how exactl

[HACKERS] Re: Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-29 Thread Greg Stark
On Wed, Oct 29, 2014 at 3:52 PM, Robert Haas wrote: > And it still doesn't protect against the case where you hardlink to a file > and then the permissions on that file are later changed. Fwiw that's not how hard links work, at least UFS semantics permissions such as ext2 etc. Hard links are link

Re: [HACKERS] CREATE INDEX CONCURRENTLY?

2014-10-31 Thread Greg Stark
On Fri, Oct 31, 2014 at 2:28 PM, Mark Woodward wrote: > I have not kept up with PostgreSQL changes and have just been using it. A > co-worker recently told me that you need to word "CONCURRENTLY" in "CREATE > INDEX" to avoid table locking. I called BS on this because to my knowledge > PostgreSQL d

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-03 Thread Greg Stark
On Sat, Nov 1, 2014 at 9:09 PM, Robert Haas wrote: > 1. Any non-trivial piece of PostgreSQL code is likely to contain > syscache lookups. > 2. Syscache lookups had better work in parallel workers, or they'll be > all but useless. I've been using parallel sorts and index builds in my mental model

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-05 Thread Greg Stark
On Thu, Nov 6, 2014 at 12:32 AM, Josh Berkus wrote: > When the recovery_target_time is reached, switch to streaming > replication and stay a standby. Then shouldn't he just not specify a recovert_target at all? That's the default behaviour for standby_mode on, the whole point of recovery_target i

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-06 Thread Greg Stark
On Thu, Nov 6, 2014 at 9:30 PM, Andres Freund wrote: > I think the retry logical is a largely pointless complication of already > complex enough code. You're fixing a problem for which there is > absolutely no evidence of its existance. Yes, this happens > occasionally. But it's going to be so abs

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Greg Stark
On Sun, Nov 9, 2014 at 9:18 AM, Heikki Linnakangas wrote: > Speaking of which, Alvaro, any chance we could get such on opclass still > included into 9.5? It would be nice to have one, just to be sure that > nothing minmax-specific has crept into the BRIN code. I'm trying to do a bloom filter Brin

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Greg Stark
On Sun, Nov 9, 2014 at 5:06 PM, Greg Stark wrote: > I'm trying to do a bloom filter Brin index. I'm already a bit puzzled > by a few things but I've just started so maybe it'll become clear. So some quick comments from pretty early goings -- partly because I'm a

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-09 Thread Greg Stark
On Sun, Nov 9, 2014 at 5:57 PM, Greg Stark wrote: > 2) The mention about additional opclass operators and to number them > from 11 up is fine -- but there's no explanation of how to decide what > operators need to be explicitly added like that. Specifically I gather > from re

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Greg Stark
On Mon, Nov 10, 2014 at 9:31 PM, Alvaro Herrera wrote: > Every time the index is accessed, yeah. I'm not sure about figuring the > initial creation details. Do you think we need another support > procedure to help with that? We can add it if needed; minmax would just > define it to InvalidOid.

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
On Tue, Nov 11, 2014 at 2:14 AM, Alvaro Herrera wrote: > I'm not sure why you say opaque is not associated with the specific > scan. Are you thinking we could reuse opaque for a future scan? I > think we could consider that opaque *is* the place to cache things such > as the hashed value of the

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
On Tue, Nov 11, 2014 at 12:12 PM, Alvaro Herrera wrote: > As far as I understand, the scan keys don't change within any given > scan; if they do, the rescan AM method is called, at which point we > should reset whatever is cached about the previous scan. But am I guaranteed that rescan will throw

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
It might be clearer to have an opclassinfo and a scaninfo which can store information in separate opc_opaque and scan_opaque fields with distinct liftetimes. In the bloom filter case the longlived info is the (initial?) size of the bloom filter and the number of hash functions. But I still haven't

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-11 Thread Greg Stark
On Tue, Nov 11, 2014 at 1:04 PM, Alvaro Herrera wrote: >> It might be clearer ... > > Patches welcome :-) Or perhaps there could still be a single opaque field but have two optional opclass methods "scaninit" and "rescan" which allow the op class to set or reset whichever fields inside opaque tha

Re: [HACKERS] pg_multixact not getting truncated

2014-11-21 Thread Greg Stark
On Thu, Nov 20, 2014 at 9:21 PM, Josh Berkus wrote: > Well, the first thing that comes to mind is that template0 should be > "permanently frozen". That is, all objects in it should be created with > frozen xid and mxids. After all, nobody can modify anything in it. This is actually the way it use

Re: [HACKERS] [pgsql-packagers] Palle Girgensohn's ICU patch

2014-11-26 Thread Greg Stark
I find it hard to believe the original premise of this thread. We knew there were some problems with OSX and FreeBSD but surely they can't be completely broken? What happens if you run "ls" with your locale set to something like fr_FR.UTF8 ? Does Apple not sell Macs in countries other than the US?

Re: [HACKERS] LLVM miscompiles numeric.c access to short numeric var headers

2015-11-18 Thread Greg Stark
Fwiw it looks like the LLVM folk think this is an asan bug. https://llvm.org/bugs/show_bug.cgi?id=25550 It looks like the fix is that the compiler should avoid this optimization if the code is being compiled with instrumentation. This worries me a bit but I think our code is safe as the Datum wil

Re: [HACKERS] Using quicksort for every external sort run

2015-11-18 Thread Greg Stark
On Wed, Nov 18, 2015 at 11:29 PM, Peter Geoghegan wrote: > Other systems expose this explicitly, and, as I said, say in an > unqualified way that a multi-pass merge should be avoided. Maybe the > warning isn't the right way of communicating that message to the DBA > in detail, but I am confident t

Re: [HACKERS] Using quicksort for every external sort run

2015-11-19 Thread Greg Stark
On Thu, Nov 19, 2015 at 6:56 PM, Peter Geoghegan wrote: > Yes, I really do mean it when I say that the DBA is not supposed to > see this message, no matter how much or how little memory or data is > involved. There is no nuance intended here; it isn't sensible to allow > a multi-pass sort, just as

Re: [HACKERS] Using quicksort for every external sort run

2015-11-19 Thread Greg Stark
On Thu, Nov 19, 2015 at 8:35 PM, Greg Stark wrote: > Hm. So a bit of back-of-envelope calculation. If we have want to > buffer at least 1MB for each run -- I think we currently do more > actually -- and say that a 1GB work_mem ought to be enough to run > reasonably (that's per s

Re: [HACKERS] Using quicksort for every external sort run

2015-11-19 Thread Greg Stark
On Fri, Nov 20, 2015 at 12:54 AM, Peter Geoghegan wrote: > * One run can be created with replacement selection, where a > hyrbid-sort merge strategy needs to create and then merge many runs. > When I started work on this patch, I was pretty sure that case would > be noticeably regressed. I was wro

Re: [HACKERS] onlyvalue aggregate (was: First Aggregate Funtion?)

2015-11-23 Thread Greg Stark
On Wed, Oct 28, 2015 at 5:03 PM, Marko Tiikkaja wrote: > SELECT a, sum(amount), onlyvalue(rolling_count) > FROM > ( > SELECT a, amount, count(*) OVER (ORDER BY a) AS rolling_count > FROM tbl > ) ss > GROUP BY a; The same thing would happen even in the more common case of having functiona

[HACKERS] Re: Revisiting pg_stat_statements and IN() (Was: Re: pg_stat_statements fingerprinting logic and ArrayExpr)

2015-11-24 Thread Greg Stark
On Tue, Nov 24, 2015 at 7:53 AM, Peter Geoghegan wrote: > * How do other people feel about this? Personally, I've seen enough > problems of this kind in the field that "slippery slope" arguments > against this don't seem very compelling. I also always felt there should be some kind of ??? symbol

Re: [HACKERS] Using quicksort for every external sort run

2015-11-24 Thread Greg Stark
On Wed, Nov 25, 2015 at 12:33 AM, Peter Geoghegan wrote: > On Tue, Nov 24, 2015 at 3:32 PM, Simon Riggs wrote: >> My feeling is that numbers rarely speak for themselves, without LSD. (Which >> numbers?) > > Guffaw. Actually I kind of agree. What I would like to see is a series of numbers for inc

Re: [HACKERS] New email address

2015-11-24 Thread Greg Stark
On Tue, Nov 24, 2015 at 10:03 PM, José Luis Tallón wrote: >> From: Persons Real Name >> Reply-To: ... >> so that at least the person's name would still be readable in MUA >> displays. > > Yup It'll still mess up everyone's contact book which will fill up with these fake email add

Re: [HACKERS] Using quicksort for every external sort run

2015-11-25 Thread Greg Stark
On Wed, Nov 25, 2015 at 2:31 AM, Peter Geoghegan wrote: > > There already was a test case involving a 1TB/16 billion tuple sort > [1] (well, a 1TB gensort Postgres table [2]). Granted, I don't have a > large number of similar test cases across a variety of scales, but > there are only so many hour

Re: [HACKERS] New email address

2015-11-25 Thread Greg Stark
On Wed, Nov 25, 2015 at 2:16 AM, Tom Lane wrote: > > I think that would put us in a situation where DKIM signatures would still > pass, at least unless the source insisted on signing Sender: too. Incidentally I'm confused about your concern about Sender. Sender has almost no significance for emai

Re: [HACKERS] New email address

2015-11-25 Thread Greg Stark
It's a reasonable idea for mailing list software to put the list in Sender given that it's the "agent responsible for the actual transmission of the message" as RFC2822 specifies. But you could just as easily argue that the list is just relaying the message and the agent actually transmitting the m

Re: [HACKERS] New email address

2015-11-26 Thread Greg Stark
On Wed, Nov 25, 2015 at 6:55 PM, Tom Lane wrote: > > But my point was that while the RFC says what to put there there's > > absolutely no reference anywhere for when the information should cause > > any MUA or MTA to behave differently. > > Agreed. To my mind that's a reason why Sender should not

Re: [HACKERS] New email address

2015-11-26 Thread Greg Stark
On Tue, Nov 24, 2015 at 7:50 PM, Tom Lane wrote: > Yeah, but SPF is also used as part of DMARC, which means that merely > forwarding somebody's email out of our listserv is probably going to look > like spam, even if we didn't change anything at all about the message > contents. Also, some source

Re: [HACKERS] New email address

2015-11-26 Thread Greg Stark
On Thu, Nov 26, 2015 at 11:13 PM, José Luis Tallón wrote: > From DMARC.org's Wiki: > <<< 2 Add an "Original Authentication Results" header to indicate you have > performed the authentication and you are validating it > 3 Take ownership of the email, by removing the DKIM signature and putting > you

Re: [HACKERS] New email address

2015-11-27 Thread Greg Stark
On Thu, Nov 26, 2015 at 11:26 PM, Alvaro Herrera wrote: > I don't think that's going to be anything but unwelcome noise. What > would they do if they became aware of the issue? They could switch > providers, but that only works for so long. As soon as Gmail switches > to p=reject, we've lost.

Re: [HACKERS] New email address

2015-11-27 Thread Greg Stark
On Fri, Nov 27, 2015 at 6:43 AM, Stefan Kaltenbrunner wrote: > > well not changing the subject seems like something we could do without > fuss - not changing the body would likely mean we would (again) get a > number of people asking "how do I unsubscribe", but maybe we will have > to live with th

Re: [HACKERS] New email address

2015-11-27 Thread Greg Stark
On Fri, Nov 27, 2015 at 10:02 AM, Magnus Hagander wrote: > Do you have any examples of lists where it *does* work? LIke, could it be > because our list-unsubscribe links are mailto: links and not http(s) links > for example? >From what I read they prefer mailto links. But apparently they only dis

Re: [HACKERS] silent data loss with ext4 / all current versions

2015-11-27 Thread Greg Stark
On Fri, Nov 27, 2015 at 11:17 AM, Tomas Vondra wrote: > I plan to do more power failure testing soon, with more complex test > scenarios. I suspect there might be other similar issues (e.g. when we > rename a file before a checkpoint and don't fsync the directory - then the > rename won't be repla

Re: [HACKERS] parallel joins, and better parallel explain

2015-11-30 Thread Greg Stark
On Mon, Nov 30, 2015 at 4:52 PM, Robert Haas wrote: > Not only does this build only one copy of the hash table instead of N > copies, but we can parallelize the hash table construction itself by > having all workers insert in parallel, which is pretty cool. Hm. The case where you don't want paral

Re: [HACKERS] Using quicksort for every external sort run

2015-11-30 Thread Greg Stark
Hm. Here is a log-log chart of those results (sorry for html mail). I'm not really sure if log-log is the right tool to use for a O(nlog(n)) curve though. I think the take-away is that this is outside the domain where any interesting break points occur. Maybe run more tests on the low end to find

Re: [HACKERS] [RFC] overflow checks optimized away

2015-12-01 Thread Greg Stark
On Fri, Oct 9, 2015 at 2:52 PM, Bruce Momjian wrote: > Any news on this item from 2013, worked on again 2014? > Sorry, I didn't look at it since. At the time I was using Xi Wang's software to find the overflow checks that need to be redone. He published a paper on it and it's actually pretty imp

Re: [HACKERS] pgsql: Further tweaking of print_aligned_vertical().

2015-12-01 Thread Greg Stark
On 1 Dec 2015 19:48, "Tom Lane" wrote: > > In passing, avoid possible calculation of log10(0). Probably that's > harmless, given the lack of field complaints, but it seems risky: > conversion of NaN to an integer isn't well defined. Am I going to have to fire up the emulator again?

Re: [HACKERS] [RFC] overflow checks optimized away

2015-12-03 Thread Greg Stark
On Tue, Dec 1, 2015 at 5:17 PM, Greg Stark wrote: > Sorry, I didn't look at it since. At the time I was using Xi Wang's software > to find the overflow checks that need to be redone. He published a paper on > it and it's actually pretty impressive. It constructs a const

Re: [HACKERS] [RFC] overflow checks optimized away

2015-12-03 Thread Greg Stark
On Thu, Dec 3, 2015 at 2:51 PM, Tom Lane wrote: > Greg Stark writes: >> What version of GCC and other compilers did we decide we're targeting now? > > I can't see us moving the compiler goalposts one inch for this. That's not the question I asked :/ -- gre

Re: [HACKERS] [RFC] overflow checks optimized away

2015-12-03 Thread Greg Stark
On Thu, Dec 3, 2015 at 2:51 PM, Tom Lane wrote: > I can't see us moving the compiler goalposts one inch for this. > "I'm going to break building on your compiler in order to work around > bugs in somebody else's compiler" isn't gonna fly. Fwiw the builtins offer a carrot as well. They promise to

Re: [HACKERS] atomic reads & writes (with no barriers)

2015-12-04 Thread Greg Stark
On Thu, Dec 3, 2015 at 10:10 PM, Kevin Grittner wrote: > Is the c.h change above on anything resembling the right track for > a patch for this? If not, what would such a patch look like? It would be nicer if we could come up with an interface that didn't require #ifdefs everywhere it's used. So

Re: [HACKERS] Size of Path nodes

2015-12-05 Thread Greg Stark
On Sat, Dec 5, 2015 at 5:35 PM, Tom Lane wrote: > The planner does not > normally[1] use the same path type to represent two fundamentally different > execution plans with enormously different cost estimates Eh, this is a typical object modelling dilemma. There are lots of different path types a

Re: [HACKERS] Using quicksort for every external sort run

2015-12-07 Thread Greg Stark
​So incidentally I've been running some benchmarks myself. Mostly to understand the current scaling behaviour of sorting to better judge whether Peter's analysis of where the pain points are and why we should not worry about optimizing for the multiple merge pass case were on target. I haven't actu

Re: [HACKERS] Using quicksort for every external sort run

2015-12-08 Thread Greg Stark
On Wed, Dec 9, 2015 at 12:02 AM, Jeff Janes wrote: > > > Then in the next (final) merge, it is has to read in this huge > fragmented tape run emulation, generating a lot of random IO to read > it. This seems fairly plausible. Logtape.c is basically implementing a small filesystem and doesn't real

Re: [HACKERS] Using quicksort for every external sort run

2015-12-08 Thread Greg Stark
On 9 Dec 2015 02:44, "Peter Geoghegan" wrote: > > I guess you mean insertion sort. What's the theoretical justification > for the change? Er, right. Insertion sort. The sort networks I used here are optimal both in number of comparisons and depth. I suspect modern CPUs actually manage to do some

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-12-09 Thread Greg Stark
On Wed, Dec 9, 2015 at 2:27 PM, David Fetter wrote: > Agreed that the "whole new language" aspect seems like way too big a > hammer, given what it actually does. Which would be easier to update when things change? Which would be possible to automatically generate from gram.y? -- greg -- Sent

Re: [HACKERS] mdnblocks() sabotages error checking in _mdfd_getseg()

2015-12-10 Thread Greg Stark
On Thu, Dec 10, 2015 at 4:47 PM, Robert Haas wrote: > > It's not straightforward, but I don't think that's the reason. What > we could do is look at the call sites that use > RelationGetNumberOfBlocks() and change some of them to get the > information some other way instead. I believe get_relati

Re: [HACKERS] mdnblocks() sabotages error checking in _mdfd_getseg()

2015-12-10 Thread Greg Stark
On Thu, Dec 10, 2015 at 7:09 PM, Robert Haas wrote: >> I really don't like Robert's proposal of a metapage though. We've got too >> darn many forks per relation already. > > Oh, I wasn't thinking of adding a fork, just repurposing block 0 of > the main fork, as we do for some index types. Not to

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-12-11 Thread Greg Stark
On Fri, Dec 11, 2015 at 8:12 AM, Michael Paquier wrote: > Also, if > we prioritize a dynamically generated tab completion using gram.y, so > be it and let's reject both patches then... Fwiw I poked at the bison output to see if it would be possible to do. I think it's theoretically possible but

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-11 Thread Greg Stark
There may be other errors that would be surprising for Tom or Robert. What I did with the string argument fuzzer was printed counts for each sqlstate for the errors and watched for errors that only occurred occasionally or didn't make sense to me. Also, do you have any timeouts? Do you have any s

Re: [HACKERS] Using quicksort for every external sort run

2015-12-11 Thread Greg Stark
On Wed, Dec 9, 2015 at 2:44 AM, Peter Geoghegan wrote: > On Tue, Dec 8, 2015 at 6:39 PM, Greg Stark wrote: > > I guess you mean insertion sort. What's the theoretical justification > for the change? Well my thinking was that hard coding a series of comparisons would be faster

Re: [HACKERS] Using quicksort for every external sort run

2015-12-11 Thread Greg Stark
On Fri, Dec 11, 2015 at 10:41 PM, Greg Stark wrote: > > Interestingly it looks like we could raise the threshold to switching > to insertion sort. At least on my machine the insertion sort is faster > in real time as well as fewer comparisons up to 9 elements. It's > actu

Re: [HACKERS] Using quicksort for every external sort run

2015-12-12 Thread Greg Stark
On Sat, Dec 12, 2015 at 7:42 PM, Ants Aasma wrote: > As the open coding doesn't help with eliminating control flow > dependencies, so my idea is to encode the sort network comparison > order in an array and use that to drive a simple loop. The code size > would be pretty similar to insertion sort

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-12 Thread Greg Stark
On Sat, Dec 12, 2015 at 8:30 PM, Andreas Seltenreich wrote: > I currently set statement_timeout to 1s to avoid wasting time letting > postgres crunch numbers. Less than 0.5% of the queries run into this > timeout. I wonder if any of these timeouts would be interesting to look at. Some may just

Re: [HACKERS] Fixing warnings in back branches?

2015-12-14 Thread Greg Stark
On Mon, Dec 14, 2015 at 10:20 AM, Andres Freund wrote: > I personally am not bothered by a handful of spurious warnings in the > back branches, but at this point you're very unlikely to see a new > warning introduced into 9.1 while backpatching. These are new warnings older compilers didn't detec

Re: [HACKERS] Using quicksort for every external sort run

2015-12-14 Thread Greg Stark
I ran sorts with various parameters on my small NAS server. This is a fairly slow CPU and limited memory machine with lots of disk so I thought it would actually make a good test case for smaller servers. The following is the speedup (for values < 100%) or slowdown (values > 100%) for the first pat

[HACKERS] Re: --enable-depend by default (was Re: Patch: fix lock contention for HASHHDR.mutex)

2015-12-30 Thread Greg Stark
Alternately the buildfarm could be changed to retain earlier builds and use dependencies to reduce build times. This would have the benefit of detecting dependency failures. At the expense of a lot more noise, at least until we Orrin out whatever dependency failures are there now. Or we could add

Re: [HACKERS] pgsql: Further tweaking of print_aligned_vertical().

2016-01-04 Thread Greg Stark
On Mon, Jan 4, 2016 at 6:24 AM, Michael Paquier wrote: > Arg, actually it is not: > https://wiki.postgresql.org/wiki/PostgreSQL_Conference_Europe_Talks_2015 > Greg could you add it there? No, apparently that page is restricted to only some users. I used plenty of images I pulled off the net with

Re: [HACKERS] pgsql: Further tweaking of print_aligned_vertical().

2016-01-04 Thread Greg Stark
On Mon, Jan 4, 2016 at 6:18 PM, Tom Lane wrote: >> VAX (simh): >> https://docs.google.com/presentation/d/1PG-bMU4WiS1pjtBwRvH4cE-nN9y5gj8ZLCO7KMrlvYg/view > >> Fuzzer: >> https://docs.google.com/presentation/d/12Dd9Bhcugkdi2w0ye4T1fy9ccjconJEz9cNthdeyH7k/view > > Very cool, thanks! Incidentally,

Re: [HACKERS] pglogical_output - a general purpose logical decoding output plugin

2016-01-06 Thread Greg Stark
On Wed, Jan 6, 2016 at 5:17 PM, Peter Eisentraut wrote: > I can't imagine that there is a lot of interest in a replication tool > where you only get one side of it, no matter how well-designed or > general it is. Well I do have another purpose in mind myself so I do appreciate it being available

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-06 Thread Greg Stark
On Wed, Jan 6, 2016 at 11:42 PM, Jim Nasby wrote: > Right. Personally, I feel the TODO has pretty much outlived it's usefulness. > An issue tracker would make maintaining items like this a lot more > reasonable, but it certainly wouldn't be free. Eh, a bug tracker that tracks actual bugs would be

Re: [HACKERS] Proposal: BSD Authentication support

2016-01-07 Thread Greg Stark
This sounds like a sensible thing to me. I'm actually surprised, it sounds like something we would have already seen. Do some people just use PAM on OpenBSD? Are both supported? You should add the patch to https://commitfest.postgresql.org to ensure it doesn't slip through the cracks. It's too lat

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2016-01-08 Thread Greg Stark
On Thu, Jan 7, 2016 at 6:30 PM, Jeff Janes wrote: > I don't completely agree with that. I have often wanted to know when > a specific item was added to the TODO page, and/or its individual edit > history. Sure, there might be other things it would be better at. But my point is that it would have

Re: [HACKERS] Enabling Checksums

2012-12-18 Thread Greg Stark
>> There is no good way to make the poor soul who has no standby server >> happy here. You're just choosing between bad alternatives. The first >> block error is often just that--the first one, to be joined by others >> soon afterward. My experience at how drives fail says the second error >> is a

Re: [HACKERS] Parser Cruft in gram.y

2012-12-19 Thread Greg Stark
On Tue, Dec 18, 2012 at 10:44 PM, Robert Haas wrote: > Yeah, that's why I don't know how to make it work. It feels like this > is partly artifact of the tool, though. I mean, suppose we haven't > read anything yet. Then, the next token can't be an IDENT, so if we > see an unreserved keyword, we

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Greg Stark
On Thu, Dec 20, 2012 at 3:18 AM, Tom Lane wrote: > Greg Stark writes: >> But I'm not entirely convinced any of this is actually useful. Just >> becuase the transition table is large doesn't mean it's inefficient. > > That's a fair point. However, I

Re: [HACKERS] initdb and share/postgresql.conf.sample

2012-12-23 Thread Greg Stark
On Sun, Dec 23, 2012 at 11:11 PM, Jeff Janes wrote: > You could say that benchmarks should run long enough to average out such > changes I would say any benchmark needs to be run long enough to reach a steady state before the measurements are taken. The usual practice is to run a series groups an

<    1   2   3   4   5   6   7   8   9   10   >