Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-17 Thread Jim Nasby
On 2/15/17 12:05 PM, Alvaro Herrera wrote: Jim Nasby wrote: ISTR previous discussion of allowing more stats files; if that happened I think having stats that were dedicated to (auto)vacuum would be very useful. That's clearly a lot more work though. What? There's a bunch of information

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-17 Thread Jim Nasby
0.02 sys decibel@decina:[12:41]~$ -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-15 Thread Jim Nasby
Accidentally sent o--list. On 2/14/17 12:57 PM, Jim Nasby wrote: On 2/13/17 12:36 AM, Masahiko Sawada wrote: It seems sensible to me to do something like this. We already report a lot of other fine details, so what's one more? And it could be useful. Also, I've been proposing to report

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-02-14 Thread Jim Nasby
if I can find any holes. Are you only worried about the removal of "useless" rels or is there more? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE

Re: [HACKERS] Official adoption of PGXN

2017-02-14 Thread Jim Nasby
I certainly don't think that's mandatory for having officially blessed "core extensions". -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Jim Nasby
sed methodology just gets in my way I'd likely use some other harness to actually run the tests). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2

Re: [HACKERS] Official adoption of PGXN

2017-02-14 Thread Jim Nasby
On 2/14/17 2:05 PM, Tom Lane wrote: Jim Nasby <jim.na...@bluetreble.com> writes: First, just to clarify: my reasons for proposing "core adoption" of PGXN are not technical in nature. What do you think "core adoption" means? Surely not that anything associated with

[HACKERS] Official adoption of PGXN (was: removing tsearch2)

2017-02-14 Thread Jim Nasby
/14/17 8:24 AM, Robert Haas wrote: On Tue, Feb 14, 2017 at 8:37 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: Right; I think we need at least some amount of pgxn buildfarm coverage. There probably also needs to be a way to officially bless certain distributions. Unless there's a pre

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-14 Thread Jim Nasby
't readily think of another use for flags in GetOldestXmin, but ISTM it's better to just go with "flags" instead of "ignoreFlags". -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Tre

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Jim Nasby
, and pg_regress's idea of what a test failure is just gets in my way. But breaking that out is far more invasive than allowing a template database. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http

Re: [HACKERS] log_autovacuum_min_duration doesn't log VACUUMs

2017-02-14 Thread Jim Nasby
uot;FWIW" but obviously that wasn't explicit enough. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-14 Thread Jim Nasby
. Presumably we already know how to parse the contents, so I'd think that shouldn't be that difficult. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532

[HACKERS] Add checklist item for psql completion to commitfest review

2017-02-14 Thread Jim Nasby
that checks to see if the psql completion code has been touched if any of the grammar has been. That could certainly trigger false positives so it'd need to be easy to over-ride, but AFAIK that could be done via a special phrase in the commit message. -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] Possible TODO: allow arbitrary expressions in event trigger WHEN

2017-02-14 Thread Jim Nasby
On 2/13/17 9:34 PM, Tom Lane wrote: Jim Nasby <jim.na...@bluetreble.com> writes: Is there a reason not to allow $SUBJECT? Specifically, it'd be nice to be able to do something like WHEN tag LIKE 'ALTER%'. Seems like it would be a seriously bad idea for such an expression to be able to

Re: [HACKERS] libpq Alternate Row Processor

2017-02-14 Thread Jim Nasby
, or whether palloc is so much faster than malloc that it's not an issue. I suspect that some of the effects are being masked by other things since presumably palloc and memcpy are pretty cheap on small volumes of data... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts

Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-14 Thread Jim Nasby
alculating sum, I expect to receive more or less precise result. Certainly I realize that even in case of using double it is ... but now you want to trade performance for accuracy? Why would you expect the database to magically come to that conclusion? -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] removing tsearch2

2017-02-14 Thread Jim Nasby
buildfarm). Right; I think we need at least some amount of pgxn buildfarm coverage. There probably also needs to be a way to officially bless certain distributions. Unless there's a pretty significant need for an official extension to be in contrib, it should go into PGXN. -- Jim Nasby, Data

[HACKERS] Possible TODO: allow arbitrary expressions in event trigger WHEN

2017-02-13 Thread Jim Nasby
Is there a reason not to allow $SUBJECT? Specifically, it'd be nice to be able to do something like WHEN tag LIKE 'ALTER%'. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http

Re: [HACKERS] Commit fest 2017-01 will begin soon!

2017-02-12 Thread Jim Nasby
ications to relevant threads as well. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-12 Thread Jim Nasby
On 2/3/17 7:34 PM, Andres Freund wrote: On 2017-02-03 19:26:55 -0600, Jim Nasby wrote: On 2/3/17 6:20 PM, Andres Freund wrote: - The ringbuffers in shared buffers can be problematic. One possible way of solving that is to get rid of ringbuffers entirely and rely on different initial values

Re: [HACKERS] 'text' instead of 'unknown' in Postgres 10

2017-02-12 Thread Jim Nasby
an empty array isn't treated the same as NULL, which means you need some way to create an empty array. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873

Re: [HACKERS] drop support for Python 2.3

2017-02-12 Thread Jim Nasby
(PY_MINOR_VERSION << 16) | \ (PY_MICRO_VERSION << 8) | \ (PY_RELEASE_LEVEL << 4) | \ (PY_RELEASE_SERIAL << 0)) -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Dat

Re: [HACKERS] Removal of deprecated views pg_user, pg_group, pg_shadow

2017-02-12 Thread Jim Nasby
cate. My vote would be to either kill the views or explicitly deprecate them and move them to contrib. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (85

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-12 Thread Jim Nasby
red memory already. If there was a function that reported checkpoint status pg_basebackup could poll that to provide users with live status. That should be a separate patch though. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and Postg

Re: [HACKERS] log_autovacuum_min_duration doesn't log VACUUMs

2017-02-12 Thread Jim Nasby
becomes completely useless. I think the ability to filter logging based on context would be very valuable. AFAIK you could actually do that for manual logging with existing plpgsql support, but obviously that won't help for anything else. -- Jim Nasby, Data Architect, Blue Treble Consulting

Re: [HACKERS] removing tsearch2

2017-02-12 Thread Jim Nasby
run extension/package/module framework has proven to be extremely valuable for other programming environments, and IMHO we should be striving to improve in that area. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-12 Thread Jim Nasby
, with an N-way associative cache 2x more padding than necessary cuts the amount you can fit into the cache by half. That could be meaningful in some cases. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get

Re: [HACKERS] Checksums by default?

2017-02-12 Thread Jim Nasby
that next time this happens they save the bad WAL. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Adding the optional clause 'AS' in CREATE TRIGGER

2017-02-12 Thread Jim Nasby
you to specify an arbitrary expression to test against NEW and OLD. 1: Second, WIP example: https://github.com/decibel/tg_sanity/blob/master/sql/tg_sanity.sql -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-02-12 Thread Jim Nasby
ant ID values, but that's even more involved. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-02-09 Thread Jim Nasby
to change their tools. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-02-09 Thread Jim Nasby
what people gravitate to, but I suspect most users would be happier with a construct that worked like the array trick does, but was easier to accomplish. I wonder if any other databases have come up with non-standard syntax to do this. -- Jim Nasby, Data Architect, Blue Treble Consulting

Re: [HACKERS] libpq Alternate Row Processor

2017-02-09 Thread Jim Nasby
be interesting to see how things change if you eliminate the filler column from the SELECT. Also, the backend should be buffering ~8kb of data before handing that to the socket. If that's more than the kernel can buffer I'd expect a serious performance hit. -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] Press Release Draft - 2016-02-09 Cumulative Update

2017-02-08 Thread Jim Nasby
ion of git references in release notes, but I'd find it useful to be able to at least get a complete list. Not hard for me to do that since I know git and our naming scheme, but maybe we should include directions for doing so? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in

Re: [HACKERS] Press Release Draft - 2016-02-09 Cumulative Update

2017-02-08 Thread Jim Nasby
want to without much extra effort. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers

[HACKERS] Improve OR conditions on joined columns.

2017-02-08 Thread Jim Nasby
attached example); I believe that would work regardless of uniqueness. Just to be clear; the OR by itself is not a problem (as shown by the first fast query); it's the OR with the JOIN that's a problem. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data

Re: [HACKERS] WIP: About CMake v2

2017-02-08 Thread Jim Nasby
ither edit config.cache and delete the relevant lines or just temporarily move it out of the way (or just nuke it...). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-03 Thread Jim Nasby
triggered backend writes and how long they took. Good idea. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-02-03 Thread Jim Nasby
On 2/3/17 6:39 PM, Andres Freund wrote: Hi, On 2017-02-03 18:32:03 -0600, Jim Nasby wrote: Commit 48354581a49c30f5757c203415aa8412d85b0f70 (Allow Pin/UnpinBuffer to operate in a lockfree manner) removed the code in PinBuffer that conditionally incremented usage_count when a ring buffer

Re: [HACKERS] Checksums by default?

2017-02-03 Thread Jim Nasby
forks you might have. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] libpq Alternate Row Processor

2017-02-03 Thread Jim Nasby
submission. I just did essentially the same thing for SPI (use a callback to allow the caller to handle the tuple instead of shoving it into a tuplestore). A simple test in plpython showed a 460% improvement. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-03 Thread Jim Nasby
On 2/2/17 4:39 PM, Corey Huinker wrote: On Wed, Feb 1, 2017 at 4:58 PM, Jim Nasby <jim.na...@bluetreble.com <mailto:jim.na...@bluetreble.com>> wrote: I think the issue here is that the original case for this is a user accidentally getting into an \if and then having no

[HACKERS] PinBuffer() no longer makes use of strategy

2017-02-03 Thread Jim Nasby
Commit 48354581a49c30f5757c203415aa8412d85b0f70 (Allow Pin/UnpinBuffer to operate in a lockfree manner) removed the code in PinBuffer that conditionally incremented usage_count when a ring buffer was in use. Was that intentional? ISTM the old behavior should have been retained. -- Jim Nasby

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-03 Thread Jim Nasby
than the cluster, yet there's a non-trivial amount of buffers being written by backends. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532

Re: [HACKERS] Checksums by default?

2017-02-03 Thread Jim Nasby
trivial to write an extension that did that though? foreach r in pg_class where relkind in (...) for (b = 0; b < r.relpages; b++) ReadBufferExtended(..., BAS_BULKREAD); -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL D

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-02 Thread Jim Nasby
On 2/1/17 4:28 PM, Andres Freund wrote: On 2016-11-28 11:40:53 -0800, Jim Nasby wrote: With current limits, the most bgwriter can do (with 8k pages) is 1000 pages * 100 times/sec = 780MB/s. It's not hard to exceed that with modern hardware. Should we increase the limit on bgwriter_lru_maxpages

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-01 Thread Jim Nasby
On 2/1/17 4:27 PM, Andres Freund wrote: On 2017-02-02 09:22:46 +0900, Michael Paquier wrote: On Thu, Feb 2, 2017 at 9:17 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: Speaking of which... I have a meeting in 15 minutes to discuss moving to a server with 4TB of memory. With current

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-01 Thread Jim Nasby
On 2/1/17 3:36 PM, Michael Paquier wrote: On Thu, Feb 2, 2017 at 7:01 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: On 2/1/17 10:27 AM, Robert Haas wrote: This looks fine to me. This could go without the comments, they are likely going to be forgotten if any updates happen in the

Re: [HACKERS] Cast jsonb to numeric, int, float, bool

2017-02-01 Thread Jim Nasby
sure what your intent here is, but if the idea is that a json array would magically cast to a bool or a number data type I think that's a bad idea. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Tr

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-02-01 Thread Jim Nasby
On 2/1/17 10:27 AM, Robert Haas wrote: On Tue, Jan 31, 2017 at 5:07 PM, Jim Nasby <jim.na...@bluetreble.com> wrote: On 11/29/16 9:58 AM, Jeff Janes wrote: Considering a single SSD can do 70% of that limit, I would say yes. Next question becomes... should there even be an

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-01 Thread Jim Nasby
to what happens when you miss a quote or a semicolon. We handle those cases with %R, and I think %R needs to support if as well. Perhaps there's value to providing more info (active branch, etc), but ISTM trying to do that will just confuse the original (%R) case. -- Jim Nasby, Data Architect

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-02-01 Thread Jim Nasby
On 1/27/17 4:14 AM, Greg Stark wrote: On 25 January 2017 at 20:06, Jim Nasby <jim.na...@bluetreble.com> wrote: GUCs support SET LOCAL, but that's not the same as local scoping because the setting stays in effect unless the substrans aborts. What I'd like is the ability to set a GUC in a p

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-01-31 Thread Jim Nasby
to INT_MAX / 2, which is the same as shared_buffers. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) commit

Re: [HACKERS] Superowners

2017-01-29 Thread Jim Nasby
: they wanted to grant specific roles the ability to read everything in the database (or maybe cluster; I don't think the conversation got into that level of detail). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-29 Thread Jim Nasby
Execution time: 48.917 ms (5 rows) In any case, +1 for not promoting count(*) <> 0; that's a really, really bad way to test for existence. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treb

Re: [HACKERS] GSoC 2017

2017-01-27 Thread Jim Nasby
having a timestamp type that stores the original timezone. BTW, time itself fits in the multi-unit pattern, since months don't have a fixed conversion to days (and technically seconds don't have a fixed conversion to anything thanks to leap seconds). -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-27 Thread Jim Nasby
on the issues that have already been identified before trying to add more features. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-01-25 Thread Jim Nasby
in a plpgsql block *and have the setting revert on block exit*. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-25 Thread Jim Nasby
On 1/25/17 1:46 PM, Jim Nasby wrote: Based on that and other feedback I'm going to mark this as returned with feedback, though if you're able to get a revised patch in the next few days please do. Actually, based on the message that popped up when I went to do that I guess it's better

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-25 Thread Jim Nasby
On 1/24/17 11:13 PM, Beena Emerson wrote: On Wed, Jan 25, 2017 at 10:36 AM, Jim Nasby <jim.na...@bluetreble.com <mailto:jim.na...@bluetreble.com>> wrote: On 1/24/17 2:26 AM, Mithun Cy wrote: Thanks for looking into this patch, I just downloaded the patch and

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-24 Thread Jim Nasby
25.012 IST [91755] LOG: Buffer Dump: saved metadata of 59 blocks. Yeah, I wasn't getting that at all, though I did see the shared library being loaded. If I get a chance I'll try it again. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-24 Thread Jim Nasby
terminology other than "buffer dump", because that makes it sound like we're dumping the contents of the buffers themselves. Maybe block_map? Buffer_map? -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Tr

Re: [HACKERS] Checksums by default?

2017-01-24 Thread Jim Nasby
not completely grokking your second paragraph, but I would think that an average user would love got get a heads-up that their hardware is failing. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-01-24 Thread Jim Nasby
I'll post a plpython patch that doesn't add the output format control. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hacke

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-01-23 Thread Jim Nasby
On 1/5/17 9:50 PM, Jim Nasby wrote: The * on that is there's something odd going on where plpython starts out really fast at this, then gets 100% slower. I've reached out to some python folks about that. Even so, the overall results from a quick test on my laptop are (IMHO) impressive

Re: [HACKERS] Online enabling of page level checksums

2017-01-23 Thread Jim Nasby
technical standpoint* to enable by default in 10, with only the ability to dynamically disable. Given the concerns that keep popping up about dynamically enabling, I'm not at all sure that we could get that into 10. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics,

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
On 1/23/17 8:24 PM, Tom Lane wrote: Jim Nasby <jim.na...@bluetreble.com> writes: On 1/23/17 7:47 PM, Stephen Frost wrote: It might be interesting to consider checking them in 'clean' pages in shared_buffers in a background process, as that, presumably, *would* detect shared buffers corr

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
it presumably wouldn't be difficult to do that in an extension, as a bgworker. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
can we go about collecting data that checksums help? We certainly know people suffer data corruption. We can only guess at how many of those incidents would be caught by checksums. I don't see how we can get data on that unless we get a lot more users running checksums. -- Jim Nasby, Data

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
On 1/23/17 6:55 PM, Stephen Frost wrote: * Jim Nasby (jim.na...@bluetreble.com) wrote: As others have mentioned, right now practically no one enables this, so we've got zero data on how useful it might actually be. Uhm, Peter G just said that Heroku enables this on all their databases and have

Re: [HACKERS] Online enabling of page level checksums

2017-01-23 Thread Jim Nasby
I think it's OK to force users that are enabling this to manually connect to datallowcon=false and run vacuum. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855

Re: [HACKERS] [PATCH] Generic type subscription

2017-01-23 Thread Jim Nasby
there's a static variable that gets set to the relevant OID it's not that bad performance-wise from what I can tell, but I suspect that's not something we want to be recommending to others... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
is already well out of the barn by then. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Checksums by default?

2017-01-23 Thread Jim Nasby
to try and significantly increase the rate of buffers being written out. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-h

Re: [HACKERS] GSoC 2017

2017-01-23 Thread Jim Nasby
the contents of TZDATA any time it changed, as well as a fast way to find the surrogate key for the current TZDATA. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855

Re: [HACKERS] Contrib: alternative MATERIALIZED VIEWs

2017-01-23 Thread Jim Nasby
working on. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-23 Thread Jim Nasby
I took a look at this again, and it doesn't appear to be working for me. The library is being loaded during startup, but I don't see any further activity in the log, and I don't see an autoprewarm file in $PGDATA. There needs to be some kind of documentation change as part of this patch. I'm

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-23 Thread Jim Nasby
On 1/21/17 6:42 PM, Jim Nasby wrote: On 12/26/16 2:31 AM, Kyotaro HORIGUCHI wrote: The points of discussion are the following, I think. 1. The first patch seems working well. It costs the time to scan the whole of a catcache that have negative entries for other reloids. However

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-01-23 Thread Jim Nasby
as an option to exec_*. finit_ would be better named free_. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-23 Thread Jim Nasby
of those concerns. Or perhaps the original complaint about STATRELATTINH could be solved by just disabling negative entries on that cache. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-01-22 Thread Jim Nasby
d be great if there was a more automated method. Such a method might also be very useful for transforming expressions like date_part('quarter', ...) into something that could use existing indexes. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Ar

Re: [HACKERS] Online enabling of page level checksums

2017-01-22 Thread Jim Nasby
'd be fine. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Jim Nasby
On 1/22/17 4:41 PM, Jim Nasby wrote: On 1/21/17 8:54 PM, Tom Lane wrote: Jim Nasby <jim.na...@bluetreble.com> writes: The other (possibly naive) question I have is how useful negative entries really are? Will Postgres regularly incur negative lookups, or will these only happen due t

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Jim Nasby
On 1/21/17 8:54 PM, Tom Lane wrote: Jim Nasby <jim.na...@bluetreble.com> writes: The other (possibly naive) question I have is how useful negative entries really are? Will Postgres regularly incur negative lookups, or will these only happen due to user activity? It varies dep

Re: [HACKERS] Too many autovacuum workers spawned during forced auto-vacuum

2017-01-22 Thread Jim Nasby
like the attached. On another note, does anyone else find the database selection logic rather difficult to trace through? The logic is kinda spread throughout several functions. The naming of rebuild_database_list() and get_database_list() is rather confusing too. -- Jim Nasby, Data Architect

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-21 Thread Jim Nasby
than most of the other ideas. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Checksums by default?

2017-01-21 Thread Jim Nasby
ntial runs tonight. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: Updating MATERIALIZED VIEWs (Re: [HACKERS] delta relations in AFTER triggers)

2017-01-21 Thread Jim Nasby
for the extension to track arbitrary database objects that were created. 1: https://github.com/decibel/pg_classy/blob/master/doc/pg_classy.asc -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble

Re: [HACKERS] GSoC 2017

2017-01-16 Thread Jim Nasby
in the towel and said a cast was impossible. 1: https://github.com/BlueTreble/variant/ -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via

Re: [HACKERS] GSoC 2017

2017-01-16 Thread Jim Nasby
On 1/13/17 4:08 PM, Alvaro Herrera wrote: Jim Nasby wrote: On 1/10/17 1:53 AM, Alexander Korotkov wrote: 1. What project ideas we have? Perhaps allowing SQL-only extensions without requiring filesystem files would be a good project. Don't we already have that in patch form? Dimitri

Re: [HACKERS] PSQL commands: \quit_if, \quit_unless

2017-01-16 Thread Jim Nasby
it... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Packages: Again

2017-01-16 Thread Jim Nasby
ark those namespaces (and possibly other objects) as private. - A way to reference extensions from other extensions and deal with extensions being moved to a different schema (or namespace). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architec

[HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-01-15 Thread Jim Nasby
it-in-2016-fad96ae22a15?imm_mid=0ec3e0=em-prog-na-na-newsltr_20170114#.shgj609ad -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgs

Re: [HACKERS] plpgsql - additional extra checks

2017-01-12 Thread Jim Nasby
. Shouldn't this also apply to var := blah FROM some_table WHERE ...; ? AIUI that's one of the beefs the plpgsql2 project has. FWIW, I'd also be in favor of a NOMULTI option to INTO, but I don't see any way to do something like that with var := blah FROM. -- Jim Nasby, Data Architect, Blue

Re: [HACKERS] GSoC 2017

2017-01-12 Thread Jim Nasby
On 1/10/17 1:53 AM, Alexander Korotkov wrote: 1. What project ideas we have? Perhaps allowing SQL-only extensions without requiring filesystem files would be a good project. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL

Re: [HACKERS] Off-by-one oddity in minval for decreasing sequences

2017-01-12 Thread Jim Nasby
hole" that you can end up with now. I agree it makes sense to sen the minimum value correctly. Not sure if this necessitates changes in pg_upgrade... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it

Re: [HACKERS] remove floats from bootstrap scanner/parser

2017-01-12 Thread Jim Nasby
rip it out, per the attached. Verified this works for make check. Looks sane to me. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent

Re: [HACKERS] WARM and indirect indexes

2017-01-12 Thread Jim Nasby
. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] merging some features from plpgsql2 project

2017-01-12 Thread Jim Nasby
compatibility with plsql, as well as pulling PSM into core. The former would be to help migrating from Oracle; the latter would be to provide everyone a cleaner built-in PL. (IMHO a PLSQL equivalent could certainly be an external extension). -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-12 Thread Jim Nasby
there. But that's certainly not the case. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com 855-TREBLE2 (855-873-2532) -- Sent via pgsql-hackers mailing list (pgsql-hackers

<    1   2   3   4   5   6   7   8   9   10   >