Re: [HACKERS] FSM corruption leading to errors

On Thu, Oct 20, 2016 at 2:50 PM, Pavan Deolasee wrote: > Actually, if we could add an API which can truncate FSM to the given heap > block, then the user may not even need to run VACUUM, which could be costly > for very large tables. FreeSpaceMapTruncateRel()? > Also,

Re: [HACKERS] FSM corruption leading to errors

On Thu, Oct 20, 2016 at 11:34 AM, Michael Paquier wrote: > On Thu, Oct 20, 2016 at 2:50 PM, Pavan Deolasee > wrote: > > Actually, if we could add an API which can truncate FSM to the given heap > > block, then the user may not even need to

Re: [HACKERS] Gather Merge

On Thu, Oct 20, 2016 at 12:22 AM, Peter Geoghegan wrote: > On Tue, Oct 4, 2016 at 11:05 PM, Rushabh Lathia > wrote: > > Query 4: With GM 7901.480 -> Without GM 9064.776 > > Query 5: With GM 53452.126 -> Without GM 55059.511 > > Query 9: With GM

Re: [HACKERS] Danger of automatic connection reset in psql

On Thu, Oct 20, 2016 at 12:28 PM, Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > > Since this is already an improvement, I'm attaching a patch. > > If on the other hand, someone is pasting into psql's terminal a block of > commands enclosed in BEGIN/COMMIT, the same bug is triggered:

Re: [HACKERS] Danger of automatic connection reset in psql

On Thu, Oct 20, 2016 at 12:28 PM, Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > > > I'm not the first one to discover that, a search in archives gives at least 3 results: > >

[HACKERS] Danger of automatic connection reset in psql

Hi Hackers! When using psql interactively one might be tempted to guard potentially destructive commands such as "UPDATE / DELETE / DROP " by starting the input line with an explicit "BEGIN; ...". This has the added benefit that then you invoke the command by reverse-searching the command

Re: [HACKERS] Gather Merge

On Tue, Oct 18, 2016 at 5:29 PM, Rushabh Lathia wrote: > On Mon, Oct 17, 2016 at 2:26 PM, Amit Kapila > wrote: >> >> There is lot of common code between ExecGatherMerge and ExecGather. >> Do you think it makes sense to have a common function to

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

> > >> In fact, I am not in >> favour of tracking the query dependencies for UPDATE/DELETE since we >> don't have any concrete example as to when that would be needed. > > > Right, but as I said before, some FDW might consult FDW options stored in > those objects during AddForeignUpdateTargets, so

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote: > I agree with these conclusions. I had a chance to talk with Andres > this morning at Postgres Vision and based on that conversation I'd > like to suggest a couple of additional tests: > > 1. Repeat this test on x86.

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

On Thu, Oct 20, 2016 at 12:49 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > The patch compiles and make check-world doesn't show any failures. > > >> > > > > > > I have tried it. Attached separate patch for it. > > However I have noticed that istoplevel is always false (at-least

[HACKERS] File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all')

> > > According to my colleagues it would be very nice to have this feature. > > > For instance, if you are trying to optimize PostgreSQL for application > > > that uses COPY and you don't have access to or something like this. > > > It could also be useful in some other cases. > > > > This

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

On 2016/10/20 16:27, Ashutosh Bapat wrote: I wrote: Besides that, I modified add_rte_to_flat_rtable so that the plan's dependencies are tracked, but that would lead to tracking the dependencies of unreferenced foreign tables in dead subqueries or the dependencies of foreign tables excluded from

Re: [HACKERS] Indirect indexes

On 10/20/2016 06:39 AM, Alvaro Herrera wrote: Bruce Momjian wrote: Also, it seems indirect indexes would be useful for indexing columns that are not updated frequently on tables that are updated frequently, and whose primary key is not updated frequently. That's quite a logic problem for

Re: [HACKERS] Remove vacuum_defer_cleanup_age

On Wed, Oct 19, 2016 at 9:09 PM, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote: >> On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian wrote: >> > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms >> > of

Re: [HACKERS] Disable autovacuum guc?

On 10/19/2016 07:22 PM, Josh Berkus wrote: On 10/19/2016 06:27 PM, Joshua D. Drake wrote: Hello, After all these years, we are still regularly running into people who say, "performance was bad so we disabled autovacuum". I am not talking about once in a while, it is often. I would like us to

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 10:39:23AM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Just to clarify, if a feature improves performance by 1%, but is enabled > > by default, that is 10x more useful across our entire user base as the > > feature numbers listed above, 1% vs 0.1%. > >

Re: [HACKERS] Remove autovacuum GUC?

On Wed, Oct 19, 2016 at 9:24 PM, Joshua D. Drake wrote: > After all these years, we are still regularly running into people who say, > "performance was bad so we disabled autovacuum". I am not talking about once > in a while, it is often. I would like us to consider

Re: [HACKERS] Indirect indexes

Joshua D. Drake wrote: > That said would it be possible to make this index an extension (like rum?). > Assuming of course we can get any required infrastructure changes done in a > general way. Well, the patch I currently have creates a separate index AM called "ibtree" which is an indirect

Re: [HACKERS] WIP: Fix invalid XML explain plans for track_io_timing

Markus Winand writes: > The XML output of explain potentially outputs the XML tag names > "I/O-Write-Time" > and "I/O-Read-Time", which are invalid due to the slash. Ooops. > Although the patch fixes the problem for the moment, it is incomplete in that > sense that it

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On 10/20/2016 09:12 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Robert Haas writes: That said, I'd also like to see a --force or similar option or mechanism put in place to reduce the risk of users trashing their system because they think

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 12:05 PM, Stephen Frost wrote: >> To be honest, I don't really like either pg_transaction or pg_xact. > >> Neither name captures the fact that what we're really tracking here is >> the transaction *status*. pg_xact is slightly worse because it's a >>

Re: [HACKERS] Remove autovacuum GUC?

On Thu, Oct 20, 2016 at 12:32 PM, Joshua D. Drake wrote: > That argument suggests we shouldn't have autovacuum :P It certainly does not. That, too, would be removing a useful option. In fact, it would be removing the most useful option that is the right choice for most

Re: [HACKERS] Indirect indexes

On 20/10/16 17:24, Bruce Momjian wrote: > On Thu, Oct 20, 2016 at 05:14:51PM +0200, Petr Jelinek wrote: >>> Also, it seems indirect indexes would be useful for indexing columns >>> that are not updated frequently on tables that are updated frequently, >>> and whose primary key is not updated

Re: [HACKERS] Remove autovacuum GUC?

On Thu, Oct 20, 2016 at 9:24 AM, Robert Haas wrote: > On Thu, Oct 20, 2016 at 11:53 AM, Joshua D. Drake > wrote: >> The right answer isn't the answer founded in the reality for many if not >> most of our users. > > I think that's high-handed

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Wed, Oct 19, 2016 at 7:05 AM, Christoph Berg wrote: > (tl;dr: rename pg_xlog yes, rename pg_resetxlog only if we have a good > alternative.) I'm amused by the idea of a TL;DR in parentheses at the very bottom of the email, but maybe I'm just easily amused. One idea would be

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 05:14:51PM +0200, Petr Jelinek wrote: > > Also, it seems indirect indexes would be useful for indexing columns > > that are not updated frequently on tables that are updated frequently, > > and whose primary key is not updated frequently. That's quite a logic > > problem

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Wed, Oct 12, 2016 at 10:22 PM, Michael Paquier wrote: > OK. I can live with that as well. Attached are three patches. The > pg_xlog -> pg_wal move, the pg_clog -> pg_transaction move, and the > pg_clog -> pg_xact move. Only one survivor to be chosen among the last >

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 8:44 PM, Petr Jelinek wrote: > > > WARM can do WARM update 50% of time, indirect index can do HOT update > 100% of time (provided the column is not changed), I don't see why we > could not have both solutions. > > I think the reason why I restricted

Re: [HACKERS] Renaming of pg_xlog and pg_clog

Robert Haas writes: > One idea would be to rename pg_resetxlog to pg_resetwal. I think > that's actually an improvement. This would fit in as part of a general plan to s/xlog/wal/g throughout our user-visible names and documentation. Which seems like a good idea to me;

Re: [HACKERS] Disable autovacuum guc?

On 10/20/2016 06:34 AM, Joshua D. Drake wrote: > On 10/19/2016 07:22 PM, Josh Berkus wrote: >> On 10/19/2016 06:27 PM, Joshua D. Drake wrote: >>> Hello, >>> >>> After all these years, we are still regularly running into people who >>> say, "performance was bad so we disabled autovacuum". I am not

Re: [HACKERS] LLVM Address Sanitizer (ASAN) and valgrind support

On Oct 20, 2016 5:27 PM, "Noah Misch" wrote: > > On Wed, Oct 19, 2016 at 11:08:39AM +0100, Greg Stark wrote: > > > The MEMPOOL_FREE doesn't take any size argument and mcxt.c doesn't > > have convenient access to a size argument. It could call the > > GetChunkSpace method but

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 12:14 PM, Petr Jelinek wrote: > WARM can do WARM update 50% of time, indirect index can do HOT update > 100% of time (provided the column is not changed), I don't see why we > could not have both solutions. > > That all being said, it would be

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 12:30 PM, Pavan Deolasee wrote: > > > On Thu, Oct 20, 2016 at 8:44 PM, Petr Jelinek wrote: >> >> >> >> WARM can do WARM update 50% of time, indirect index can do HOT update >> 100% of time (provided the column is not

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 9:20 PM, Claudio Freire wrote: > > > With indirect indexes, since you don't need to insert a tid, you can > just "insert on conflict do nothing" on the index. > Would that work with non-unique indexes? Anyways, the point I was trying to make is

Re: [HACKERS] Renaming of pg_xlog and pg_clog

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > One idea would be to rename pg_resetxlog to pg_resetwal. I think > > that's actually an improvement. > > This would fit in as part of a general plan to s/xlog/wal/g throughout > our user-visible names and

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 12:12 PM, Stephen Frost wrote: > That said, I'd also like to see a --force or similar option or mechanism > put in place to reduce the risk of users trashing their system because > they think pg_resetwal is "safe." ("It's just gonna reset things to make

Re: [HACKERS] Remove autovacuum GUC?

Hello, What about a simpler solution to all of this. Let's just remove it from postgresql.conf. Out of sight. If someone needs to test they can but a uneducated user won't immediately know what to do about that "autovacuum process" and when they look it up the documentation is exceedingly

Re: [HACKERS] Parallel Index Scans

On Wed, Oct 19, 2016 at 11:07 PM, Amit Kapila wrote: >> Ideally, the parallel_workers storage parameter will rarely be >> necessary because the optimizer will generally do the right thing in >> all case. > > Yeah, we can choose not to provide any parameter for parallel

Re: [HACKERS] File content logging during execution of COPY queries

Grigory, * Grigory Smolkin (g.smol...@postgrespro.ru) wrote: > On 10/20/2016 12:36 PM, Aleksander Alekseev wrote: > According to my colleagues it would be very nice to have this feature. > For instance, if you are trying to optimize PostgreSQL for application > that uses COPY and you

Re: [HACKERS] Indirect indexes

On 20/10/16 14:29, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 01:04:16PM -0400, Bruce Momjian wrote: >> On Wed, Oct 19, 2016 at 06:58:05PM +0200, Simon Riggs wrote: > I agree. Also, I think the recheck mechanism will have to be something > like > what I wrote for WARM i.e. only

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 10/20/2016 09:36 AM, Dilip Kumar wrote: On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote: I agree with these conclusions. I had a chance to talk with Andres this morning at Postgres Vision and based on that conversation I'd like to suggest a couple of additional

Re: [HACKERS] Remove autovacuum GUC?

On 10/20/2016 07:12 AM, Robert Haas wrote: On Wed, Oct 19, 2016 at 9:24 PM, Joshua D. Drake wrote: Setting autovacuum=off is at least useful for testing purposes and I've used it that way. On the other hand, I haven't seen a customer disable this unintentionally in

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Oct 20, 2016 at 3:36 AM, Dilip Kumar wrote: > On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote: >> I agree with these conclusions. I had a chance to talk with Andres >> this morning at Postgres Vision and based on that conversation I'd >>

Re: [HACKERS] Disable autovacuum guc?

On 10/20/2016 08:54 AM, Josh Berkus wrote: On 10/20/2016 06:34 AM, Joshua D. Drake wrote: On 10/19/2016 07:22 PM, Josh Berkus wrote: On 10/19/2016 06:27 PM, Joshua D. Drake wrote: Hrm, true although that is by far a minority of our users. What if we made it so we disabled the autovacuum guc

Re: [HACKERS] Renaming of pg_xlog and pg_clog

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 12, 2016 at 10:22 PM, Michael Paquier > wrote: > > OK. I can live with that as well. Attached are three patches. The > > pg_xlog -> pg_wal move, the pg_clog -> pg_transaction move, and the > > pg_clog -> pg_xact

Re: [HACKERS] Remove autovacuum GUC?

On Thu, Oct 20, 2016 at 11:53 AM, Joshua D. Drake wrote: > The right answer isn't the answer founded in the reality for many if not > most of our users. I think that's high-handed nonsense. Sure, there are some unsophisticated users who do incredibly stupid things and

Re: [HACKERS] Remove autovacuum GUC?

On 10/20/2016 09:24 AM, Robert Haas wrote: On Thu, Oct 20, 2016 at 11:53 AM, Joshua D. Drake wrote: The right answer isn't the answer founded in the reality for many if not most of our users. I think that's high-handed nonsense. Sure, there are some unsophisticated

Re: [HACKERS] Renaming of pg_xlog and pg_clog

Robert Haas writes: > Is pg_xact actually better than pg_clog? Yes, because it doesn't contain the three letters "log". We have the two precedents "pg_subtrans" and "pg_multixact", so unless we want to get into renaming those too, I think "pg_trans" and "pg_xact" are

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 2:09 PM, Tom Lane wrote: > Robert Haas writes: >> Is pg_xact actually better than pg_clog? > > Yes, because it doesn't contain the three letters "log". I figured somebody was going to say that. > We have the two precedents

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 02:02:27PM -0400, Robert Haas wrote: > On Thu, Oct 20, 2016 at 1:39 PM, Bruce Momjian wrote: > > On Thu, Oct 20, 2016 at 12:29:47PM -0400, Robert Haas wrote: > >> > When it comes to the name, I tend to think of 'pg_xact' as saying "this > >> > is where we

Re: [HACKERS] emergency outage requiring database restart

On Wed, Oct 19, 2016 at 2:39 PM, Merlin Moncure wrote: > On Wed, Oct 19, 2016 at 9:56 AM, Bruce Momjian wrote: >> On Wed, Oct 19, 2016 at 08:54:48AM -0500, Merlin Moncure wrote: >>> > Yeah. Believe me -- I know the drill. Most or all the damage seemed >>>

Re: [HACKERS] Renaming of pg_xlog and pg_clog

* David Fetter (da...@fetter.org) wrote: > On Thu, Oct 20, 2016 at 02:23:32PM -0400, Tom Lane wrote: > > Robert Haas writes: > > > On Thu, Oct 20, 2016 at 2:09 PM, Tom Lane wrote: > > >> We have the two precedents "pg_subtrans" and "pg_multixact", so >

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 1:39 PM, Bruce Momjian wrote: > On Thu, Oct 20, 2016 at 12:29:47PM -0400, Robert Haas wrote: >> > When it comes to the name, I tend to think of 'pg_xact' as saying "this >> > is where we persist info we need to keep about transactions." Today >> > that's

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 2:23 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Oct 20, 2016 at 2:09 PM, Tom Lane wrote: >>> We have the two precedents "pg_subtrans" and "pg_multixact", so >>> unless we want to get into renaming

Re: [HACKERS] emergency outage requiring database restart

Merlin Moncure writes: > single user mode dumps core :( You've got a mess there :-( > Missing separate debuginfos, use: debuginfo-install > postgresql95-server-9.5.2-1PGDG.rhel6.x86_64 This backtrace would likely be much more informative if you did the above.

Re: [HACKERS] emergency outage requiring database restart

On Thu, Oct 20, 2016 at 2:07 PM, Tom Lane wrote: > Merlin Moncure writes: >> single user mode dumps core :( > > You've got a mess there :-( > >> Missing separate debuginfos, use: debuginfo-install >> postgresql95-server-9.5.2-1PGDG.rhel6.x86_64 > > This

Re: [HACKERS] Renaming of pg_xlog and pg_clog

Stephen Frost writes: > * David Fetter (da...@fetter.org) wrote: >> On Thu, Oct 20, 2016 at 02:23:32PM -0400, Tom Lane wrote: >>> I don't see one single one of those subdirectory names that I'd call >>> self-documenting. >> That's a problem we should do something about, even

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 12:29:47PM -0400, Robert Haas wrote: > > When it comes to the name, I tend to think of 'pg_xact' as saying "this > > is where we persist info we need to keep about transactions." Today > > that's just the commit status info, but I could imagine that there > > might,

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 02:23:32PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Thu, Oct 20, 2016 at 2:09 PM, Tom Lane wrote: > >> We have the two precedents "pg_subtrans" and "pg_multixact", so > >> unless we want to get into renaming those too,

Re: [HACKERS] emergency outage requiring database restart

On Thu, Oct 20, 2016 at 1:52 PM, Merlin Moncure wrote: > On Wed, Oct 19, 2016 at 2:39 PM, Merlin Moncure wrote: >> On Wed, Oct 19, 2016 at 9:56 AM, Bruce Momjian wrote: >>> On Wed, Oct 19, 2016 at 08:54:48AM -0500, Merlin Moncure wrote:

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 10/20/2016 07:59 PM, Robert Haas wrote: On Thu, Oct 20, 2016 at 11:45 AM, Robert Haas wrote: On Thu, Oct 20, 2016 at 3:36 AM, Dilip Kumar wrote: On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote: >> ... So here's

Re: [HACKERS] emergency outage requiring database restart

Merlin Moncure wrote: > single user mode dumps core :( > > bash-4.1$ postgres --single -D /var/lib/pgsql/9.5/data castaging > LOG: 0: could not change directory to "/root": Permission denied > LOCATION: resolve_symlinks, exec.c:293 > Segmentation fault (core dumped) > > Core was generated

Re: [HACKERS] Improve output of BitmapAnd EXPLAIN ANALYZE

Jim Nasby writes: > A customer just pinged me wondering how it was that a BitmapAnd node was > reporting 0 tuples when the Bitmap Heap Scan above it showed it had in > fact generated tuples. > While this is mentioned in the docs, I think it would be very helpful to >

Re: [HACKERS] Improve output of BitmapAnd EXPLAIN ANALYZE

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Jim Nasby writes: > > A customer just pinged me wondering how it was that a BitmapAnd node was > > reporting 0 tuples when the Bitmap Heap Scan above it showed it had in > > fact generated tuples. > > > While this is

Re: [HACKERS] WIP: Fix invalid XML explain plans for track_io_timing

I wrote: > Markus Winand writes: >> The XML output of explain potentially outputs the XML tag names >> "I/O-Write-Time" >> and "I/O-Read-Time", which are invalid due to the slash. > Ooops. After further thought I decided we should go with the whitelist solution. The

Re: [HACKERS] emergency outage requiring database restart

On Thu, Oct 20, 2016 at 3:16 PM, Alvaro Herrera wrote: > Merlin Moncure wrote: > >> single user mode dumps core :( >> >> bash-4.1$ postgres --single -D /var/lib/pgsql/9.5/data castaging >> LOG: 0: could not change directory to "/root": Permission denied >> LOCATION:

[HACKERS] Improve output of BitmapAnd EXPLAIN ANALYZE

A customer just pinged me wondering how it was that a BitmapAnd node was reporting 0 tuples when the Bitmap Heap Scan above it showed it had in fact generated tuples. While this is mentioned in the docs, I think it would be very helpful to have ANALYZE spit out "N/A" instead of 0 for these

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Fri, Oct 21, 2016 at 12:35 AM, Robert Haas wrote: > On Wed, Oct 12, 2016 at 10:22 PM, Michael Paquier > wrote: >> OK. I can live with that as well. Attached are three patches. The >> pg_xlog -> pg_wal move, the pg_clog -> pg_transaction move,

Re: [HACKERS] Improve output of BitmapAnd EXPLAIN ANALYZE

Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> That would break code that tries to parse that stuff, eg depesz.com. > I don't believe Jim was suggesting that we back-patch such a change. I don't either. > Changing it in a new major release seems entirely

Re: [HACKERS] Indirect indexes

On Thu, Oct 20, 2016 at 1:08 PM, Pavan Deolasee wrote: > On Thu, Oct 20, 2016 at 9:20 PM, Claudio Freire > wrote: >> >> >> >> With indirect indexes, since you don't need to insert a tid, you can >> just "insert on conflict do nothing" on the

[HACKERS] Typo in pgstat.c

Hi, Attached patch fixes a typo in pgstat.c s/addtions/additions/g Regards, Vinayak Pokale NTT Open Source Software Center typo-pgstat-c.patch Description: application/download -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 3:46 PM, Tom Lane wrote: > I'm mostly with Stephen on this. As the names stand, they encourage > people to go look at the documentation, > https://www.postgresql.org/docs/devel/static/storage-file-layout.html > which will provide more information than

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Thu, Oct 20, 2016 at 6:03 PM, Robert Haas wrote: > On Thu, Oct 20, 2016 at 3:46 PM, Tom Lane wrote: > > I'm mostly with Stephen on this. As the names stand, they encourage > > people to go look at the documentation, > >

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Oct 20, 2016 at 9:03 PM, Tomas Vondra wrote: > In the results you've posted on 10/12, you've mentioned a regression with 32 > clients, where you got 52k tps on master but only 48k tps with the patch (so > ~10% difference). I have no idea what scale was used

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Oct 20, 2016 at 9:15 PM, Robert Haas wrote: > So here's my theory. The whole reason why Tomas is having difficulty > seeing any big effect from these patches is because he's testing on > x86. When Dilip tests on x86, he doesn't see a big effect either, >

[HACKERS] Typo in pgstat.h

Hi, - * st_progress_command_target, and st_progress_command[]. + * st_progress_command_target, and st_progress_param[]. Attached patch fixed typo. Regards, Vinayak Pokale NTT Open Source Software Center typo-pgstat-h.patch Description: application/download -- Sent via

Re: [HACKERS] Renaming of pg_xlog and pg_clog

On Fri, Oct 21, 2016 at 10:03 AM, Robert Haas wrote: > On Thu, Oct 20, 2016 at 3:46 PM, Tom Lane wrote: >> I'm mostly with Stephen on this. As the names stand, they encourage >> people to go look at the documentation, >>

Re: [HACKERS] FSM corruption leading to errors

On Thu, Oct 20, 2016 at 3:37 PM, Pavan Deolasee wrote: > Just to clarify, I meant if we truncate the entire FSM then we'll need API > to truncate VM as well so that VACUUM rebuilds everything completely. OTOH > if we provide a function just to truncate FSM to match the

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Oct 20, 2016 at 4:04 PM, Tomas Vondra wrote: >> I then started a run at 96 clients which I accidentally killed shortly >> before it was scheduled to finish, but the results are not much >> different; there is no hint of the runaway CLogControlLock contention

Re: [HACKERS] Fun fact about autovacuum and orphan temp tables

On Fri, Oct 21, 2016 at 2:29 PM, Michael Paquier wrote: > On Thu, Oct 20, 2016 at 9:30 PM, Constantin S. Pan wrote: >> I tried to fix the problem with a new backend not being >> able to reuse a temporary namespace when it contains >> thousands of

Re: [HACKERS] Remove autovacuum GUC?

On 21 Oct. 2016 12:57 am, "Joshua D. Drake" wrote: > > Hello, > > What about a simpler solution to all of this. Let's just remove it from postgresql.conf. Out of sight. If someone needs to test they can but a uneducated user won't immediately know what to do about that

Re: [HACKERS] Fun fact about autovacuum and orphan temp tables

On Thu, Sep 8, 2016 at 12:38 AM, Robert Haas wrote: > On Mon, Sep 5, 2016 at 1:14 PM, Bruce Momjian wrote: >> I don't think we look at those temp tables frequently enough to justify >> keeping them around for all users. > > +1. I think it would be much

Re: [HACKERS] Fun fact about autovacuum and orphan temp tables

On Thu, Oct 20, 2016 at 9:30 PM, Constantin S. Pan wrote: > I tried to fix the problem with a new backend not being > able to reuse a temporary namespace when it contains > thousands of temporary tables. I disabled locking of objects > during namespace clearing process. See the

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Oct 19, 2016 at 9:17 PM, Robert Haas wrote: > On Thu, Oct 13, 2016 at 7:27 AM, Amit Langote > wrote: >> However, when I briefly read the description in "Transaction Management in >> the R* Distributed Database Management System (C.

[HACKERS] WIP: Fix invalid XML explain plans for track_io_timing

Hi! The XML output of explain potentially outputs the XML tag names "I/O-Write-Time" and "I/O-Read-Time", which are invalid due to the slash. This can easily be seen with this: set track_io_timing = on; explain (analyze true, buffers true, format xml) select 1; [...] 0.000 0.000

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Oct 20, 2016 at 11:45 AM, Robert Haas wrote: > On Thu, Oct 20, 2016 at 3:36 AM, Dilip Kumar wrote: >> On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote: >>> I agree with these conclusions. I had a chance to talk

Re: [HACKERS] Renaming of pg_xlog and pg_clog

Robert Haas writes: > On Thu, Oct 20, 2016 at 2:09 PM, Tom Lane wrote: >> We have the two precedents "pg_subtrans" and "pg_multixact", so >> unless we want to get into renaming those too, I think "pg_trans" >> and "pg_xact" are really the only options

Re: [HACKERS] Fun fact about autovacuum and orphan temp tables

On Mon, 5 Sep 2016 14:54:05 +0300 Grigory Smolkin wrote: > Hello, hackers! > > We were testing how well some application works with PostgreSQL and > stumbled upon an autovacuum behavior which I fail to understand. > Application in question have a habit to heavily use

Re: [HACKERS] PATCH: two slab-like memory allocators

On 10/19/2016 02:51 PM, Tomas Vondra wrote: ... > Yeah. There are three contexts in reorder buffers: - changes (fixed size) - txns (fixed size) - tuples (variable size) The first two work perfectly fine with Slab. The last one (tuples) is used to allocate variable-sized bits, so I've tried

Re: [HACKERS] Indirect indexes

Bruce Momjian wrote: > Just to clarify, if a feature improves performance by 1%, but is enabled > by default, that is 10x more useful across our entire user base as the > feature numbers listed above, 1% vs 0.1%. Great. But not all users are alike. We have big profile users that write blog

[HACKERS] Re: File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all')

Aleksander, * Aleksander Alekseev (a.aleks...@postgrespro.ru) wrote: > > The idea is to record application workload in real environment and write > > a benchmark based on this record. Then using this benchmark we could try > > different OS/DBMS configuration (or maybe hardware), find an extremum,

[HACKERS] Remove autovacuum GUC?

Hello, After all these years, we are still regularly running into people who say, "performance was bad so we disabled autovacuum". I am not talking about once in a while, it is often. I would like us to consider removing the autovacuum option. Here are a few reasons: 1. It does not hurt

Re: [HACKERS] Indirect indexes

On Wed, Oct 19, 2016 at 01:04:16PM -0400, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 06:58:05PM +0200, Simon Riggs wrote: > > >> I agree. Also, I think the recheck mechanism will have to be something > > >> like > > >> what I wrote for WARM i.e. only checking for index quals won't be enough

Re: [HACKERS] LLVM Address Sanitizer (ASAN) and valgrind support

On Wed, Oct 19, 2016 at 11:08:39AM +0100, Greg Stark wrote: > On Sat, Feb 6, 2016 at 4:52 AM, Noah Misch wrote: > > aset.c relies on the fact that VALGRIND_MEMPOOL_ALLOC() has an implicit > > VALGRIND_MAKE_MEM_UNDEFINED() and VALGRIND_MEMPOOL_FREE() has an implicit > >

Re: [HACKERS] PATCH: two slab-like memory allocators

On Tue, Oct 18, 2016 at 6:27 PM, Petr Jelinek wrote: > I agree though that the usability beyond the ReoderBuffer is limited > because everything that will want to use it for part of allocations will > get much more complicated by the fact that it will have to use two >

Re: [HACKERS] Hash Indexes

On Tue, Oct 18, 2016 at 8:27 PM, Amit Kapila wrote: > By this problem, I mean to say deadlocks for suspended scans, that can > happen in btree for non-Mvcc or other type of scans where we don't > release pin during scan. In my mind, we have below options: > > a. problem

Re: [HACKERS] incorrect libpq comment

On Wed, Oct 19, 2016 at 1:35 PM, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 01:16:28PM -0400, Robert Haas wrote: >> Bruce's commit 5d305d86bd917723f09ab4f15c075d90586a210a back in April >> of 2014 includes this change: >> >> /* See PQconnectPoll() for how we use 'int'

Re: [HACKERS] Avoiding pin scan during btree vacuum

On Wed, Oct 19, 2016 at 6:30 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Mon, Jan 4, 2016 at 10:30 AM, Tom Lane wrote: >> >> This seems like a might subtle thing to backpatch. If we really want to >> >> go there, ISTM that the relevant code

Re: [HACKERS] File content logging during execution of COPY queries

On 10/20/2016 12:36 PM, Aleksander Alekseev wrote: According to my colleagues it would be very nice to have this feature. For instance, if you are trying to optimize PostgreSQL for application that uses COPY and you don't have access to or something like this. It could also be useful in some