Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-14 Thread Thomas Munro
On Mon, Aug 14, 2017 at 12:32 PM, Andres Freund wrote: > Review for 0001: > > I think you made a few long lines even longer, like: > > @@ -1106,11 +1106,11 @@ pltcl_trigger_handler(PG_FUNCTION_ARGS, > pltcl_call_state *call_state, >

Re: [HACKERS] INSERT .. ON CONFLICT DO SELECT [FOR ..]

2017-08-14 Thread Peter Geoghegan
On Mon, Aug 14, 2017 at 6:23 PM, Marko Tiikkaja wrote: > Attached is a patch for $SUBJECT. It might still be a bit rough around the > edges and probably light on docs and testing, but I thought I'd post it > anyway so I won't forget. Is it possible for ON CONFLICT DO SELECT to

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Michael Paquier
On Tue, Aug 15, 2017 at 12:45 PM, Alvaro Herrera wrote: > I'm thinking that this data is useful to analyze as a stream of related > events, rather than as individual data points. Grepping logs in order to > extract the numbers is lame and slow. If you additionally have

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Alvaro Herrera
Michael Paquier wrote: > On Tue, Aug 15, 2017 at 11:14 AM, Alvaro Herrera > wrote: > >> In vacuum_rel()@vacuum.c, there are a couple of logs that could be > >> improved as well with the schema name. > > > > I agree that there's a lot of room for improvement there. If

Re: [HACKERS] reload-through-the-top-parent switch the partition table

2017-08-14 Thread Robert Haas
On Mon, Aug 14, 2017 at 12:40 AM, Rushabh Lathia wrote: > On Fri, Aug 11, 2017 at 10:50 PM, Robert Haas wrote: >> On Fri, Aug 11, 2017 at 5:36 AM, Rushabh Lathia >> wrote: >> > Please find attach patch with the changes.

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Michael Paquier
On Tue, Aug 15, 2017 at 3:56 AM, Andres Freund wrote: > I think there are some possibilities to close the gap here. We could > e.g. have .delete_on_crash marker files that get installed > when creating a new persistent relfilenode. If we set up things so they > get deleted

Re: [HACKERS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-14 Thread Noah Misch
On Fri, Aug 11, 2017 at 11:59:14AM -0400, Tom Lane wrote: > "Adam, Etienne (Nokia-TECH/Issy Les Moulineaux)" > writes: > > ERROR: XX000: unrecognized node type: 90 > > LOCATION: ExecReScan, execAmi.c:284 > > (gdb) p (NodeTag) 90 > $1 = T_GatherMergeState > > So,

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-08-14 Thread Peter Eisentraut
On 3/11/17 07:06, Pavel Stehule wrote: > I am sending a updated version with separated sort direction in special > variable This patch also needs a rebase. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-08-14 Thread Peter Eisentraut
On 3/15/17 11:56, David Steele wrote: >> This patch has been moved to CF 2017-07. > > I did not manage to move this patch when I said had. It is now moved. Unsurprisingly, this patch needs a major rebase. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

2017-08-14 Thread Craig Ringer
On 15 August 2017 at 10:16, Michael Paquier wrote: > On Tue, Aug 15, 2017 at 11:10 AM, Craig Ringer > wrote: > > Ooh, this finally gives us a path toward case-insensitive default > database > > collation via CLDR caseLevel. > > > >

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Michael Paquier
On Tue, Aug 15, 2017 at 11:14 AM, Alvaro Herrera wrote: >> In vacuum_rel()@vacuum.c, there are a couple of logs that could be >> improved as well with the schema name. > > I agree that there's a lot of room for improvement there. If I'm > allowed some scope creep, I'd

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

2017-08-14 Thread Michael Paquier
On Tue, Aug 15, 2017 at 11:10 AM, Craig Ringer wrote: > Ooh, this finally gives us a path toward case-insensitive default database > collation via CLDR caseLevel. > > http://userguide.icu-project.org/collation >

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Alvaro Herrera
Michael Paquier wrote: > On Tue, Aug 15, 2017 at 10:27 AM, Masahiko Sawada > wrote: > > Currently vacuum verbose outputs vacuum logs as follows. The first log > > message INFO: vacuuming "public.hoge" writes the relation name with > > schema name but subsequent vacuum logs

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

2017-08-14 Thread Craig Ringer
On 10 August 2017 at 06:49, Peter Geoghegan wrote: > There are actually very many customizations to collations that are > possible beyond what the "stock" ICU collations provide (whatever > "stock" means). Some of these are really cool, and I can imagine use > cases where they are

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-08-14 Thread Craig Ringer
On 22 March 2017 at 01:17, Robert Haas wrote: > On Sun, Mar 12, 2017 at 10:20 PM, Thomas Munro > wrote: > > Maybe someone can think of a clever way for an extension to insert a > > wait for a user-supplied LSN *before* acquiring a snapshot

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-14 Thread Craig Ringer
On 15 August 2017 at 09:11, Moon Insung wrote: > Dear Craig Ringer > > > > Frist, thank you for implementing the necessary function. > > > > but, i have some question. > > > > question 1) vacuum freeze hint bits > > If run a vacuum freeze, bits in the infomask will

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Michael Paquier
On Tue, Aug 15, 2017 at 10:27 AM, Masahiko Sawada wrote: > Currently vacuum verbose outputs vacuum logs as follows. The first log > message INFO: vacuuming "public.hoge" writes the relation name with > schema name but subsequent vacuum logs output only relation name >

Re: [HACKERS] measuring the impact of increasing WAL segment size

2017-08-14 Thread Andres Freund
Hi, Thanks for running this! On 2017-08-15 03:27:00 +0200, Tomas Vondra wrote: > Granted - this chart does not show latency, so it's not a complete > picture. That'd be quite useful to see here, too. > Also, if you care about raw OLTP performance you're probably already running > on flash,

[HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-14 Thread Masahiko Sawada
Hi all, Currently vacuum verbose outputs vacuum logs as follows. The first log message INFO: vacuuming "public.hoge" writes the relation name with schema name but subsequent vacuum logs output only relation name without schema name. I've encountered a situation where there are some same name

[HACKERS] INSERT .. ON CONFLICT DO SELECT [FOR ..]

2017-08-14 Thread Marko Tiikkaja
Hi, Attached is a patch for $SUBJECT. It might still be a bit rough around the edges and probably light on docs and testing, but I thought I'd post it anyway so I won't forget. .m insert_conflict_select_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-14 Thread Moon Insung
Dear Craig Ringer Frist, thank you for implementing the necessary function. but, i have some question. question 1) vacuum freeze hint bits If run a vacuum freeze, bits in the infomask will be 0x0300. in this case, if output the value of informsk in the run to you modified,

Re: [HACKERS] Fix a typo in sequence.c

2017-08-14 Thread Masahiko Sawada
On Tue, Aug 15, 2017 at 2:53 AM, Peter Eisentraut wrote: > On 8/14/17 08:32, Masahiko Sawada wrote: >> While reading source code, I found a typo in sequence.c file. Attached >> patch for fixing it. >> >> s/localy/locally/g > > Fixed. Thank you! Regards, --

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Aug 14, 2017 at 1:12 PM, Alvaro Herrera > wrote: > > Yeah, the problem that lwlocks aren't released is because the launcher > > is not in a transaction at that point, so AbortCurrentTransaction() > > doesn't release locks like it normally

Re: [HACKERS] postgres_fdw: evaluate placeholdervars on remote server

2017-08-14 Thread Peter Eisentraut
On 4/3/17 22:00, Etsuro Fujita wrote: > On 2017/04/04 3:21, Andres Freund wrote: >> On 2017-02-28 21:45:22 +0900, Etsuro Fujita wrote: >>> Here is a patch for $subject. >> >> This is a nontrivial patch, submitted just before the start of the last >> CF for postgres 10. Therefore I think we should

Re: [HACKERS] Statement-level rollback

2017-08-14 Thread Peter Eisentraut
On 2/28/17 02:39, Tsunakawa, Takayuki wrote: > The code for stored functions is not written yet, but I'd like your feedback > for the specification and design based on the current patch. I'll add this > patch to CommitFest 2017-3. This patch needs to be rebased for the upcoming commit fest.

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-08-14 Thread Peter Eisentraut
On 3/9/17 07:49, Ivan Kartyshov wrote: > Here I attached rebased patch waitlsn_10dev_v3 (core feature) > I will leave the choice of implementation (core/contrib) to the > discretion of the community. This patch is registered in the upcoming commit fest, but it needs to be rebased. -- Peter

Re: [HACKERS] [COMMITTERS] pgsql: Avoid GatherMerge crash when there are no workers.

2017-08-14 Thread Andres Freund
Hi, On 2017-04-03 12:56:36 +0530, Rushabh Lathia wrote: > On my local environment I was getting coverage for the heap_compare_slots > with > existing test. But it seems like due to low number of record fetch only > leader get > evolved to the fetching tuples in the shared report. > > I modified

Re: [HACKERS] pg_stat_wal_write statistics view

2017-08-14 Thread Peter Eisentraut
On 3/29/17 22:10, Haribabu Kommi wrote: > Updated patch to use shared counter instead of adding pg_stat_ calls to send > the statistics from each background process/worker. Your patch needs to be rebased and the OID assignments updated. -- Peter Eisentraut

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-08-14 Thread Peter Eisentraut
On 1/24/17 02:58, Kyotaro HORIGUCHI wrote: >> BTW, if you set a slightly larger >> context size on the patch you might be able to avoid rebases; right >> now the patch doesn't include enough context to uniquely identify the >> chunks against cacheinfo[]. > git format-patch -U5 fuses all hunks on

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2017-08-14 Thread Peter Eisentraut
On 4/4/17 01:06, Haribabu Kommi wrote: > Both pg_dump and pg_upgrade tests are passed. Updated patch attached > I will add this patch to the next commitfest. This patch needs to be rebased for the upcoming commit fest. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] pgbench more operators & functions

2017-08-14 Thread Peter Eisentraut
On 5/24/17 03:14, Fabien COELHO wrote: > I've improved it in attached v11: > - add a link to the CASE full documentation > - add an example expression with CASE ... This patch needs (at least) a rebase for the upcoming commit fest. -- Peter Eisentraut

Re: [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-08-14 Thread Peter Eisentraut
On 3/18/17 12:51, Pavel Stehule wrote: > I rewrote these patches - it allows binary export/import from psql and > the code is very simple. The size of the patch is bigger due including > 4KB binary file (in hex format 8KB). This patch needs (at least) a rebase for the upcoming commit fest. --

Re: [HACKERS] Add Roman numeral conversion to to_number

2017-08-14 Thread Merlin Moncure
On Mon, Aug 14, 2017 at 2:48 PM, Peter Eisentraut wrote: > On 8/3/17 13:45, Robert Haas wrote: >> On Thu, Aug 3, 2017 at 9:25 AM, Oliver Ford wrote: >>> Adds to the to_number() function the ability to convert Roman numerals >>> to a number.

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-08-14 Thread Tom Lane
Alexander Korotkov writes: > On Mon, Aug 14, 2017 at 2:09 PM, Mark Rofail wrote: >> I think we should cast the operands in the RI queries fired as follows >> 1. we get the array type from the right operand >> 2. compare the two array type and see

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-14 Thread Tom Lane
I wrote: > Sandeep Thakkar writes: >> We built the sources with this patch and were able to create the plperl >> extension on Windows 32bit and 64bit. > Excellent, thanks for testing. I'll finish up the configure-script part > and push this shortly. So the

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-08-14 Thread Alexander Korotkov
On Mon, Aug 14, 2017 at 2:09 PM, Mark Rofail wrote: > On Tue, Aug 8, 2017 at 3:24 PM, Alexander Korotkov > wrote: > >> On Tue, Aug 8, 2017 at 4:12 PM, Mark Rofail >> wrote: >> >>> On Tue, Aug 8, 2017 at 2:25 PM, Alexander

Re: [HACKERS] Foreign tables privileges not shown in information_schema.table_privileges

2017-08-14 Thread Tom Lane
Peter Eisentraut writes: > - Materialized views not included. I think that is an intentional > omission. It's valid to reconsider, but it would be to be a separate > discussion. Yes. The problem is that matviews are not in the SQL standard, so what are you

Re: [HACKERS] Add Roman numeral conversion to to_number

2017-08-14 Thread Peter Eisentraut
On 8/3/17 13:45, Robert Haas wrote: > On Thu, Aug 3, 2017 at 9:25 AM, Oliver Ford wrote: >> Adds to the to_number() function the ability to convert Roman numerals >> to a number. This feature is on the formatting.c TODO list. It is not >> currently implemented in either Oracle,

Re: [HACKERS] Foreign tables privileges not shown in information_schema.table_privileges

2017-08-14 Thread Peter Eisentraut
On 8/11/17 04:52, Ashutosh Bapat wrote: > On Thu, Aug 10, 2017 at 6:30 PM, Nicolas Thauvin > wrote: >> Hello, >> >> The information_schema.table_privileges view filters on regular tables >> and views. Foreign tables are not shown in this view but they are in >> other

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Chris Travers
On Mon, Aug 14, 2017 at 8:40 PM, Tom Lane wrote: > > > It would be possible to have orphaned non-temp tables if you'd suffered > a crash during the transaction that created those tables. Ordinarily > a newly-created table file wouldn't be that large, but if your workflow >

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Robert Haas
On Mon, Aug 14, 2017 at 1:12 PM, Alvaro Herrera wrote: > Yeah, the problem that lwlocks aren't released is because the launcher > is not in a transaction at that point, so AbortCurrentTransaction() > doesn't release locks like it normally would. The simplest fix (in the

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Andres Freund
On 2017-08-14 14:40:46 -0400, Tom Lane wrote: > The core problem with zapping non-temp table files is that you can't > do that unless you're sure you have consistent, up-to-date pg_class > data that nobody else is busy adding to. It's hard to see an external > application being able to do that

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Tom Lane
Chris Travers writes: > On Mon, Aug 14, 2017 at 6:33 PM, Andres Freund wrote: >> I think the fix here is to call RemovePgTempFiles() during >> crash-restarts, instead of just full starts. The previously stated need >> to be able to inspect temp files

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Chris Travers
On Mon, Aug 14, 2017 at 6:33 PM, Andres Freund wrote: > Hi, > > On 2017-08-14 14:12:22 +0200, Chris Travers wrote: > > Problem: > > The system this came up on is PostgreSQL 9.6.3 and has had repeated > trouble > > with disk space. Querying pg_database_size, as well as du on

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread David Fetter
On Sun, Aug 13, 2017 at 05:56:56PM -0700, Andres Freund wrote: > Hi, > > Since we're getting a bit into the weeds of a different topic, and since > I think it's an interesting feature, I'm detaching this into a separate > thread. > > On 2017-08-12 23:37:27 -0400, Tom Lane wrote: > > >> On

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't force-assign transaction id when exporting a snapshot.

2017-08-14 Thread Andres Freund
On 2017-08-14 13:55:29 -0400, Peter Eisentraut wrote: > On 8/12/17 07:32, Petr Jelinek wrote: > > This commit has side effect that it makes it possible to export > > snapshots on the standbys. This makes it possible to do pg_dump -j on > > standby with consistent snapshot. Here is one line patch

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't force-assign transaction id when exporting a snapshot.

2017-08-14 Thread Peter Eisentraut
On 8/12/17 07:32, Petr Jelinek wrote: > This commit has side effect that it makes it possible to export > snapshots on the standbys. This makes it possible to do pg_dump -j on > standby with consistent snapshot. Here is one line patch (+ doc update) > which allows doing that when pg_dumping from

Re: [HACKERS] Fix a typo in sequence.c

2017-08-14 Thread Peter Eisentraut
On 8/14/17 08:32, Masahiko Sawada wrote: > While reading source code, I found a typo in sequence.c file. Attached > patch for fixing it. > > s/localy/locally/g Fixed. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: [HACKERS] pgbench: Skipping the creating primary keys after initialization

2017-08-14 Thread Fabien COELHO
Hello, I think we can use it like --custom-initialize="create_table, vacuum" which is similar to what we specify a connection option to psql for example. Even if it is allowed, do not advertise it. Or use space as a separator and not comma. ISTM that with psql connections space is the

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Tom Lane
Robert Haas writes: > I think it would be a bad idea to remove both > dynamic_shared_memory_type=none and dynamic_shared_memory_type=mmap. > If you do that, then somebody who doesn't have root and whose system > configuration is messed up can't start PostgreSQL at all. I'd

[HACKERS] Re: [GSOC][weekly report 9] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-08-14 Thread Mengxing Liu
In the last week, I tried these two ideas. > -Original Messages- > From: "Alvaro Herrera" > Sent Time: 2017-08-08 01:51:52 (Tuesday) > * I wonder if a completely different approach to the finished xact > maintenance problem would be helpful. For instance, in

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-14 Thread Tom Lane
Christoph Berg writes: > HEAD as of 5a5c2feca still has the same problem on kfreebsd. Is there > anything I could dump so we understand the problem better? Yeah, I did not expect that 5a5c2feca would change anything on non-Windows. What we need to do is verify that PL/Perl's

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Andres Freund
On 2017-08-14 13:06:48 -0400, Robert Haas wrote: > On Mon, Aug 14, 2017 at 1:02 PM, Andres Freund wrote: > > I previously thought that an option to occasionally WAL log the stats > > file would be useful (e.g. just before a checkpoint). That'd make them > > persistent, and

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Alvaro Herrera
Robert Haas wrote: > Actually, now that you mention it, I think it *is* broken already, or > more to the point, if you configure that value, the autovacuum > launcher hangs up, because of the AutovacuumWorkItem stuff that Alvaro > added. When I just tested it, the AV launcher somehow ended up >

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Robert Haas
On Mon, Aug 14, 2017 at 1:02 PM, Andres Freund wrote: > I previously thought that an option to occasionally WAL log the stats > file would be useful (e.g. just before a checkpoint). That'd make them > persistent, and available on the standby. But that'd still require > somehow

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Andres Freund
On 2017-08-14 18:57:39 +0200, Magnus Hagander wrote: > On Mon, Aug 14, 2017 at 5:46 PM, Robert Haas wrote: > > > On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane wrote: > > >> b) I think our tendency to dump all stats whenever we crash isn't really > > >>

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Robert Haas
On Mon, Aug 14, 2017 at 12:36 PM, Andres Freund wrote: >> If so, why isn't choose_dsm_implementation() trying it; and if not, >> why are we carrying it? > > I think the idea was that there might be platforms that require it, but > ... Right. So, for example, POSIX shared

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Magnus Hagander
On Mon, Aug 14, 2017 at 5:46 PM, Robert Haas wrote: > On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane wrote: > >> b) I think our tendency to dump all stats whenever we crash isn't really > >>tenable, given how autovacuum etc are tied to them. > > > > Eh,

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-14 Thread Christoph Berg
Re: Sandeep Thakkar 2017-08-08 > Hi > > An update from beta3 build: We are no longer seeing this issue (handshake > failure) on Windows 64bit, but on Windows 32bit it still persists. HEAD as of 5a5c2feca still has the same

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

2017-08-14 Thread Peter Geoghegan
On Mon, Aug 14, 2017 at 9:15 AM, Peter Eisentraut wrote: > I'm having trouble finding some concrete documentation for this. The TR > 35 link you showed documents the key words and values, BCP 47 documents > the syntax, but nothing puts it all together in a form

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Andres Freund
On 2017-08-14 12:28:39 -0400, Tom Lane wrote: > Robert Haas writes: > > On Mon, Aug 14, 2017 at 12:16 PM, Tom Lane wrote: > >> Just FYI, the only values being reported by buildfarm animals are > >> "posix", "sysv", and "windows". So while mmap may be a

Re: [HACKERS] FYI: branch for v11 devel is planned for next week

2017-08-14 Thread Tom Lane
I wrote: > The release team discussed this a couple weeks ago, but I don't > think anybody mentioned it on -hackers: v10 seems to be in good > enough shape that it's okay to make the REL_10_STABLE branch soon, > and open HEAD for v11 development. > Last year we branched on Aug 15, and we should

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Andres Freund
Hi, On 2017-08-14 14:12:22 +0200, Chris Travers wrote: > Problem: > The system this came up on is PostgreSQL 9.6.3 and has had repeated trouble > with disk space. Querying pg_database_size, as well as du on the > subdirectory of base/ show total usage to be around 3.8TB. Summing up the > size

Re: [HACKERS] Secondary index access optimizations

2017-08-14 Thread Konstantin Knizhnik
On 14.08.2017 12:37, Konstantin Knizhnik wrote: Hi hackers, I am trying to compare different ways of optimizing work with huge append-only tables in PostgreSQL where primary key is something like timestamp and queries are usually accessing most recent data using some secondary keys. Size

Re: [HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Chris Travers
On Aug 14, 2017 14:12, "Chris Travers" wrote: Hi all; I am trying to track down a problem we are seeing that looks very similar to bug #12050, and would certainly consider trying to contribute a fix if we agree on one. (I am not sure we can, so absent that, the next

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Tom Lane
Robert Haas writes: > On Mon, Aug 14, 2017 at 12:16 PM, Tom Lane wrote: >> Just FYI, the only values being reported by buildfarm animals are >> "posix", "sysv", and "windows". So while mmap may be a thing, >> it's an untested thing. > I'm pretty sure

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Andres Freund
On 2017-08-14 12:21:30 -0400, Robert Haas wrote: > >> ... If somebody has a system where no other form of shared > >> memory, works dynamic_shared_memory_type = mmap is still a thing, so > >> the use case for "none" seems very thin indeed. I'd vote for just > >> ripping it out in v11. > > > >

Re: [HACKERS] Re: ICU collation variant keywords and pg_collation entries (Was: [BUGS] Crash report for some ICU-52 (debian8) COLLATE and work_mem values)

2017-08-14 Thread Peter Eisentraut
On 8/13/17 15:39, Noah Misch wrote: > This PostgreSQL 10 open item is past due for your status update. Kindly send > a status update within 24 hours, and include a date for your subsequent status > update. Refer to the policy on open item ownership: >

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Robert Haas
On Mon, Aug 14, 2017 at 12:16 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane wrote: >>> In principle we could keep the existing mechanism as a fallback. >>> Whether that's worth the trouble is

Re: [HACKERS] Adding support for Default partition in partitioning

2017-08-14 Thread Robert Haas
On Mon, Aug 14, 2017 at 7:51 AM, Jeevan Ladhe wrote: > I think even with this change there will be one level of indentation > needed for throwing the error, as the error is to be thrown only if > there exists a default partition. That's true, but we don't need two

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Tom Lane
Robert Haas writes: > On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane wrote: >> In principle we could keep the existing mechanism as a fallback. >> Whether that's worth the trouble is debatable. The current code >> in initdb believes that every platform has

Re: [HACKERS] What users can do with custom ICU collations in Postgres 10

2017-08-14 Thread Peter Eisentraut
On 8/9/17 18:49, Peter Geoghegan wrote: > There are actually very many customizations to collations that are > possible beyond what the "stock" ICU collations provide (whatever > "stock" means). This is very nice indeed, and I was not aware that this was possible with what we already have in

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Alvaro Herrera
Robert Haas wrote: > Actually, now that you mention it, I think it *is* broken already, or > more to the point, if you configure that value, the autovacuum > launcher hangs up, because of the AutovacuumWorkItem stuff that Alvaro > added. When I just tested it, the AV launcher somehow ended up >

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Andres Freund
Hi, On 2017-08-14 11:46:10 -0400, Robert Haas wrote: > On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane wrote: > > Andres Freund writes: > >> You both are obviously right. Another point of potential concern could > >> be that we'd pretyt much fully rely on

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-08-14 Thread Robert Haas
On Sun, Aug 13, 2017 at 11:05 AM, Sokolov Yura wrote: > BTW, ad-hoc hash tables already exist: at least recourse owner uses one. Yeah. :-( -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-14 Thread AP
On Fri, Aug 11, 2017 at 07:33:51AM +0530, Amit Kapila wrote: > On Thu, Aug 10, 2017 at 4:11 PM, AP wrote: > > mdstash=# select * from pgstathashindex('link_datum_id_idx'); > > version | bucket_pages | overflow_pages | bitmap_pages | unused_pages | > > live_items | dead_items |

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-14 Thread Robert Haas
On Sun, Aug 13, 2017 at 9:59 PM, Tom Lane wrote: > Andres Freund writes: >> You both are obviously right. Another point of potential concern could >> be that we'd pretyt much fully rely on dsm/dht's being available, for >> the server to function

Re: [HACKERS] ICU collation variant keywords and pg_collation entries (Was: [BUGS] Crash report for some ICU-52 (debian8) COLLATE and work_mem values)

2017-08-14 Thread Peter Eisentraut
On 8/7/17 21:00, Peter Geoghegan wrote: > Actually, it's *impossible* for ICU to fail to accept any string as a > valid locale within CREATE COLLATION, because CollationCreate() simply > doesn't sanitize ICU names. It doesn't do something like call > get_icu_language_tag(), unlike initdb (within >

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-14 Thread Stephen Frost
Robert, Michael, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Sun, Aug 6, 2017 at 3:22 AM, Robert Haas wrote: > > All of (1)-(3) are legitimate user choices, although not everyone will > > make them. (4) is unfortunately the procedure recommended by our > >

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-14 Thread Tom Lane
Peter Eisentraut writes: > It appears there is a patch that fixes the issue now. Let's wait until > Wednesday to see if that ends up being successful. Jobim reported success with add-connected-event-2.patch --- are you expecting more testing to happen? I did

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-14 Thread Tom Lane
Sandeep Thakkar writes: > On Thu, Aug 10, 2017 at 9:04 PM, Tom Lane wrote: >> Got it. So in short, it seems like the attached patch ought to fix it >> for MSVC builds. (We'd also need to teach PGAC_CHECK_PERL_EMBED_CCFLAGS >> to let

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-14 Thread Peter Eisentraut
On 8/13/17 15:46, Noah Misch wrote: > IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due > for your status update. Please reacquaint yourself with the policy on open > item ownership[1] and then reply immediately. If I do not hear from you by > 2017-08-14 20:00 UTC, I

[HACKERS] Fix a typo in sequence.c

2017-08-14 Thread Masahiko Sawada
Hi, While reading source code, I found a typo in sequence.c file. Attached patch for fixing it. s/localy/locally/g Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center fix_typo_in_sequence_c.patch Description: Binary data -- Sent via

[HACKERS] Orphaned files in base/[oid]

2017-08-14 Thread Chris Travers
Hi all; I am trying to track down a problem we are seeing that looks very similar to bug #12050, and would certainly consider trying to contribute a fix if we agree on one. (I am not sure we can, so absent that, the next question is whether it makes sense to create a utility to fix the problem

Re: [HACKERS] Adding support for Default partition in partitioning

2017-08-14 Thread Jeevan Ladhe
Hi Robert, > 0005: > > Extend default partitioning support to allow addition of new partitions. > > + if (spec->is_default) > + { > + /* Default partition cannot be added if there already > exists one. */ > + if (partdesc->nparts > 0 && >

Re: [HACKERS] parallelize queries containing initplans

2017-08-14 Thread Amit Kapila
On Sun, Aug 13, 2017 at 6:49 PM, Haribabu Kommi wrote: > On Fri, Aug 11, 2017 at 1:18 AM, Amit Kapila > wrote: >> > > Thanks for the updated patch. Patch looks fine. I just have some > minor comments. > > + * ExecEvalParamExecParams > + * > + *

Re: [HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-14 Thread Michael Paquier
On Mon, Aug 14, 2017 at 6:46 PM, Aleksander Alekseev wrote: > Hi Michael, > >> It's been one month since I have done some serious development with >> Archlinux (I was abroad and away from the laptop dedicated to that), >> and surprise, I can see failures in the PG

Re: [HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-14 Thread Aleksander Alekseev
Hi Michael, > I can confirm that I see the same errors on Arch Linux with latest > updates when PostgreSQL is compiled with --with-libxml and/or > --with-libxslt. I submitted a few details on bugs.archlinux.org [1] > since probably not all Arch Linux maintainers know how to reproduce an > issue.

Re: [HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-14 Thread Aleksander Alekseev
Hi Michael, > It's been one month since I have done some serious development with > Archlinux (I was abroad and away from the laptop dedicated to that), > and surprise, I can see failures in the PG regression tests, like the > following short extract (result compared to expected/xml.out): I can

Re: [HACKERS] pl/perl extension fails on Windows

2017-08-14 Thread Sandeep Thakkar
Hi On Thu, Aug 10, 2017 at 9:04 PM, Tom Lane wrote: > Ashutosh Sharma writes: > > On Thu, Aug 10, 2017 at 8:06 PM, Tom Lane wrote: > >> Yeah ... however, if that's there, then there's something wrong with > >> Ashutosh's

[HACKERS] Secondary index access optimizations

2017-08-14 Thread Konstantin Knizhnik
Hi hackers, I am trying to compare different ways of optimizing work with huge append-only tables in PostgreSQL where primary key is something like timestamp and queries are usually accessing most recent data using some secondary keys. Size of secondary index is one of the most critical

Re: [HACKERS] pgbench: Skipping the creating primary keys after initialization

2017-08-14 Thread Masahiko Sawada
On Tue, Aug 8, 2017 at 10:50 PM, Fabien COELHO wrote: > > Hello Mahahiko-san, > > My 0.02€ about the patch & feature, which only reflect my point of view: Thank you for reviewing the patch! > Please add a number to patches to avoid ambiguities. This was patch was the >

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-14 Thread Augustine, Jobin
Thank you Tom Lane, This patch fixes the problem. With this patch, streaming replication started working (replication to Windows) (Tested for Linux to Windows replication) -Jobin. On Mon, Aug 14, 2017 at 2:17 AM, Tom Lane wrote: > Andres Freund

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-14 Thread Magnus Hagander
On Sun, Aug 13, 2017 at 11:43 PM, Robert Haas wrote: > On Sun, Aug 13, 2017 at 5:24 PM, Tom Lane wrote: > >> I'd vote for including this in v10. There doesn't seem to be any > >> downside to this: it's a no brainer to avoid our exploding hash table >

Re: [HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-14 Thread Arthur Zakirov
Hello, On Sat, Aug 12, 2017 at 08:46:38PM +0900, Michael Paquier wrote: > I can notice as well that no buildfarm machines are running ArchLinux > now, so that's one reason why this got undetected until now. My own > machines running Archlinux ARM have been unplugged for a full month, > and I can

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-14 Thread Michael Paquier
On Sun, Aug 6, 2017 at 3:22 AM, Robert Haas wrote: > All of (1)-(3) are legitimate user choices, although not everyone will > make them. (4) is unfortunately the procedure recommended by our > documentation, which is where the problem comes in. I think it's > pretty lame

Re: [HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-14 Thread Michael Paquier
On Sat, Aug 12, 2017 at 8:46 PM, Michael Paquier wrote: > I have not investigated much this problem yet, but has somebody else > seen those diffs? Coming back to that... Downgrading down to 2.9.4+4+g3169602-1 is proving to put things back the way they should. Even