[GENERAL] Detecting corruption

2014-12-08 Thread Anthony Nowocien
Hi everyone, articles from the wiki about corruption, Christophe Petus' talk in PgCon Eu2014 were of great interest but I'm still slightly at a loss in an issue I've faced recently. Please assume the following: - Version 9.0.18 on Debian boxes; - Master/slave integrated replication and a back and

Re: FW: [GENERAL] SQL rolling window without aggregation

2014-12-08 Thread David Johnston
On Monday, December 8, 2014, Huang, Suya wrote: > -Original Message- > From: pgsql-general-ow...@postgresql.org [mailto: > pgsql-general-ow...@postgresql.org ] On Behalf Of David G > Johnston > Sent: Monday, December 08, 2014 1:18 PM > To: pgsql-general@postgresql.org > Subject: Re: FW:

Re: FW: [GENERAL] SQL rolling window without aggregation

2014-12-08 Thread Huang, Suya
-Original Message- From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of David G Johnston Sent: Monday, December 08, 2014 1:18 PM To: pgsql-general@postgresql.org Subject: Re: FW: [GENERAL] SQL rolling window without aggregation Huang, Suya wrot

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-08 Thread Tom Lane
Scott Marlowe writes: > If you're de-duping a whole table, no need to create indexes, as it's > gonna have to hit every row anyway. Fastest way I've found has been: > select a,b,c into newtable from oldtable group by a,b,c; > On pass, done. > If you want to use less than the whole row, you can

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-08 Thread Scott Marlowe
If you're de-duping a whole table, no need to create indexes, as it's gonna have to hit every row anyway. Fastest way I've found has been: select a,b,c into newtable from oldtable group by a,b,c; On pass, done. If you want to use less than the whole row, you can use select distinct on (col1, col

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-08 Thread Andy Colson
On 12/08/2014 03:59 PM, Daniel Begin wrote: Thanks for your answers Andy; I will keep in mind the procedure you proposed. About the fields required to find duplicate records, all of them are required (5-9) depending on the table. Considering that the tables are not indexed yet, am I right to th

Re: [GENERAL] Speeding up an in-progress wraparound-preventing vacuum

2014-12-08 Thread Vincent de Phily
On Monday 08 December 2014 10:17:37 Jeff Janes wrote: > On Mon, Dec 8, 2014 at 4:54 AM, Vincent de Phily < > > > bloat, which I'd like to get back asap). Currently about 80% of the IO is > > devoted to the vacuum process (on average throughout the day, as > > extrapolated > > from atop output). >

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-08 Thread John R Pierce
On 12/8/2014 1:59 PM, Daniel Begin wrote: Thanks for your answers Andy; I will keep in mind the procedure you proposed. About the fields required to find duplicate records, all of them are required (5-9) depending on the table. these tables have no field that would normally be designated 'prim

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-08 Thread Daniel Begin
Thanks for your answers Andy; I will keep in mind the procedure you proposed. About the fields required to find duplicate records, all of them are required (5-9) depending on the table. Considering that the tables are not indexed yet, am I right to think that both approaches will need a full tab

Re: Fwd: Fwd: [GENERAL] Problem with pg_dump and decimal mark

2014-12-08 Thread Adrian Klaver
On 12/08/2014 06:53 AM, Eric Svenson wrote: Hi Adrian, I try to get access to the non-VM machine, at the moment access is not possible for me unfortunately. You are right, there are more tables in the database which are restored correctly but these tables do NOT contain float values. These two

Re: [GENERAL] Streaming Replication - changing IP addresses

2014-12-08 Thread John R Pierce
On 12/8/2014 11:56 AM, Dara Unglaube wrote: We have streaming replication set up on two servers that are on our local network using their external/public IP addresses. We are switching internet providers and need to change the external/public IP addresses of both servers. I'm not sure how to go

Re: [GENERAL] Divergences in view source code - both servers 9.3.5

2014-12-08 Thread Tom Lane
Edson Richter writes: > Ubuntu is running 9.3.4 (I've installed 9.3.5 - but for some reason, it > have not started). You probably just didn't restart the server after updating the package. I know that on Red Hat it's not customary for package updates to forcibly restart daemons, and the same is

[GENERAL] Streaming Replication - changing IP addresses

2014-12-08 Thread Dara Unglaube
Greetings. We have streaming replication set up on two servers that are on our local network using their external/public IP addresses. We are switching internet providers and need to change the external/public IP addresses of both servers. I'm not sure how to go about this correctly. 1. Our loc

Re: [GENERAL] Divergences in view source code - both servers 9.3.5

2014-12-08 Thread Edson Richter
Right, Tom! (again :-) ) Ubuntu is running 9.3.4 (I've installed 9.3.5 - but for some reason, it have not started). I suppose I'll uninstall 9.3.4 and then install 9.3.5. Since it is a development machine, I can do that safely. Thanks for your fast and precise answer, as always! Atenciosament

Re: [GENERAL] Divergences in view source code - both servers 9.3.5

2014-12-08 Thread Tom Lane
Edson Richter writes: > I do have two servers running 9.3.5. > One server is CentOS 6.5 x64 with all updates. > The other server is Ubuntu 14.04-1 x64 with all updates. > Both run PostgreSQL 9.3.5 x64 - one installed using YUM, other with APT. > When I compare source code of the same view, I get

Re: [GENERAL] Speeding up an in-progress wraparound-preventing vacuum

2014-12-08 Thread Jeff Janes
On Mon, Dec 8, 2014 at 4:54 AM, Vincent de Phily < vincent.deph...@mobile-devices.fr> wrote: > Hi List, > > I have a "autovacuum: VACUUM ANALYZE public.some_big_table (to prevent > wraparound)" that has been running for over 13 days. The process is > consuming > IO so I'm confident it isn't stuck,

[GENERAL] Divergences in view source code - both servers 9.3.5

2014-12-08 Thread Edson Richter
I do have two servers running 9.3.5. One server is CentOS 6.5 x64 with all updates. The other server is Ubuntu 14.04-1 x64 with all updates. Both run PostgreSQL 9.3.5 x64 - one installed using YUM, other with APT. When I compare source code of the same view, I get the following differences: *_

Re: [GENERAL] Will modifications to unlogged tables also be flused to disk?

2014-12-08 Thread Jeff Janes
On Mon, Dec 8, 2014 at 4:43 AM, frank wrote: > Jeff Janes wrote > > That's not really true. There are no per-row WAL records. There is > still > > a per-transaction WAL record, the commit record. If you only care about > > the > > timing of the WAL and not the volume, changing to unlogged will

Re: [GENERAL] Removing duplicate records from a bulk upload

2014-12-08 Thread Andy Colson
On 12/8/2014 10:30 AM, Andy Colson wrote: On 12/7/2014 9:31 PM, Daniel Begin wrote: I have just completed the bulk upload of a large database. Some tables have billions of records and no constraints or indexes have been applied yet. About 0.1% of these records may have been duplicated during the

Re: [GENERAL] Removing duplicate records from a bulk upload

2014-12-08 Thread Andy Colson
On 12/7/2014 9:31 PM, Daniel Begin wrote: I have just completed the bulk upload of a large database. Some tables have billions of records and no constraints or indexes have been applied yet. About 0.1% of these records may have been duplicated during the upload and I need to remove them before ap

[GENERAL] Incremental update for matview - automatic trigger-in-c generator

2014-12-08 Thread Nguyễn Trần Quốc Vinh
Dear sir/madam. We build a program that can automatically generate triggers in C-language for synchronous incremental matview update. It supports queries with limitations: - inner join - group by with sum, count, avg, min, max - the join has to be of the form: t1 1-n t2 1-n t3 1-n t4 The bina

Re: Fwd: Fwd: [GENERAL] Problem with pg_dump and decimal mark

2014-12-08 Thread Adrian Klaver
On 12/08/2014 06:53 AM, Eric Svenson wrote: Hi Adrian, I try to get access to the non-VM machine, at the moment access is not possible for me unfortunately. You are right, there are more tables in the database which are restored correctly but these tables do NOT contain float values. These two

Re: Fwd: Fwd: [GENERAL] Problem with pg_dump and decimal mark

2014-12-08 Thread Adrian Klaver
On 12/08/2014 12:21 AM, Eric Svenson wrote: Hi Adrian, here are the results of today: > How where the Postgres instances installed? > From a package? Yes. It is Version 9.2 for Windows, comment of the package is "The PostgreSQL RDBMS, version 9.2, packaged by EnterpriseDB" > Compiled a

[GENERAL] Speeding up an in-progress wraparound-preventing vacuum

2014-12-08 Thread Vincent de Phily
Hi List, I have a "autovacuum: VACUUM ANALYZE public.some_big_table (to prevent wraparound)" that has been running for over 13 days. The process is consuming IO so I'm confident it isn't stuck, but it's still taking surprisingly long. PG 9.1.13 on Debian. The actual table is 584G on a SAN, plu

Re: [GENERAL] Will modifications to unlogged tables also be flused to disk?

2014-12-08 Thread frank
Jeff Janes wrote > That's not really true. There are no per-row WAL records. There is still > a per-transaction WAL record, the commit record. If you only care about > the > timing of the WAL and not the volume, changing to unlogged will not make a > difference. (These commit-only records are au

[GENERAL] Updating single/multiple fields of JSON document

2014-12-08 Thread bln prasad
Hi, Is there any way to update single/multiple fields of JSON document of a column? Thanks, BLN

Re: [GENERAL] Strange behavior in generate_series(date, date, interval) with DST

2014-12-08 Thread Francisco Olarte
Hi Sérgio: On Sun, Dec 7, 2014 at 9:11 PM, Sérgio Saquetim wrote: > I've noticed a strange behavior in the generate_series functions. > > I'm trying to get all days between a start and an end date including the > bounds. So naturally I've tried something like the query below > ​. > As both your