[GENERAL] How to delete default privileges

2017-08-15 Thread Francisco Reyes
I have a DB where we changed ownership of all objects. We had: ALTER DEFAULT PRIVILEGES FOR ROLE old_dbowner GRANT SELECT ON tables TO dbgroup_ro_group; ALTER DEFAULT PRIVILEGES FOR ROLE old_dbowner GRANT SELECT ON sequences TO dbgroup_ro_group; ALTER DEFAULT PRIVILEGES FOR ROLE

Re: [GENERAL] Large pg_xlog

2016-09-29 Thread Francisco Reyes
On 09/29/2016 01:36 AM, Michael Paquier wrote: something is broken with your installation. Don't you have in your log files something like that? LOG: could not create archive status file "pg_xlog/archive_status/00010001.ready": No such file That was the problem. FATAL:

Re: [GENERAL] Large pg_xlog

2016-09-28 Thread Francisco Reyes
On 09/26/2016 09:05 PM, Michael Paquier wrote: What are the files in pg_xlog/archive_status? Do see a lot of .ready entries? Perhaps you could de-bloat things by using archive_command = '/bin/true' (REM on Windows as far as I recall). There is no archive_status ll 9.3/main/pg_xlog/ |grep

Re: [GENERAL] Large pg_xlog

2016-09-26 Thread Francisco Reyes
On 09/26/2016 08:08 PM, Andreas Kretschmer wrote: archive_command failed? If that happens the wal's will not deleted, you should see error-messages in the log. The only times archive command failed I believe is because the volume where pg_xlog is ran out of space. FATAL: archive command

[GENERAL] Large pg_xlog

2016-09-26 Thread Francisco Reyes
Postgresql 9.3 checkpoint_segments=6 wal_keep_segments=300 Machine is master for 2, asynchronous, slaves. pg_xlog in /var/lib/postgresql/9.3/main/pg_xlog NFS mount with WAL archiving in /opt/backups/walarchives/HostName During a load of a file, using copy, the pg_xlog grew to almost 120GB

[GENERAL] Alter default privileges vs new schemas

2016-08-24 Thread Francisco Reyes
https://www.postgresql.org/docs/current/static/sql-alterdefaultprivileges.html Have a client where their development teams use extensive use of schemas. We use default privileges like this ALTER DEFAULT PRIVILEGES FOR ROLE jobs_owner_local GRANT SELECT ON tablesTO ro_group; ALTER DEFAULT

[GENERAL] Re: pg_basebackup vs archive_command - looking for WAL archive older than archive_command start

2016-07-17 Thread Francisco Reyes
On 07/17/2016 06:35 PM, Francisco Reyes wrote: Why is the pg_basebackup restore looking for a WAL file that is even older than the ones I have, when I turned on WAL archiving before I started the pg_basebackup? Figured it out.. the error is from a secondary slave trying to sync from

[GENERAL] pg_basebackup vs archive_command - looking for WAL archive older than archive_command start

2016-07-17 Thread Francisco Reyes
I turned on archive_command and have wal archiving going. I did a pg_basebackup and copied the resulting file from source machine to target, yet when I restore I am getting requested WAL segment 000508AE009B has already been removed The earliest WAL archives I have are

[GENERAL] How to audit non LDAP connections?

2015-12-03 Thread Francisco Reyes
Due to security/audits have moved most users to LDAP. Looking for a way to tell if a connection is/is not going through LDAP. Other than errors, such as bad password, have not found a way to tell if a connection is using LDAP or postgresql internal authentication in the logs. Tried going

Re: [GENERAL] Best way to sync table DML between databases

2015-10-09 Thread Francisco Reyes
On 10/05/2015 09:46 AM, jimbosworth wrote: Im not in a position to change the database setup on server A. Can you have the owners/maintainers do the needed changes to setup replication? Or that is 100% out of the question? -- Sent via pgsql-general mailing list

Re: [GENERAL] Test failover for PosgreSql 9.2

2015-09-25 Thread Francisco Reyes
On 09/25/2015 11:20 AM, yuryu wrote: According to manual I have to kill completely Master and "touch" a trigger to make Slave new Master. You don't have to do anything in the master. If you have configured the slave to check for a file, then it will become Read Write when that file is

Re: [GENERAL] Dropped connections with pg_basebackup

2015-09-25 Thread Francisco Reyes
On 09/24/2015 04:29 PM, Sherrylyn Branchaw wrote: I'm assuming based on the "SSL error" that you have ssl set to 'on'. What's your ssl_renegotiation_limit? The default is 512MB, but setting it to 0 has solved problems for a number of people on this list, including myself. I have also seen

Re: [GENERAL] Dropped connections with pg_basebackup

2015-09-25 Thread Francisco Reyes
On 09/24/2015 04:34 PM, Alvaro Herrera wrote: Sherrylyn Branchaw wrote: Moreover, the default has been set to 0, because the bugs both in our usage and in OpenSSL code itself seem never to end. Just disable it. Set it to 0 and did not help. Likely will move all machines to have it =0 since I

[GENERAL] Dropped connections with pg_basebackup

2015-09-24 Thread Francisco Reyes
Have an existing setup of 9.3 servers. Replication has been rock solid, but recently the circuits between data centers were upgraded and pg_basebackup now seems to fail often when setting up streaming replication. What used to take 10+ hours now only took 68 minutes, but had to do many

Re: [GENERAL] How to test SSL cert from CA?

2015-07-16 Thread Francisco Reyes
On 07/11/2015 07:32 PM, James Cloos wrote: FR == Francisco Reyes li...@natserv.net writes: Did you include the intermediate cert(s) in the bundle which the server presents to the client? Yes. And did you confirm that the client trusts the issuer's root? Some require explicit configurastion

Re: [GENERAL] How to test SSL cert from CA?

2015-07-09 Thread Francisco Reyes
On 07/09/2015 03:07 PM, Vick Khera wrote: On Wed, Jul 8, 2015 at 10:17 PM, Francisco Reyes li...@natserv.net mailto:li...@natserv.net wrote: openssl s_client -connect HOST:PORT -CAfile /path/to/CA.pem According to this post: http://serverfault.com/questions/79876/connecting-to-postgresql

Re: [GENERAL] How to test SSL cert from CA?

2015-07-09 Thread Francisco Reyes
On 07/08/2015 10:52 PM, Tom Lane wrote: What's the complaint exactly? The error we are getting is: The security of this transaction may be compromised. The following SSL errors have been reported: * The issuer certificate of a locally looked up certificate could not be found. * The root

Re: [GENERAL] Backup Method

2015-07-08 Thread Francisco Reyes
On 07/03/2015 08:08 AM, howardn...@selestial.com wrote: I am trying to move away from pg_dump as it is proving too slow. Have you looked into barman? http://www.pgbarman.org Also, another potential approach is to setup replication and to do the backups from the slave. -- Sent via

[GENERAL] How to test SSL cert from CA?

2015-07-08 Thread Francisco Reyes
Have a client using a commercial application. For a year plus we had been using a local self signed certificate without issues. As of a few weeks ago a change/update to the program is making it complain about the self signed cert. I bought a SSL cert and installed it, but the program is still

Re: [GENERAL] MD5 password storage - should be the same everywhere?

2015-05-26 Thread Francisco Reyes
On 05/25/2015 07:58 PM, Adrian Klaver wrote: On 05/25/2015 01:41 PM, Francisco Reyes wrote: I understood that is just a md5 hash of the password and the username with the string md5 pre-appended, so it should be the same. Mistery solved.. Because I usually do script of most of my work

[GENERAL] MD5 password storage - should be the same everywhere?

2015-05-25 Thread Francisco Reyes
Should the same password, stored in MD5, be the same across different DBs? If I did either: create user SomeUser encrypted password 'SomePassword'; alter user SomeUser encrypted password 'SomePassword'; On multiple machines, should the MD5 be the same? using select rolname,

Re: [GENERAL] Kill -9 for a session process caused all the sessions to be killed

2010-09-22 Thread Francisco Reyes
atul.g...@globaldatapoint.com writes: to be killed? What is the best way to kill a session in Postgres? Just plain kill would have worked better. I believe the issue is with the '-9'. Also see http://www.postgresonline.com/journal/index.php?/archives/134-Terminating-An

[GENERAL] Tables with autovacuum_enabled=false still getting vacuumed

2010-09-19 Thread Francisco Reyes
Is there a way to see the status of autovacuum_enabled for a table? The reason I am asking is because I am seeing tables getting vacuumed which I had set autovacuum_enabled to false. The autovacuum was already running when I set the setting for a large number of tables. If I had 5 tables a b

Re: [GENERAL] Select max(primary_key) taking a long time

2010-05-22 Thread Francisco Reyes
Merlin Moncure writes: do a big delete recently? any other open transactions? Some inserts were taking place. Roughly 2 to 5 million rows inside transactions. We were doing some ETL and each batch represented a file we were loading. We need to have the entire file or roll back so each file

[GENERAL] Select max(primary_key) taking a long time

2010-05-21 Thread Francisco Reyes
Postgres 8.4.1 CentOS 5.4 I am trying to do select max(primary_key) from some_table; The explain looks like: explain select max(primary_key) from some_table; QUERY PLAN

Re: [GENERAL] Select max(primary_key) taking a long time

2010-05-21 Thread Francisco Reyes
Tom Lane writes: Francisco Reyes li...@stringsutils.com writes: I am trying to do select max(primary_key) from some_table; Are there a whole lot of nulls in that column? Zero nulls. It is a primary key. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes

[GENERAL] Alter index .. set tablespace uses ACCESS EXCLUSIVE lock?

2010-02-11 Thread Francisco Reyes
The alter index page does not show the lock mode, but it seems it is an ACCESS EXCLUSIVE. Wouldn't an EXCLUSIVE lock be more appropriate and remove the index from planner consideration? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your

Re: [GENERAL] how much left for restore?

2010-01-11 Thread Francisco Reyes
Ivan Sergio Borgonovo writes: Is there a way to know/estimate how much is left to complete a restore? Not sure on plain ASCII files but if your pg_dump used Fc then at restore you can pass the -v flag. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes

[GENERAL] Using a lock to avoid: could not open relation with OID

2010-01-11 Thread Francisco Reyes
I need to replace a table with a new one. Example.. I create a script that continously does selects like select count(*) from tmp_deleteme_francisco; enough selects to last the duration of second script select count(*) from tmp_deleteme_francisco; Another script then does begin; select *

Re: [GENERAL] how much left for restore?

2010-01-11 Thread Francisco Reyes
Ivan Sergio Borgonovo writes: It get a bit better but even knowing what are the largest tables it is hard to get an estimate of how much is missing before complete restore. Agree. Also building indexes can also take quite some time. I'm really looking at rough figures... even a: I've read

Re: [GENERAL] Database size

2010-01-11 Thread Francisco Reyes
Leonardo M. Ramé writes: A customer of mine asked me to periodically delete old, unneeded records containing ByteA fields, because he think it will reduce the database size on disk. Is this true?. No. For example, in Firebird, the database size is not reduced until you do a Backup-Restore

Re: [GENERAL] how much left for restore?

2010-01-11 Thread Francisco Reyes
Sam Mason writes: maybe something like pv would help? http://www.ivarch.com/programs/pv.shtml I think it may help the OP, but indexes are still going to be a rough spot.. if large table has several indexes the output from pv will be missleading. -- Sent via pgsql-general mailing list

Re: [GENERAL] Using a lock to avoid: could not open relation with OID

2010-01-11 Thread Francisco Reyes
Craig Ringer writes: Possible workaround: Instead of your table creation, renaming and dropping, use TRUNCATE. Have to support both 8.1 and 8.4. If I recall correctly 8.1 did not support truncate inside of a transaction. We are in the process of upgrading everything to 8.4, but until then..

Re: [GENERAL] Not finding RPMs for 8.4.2!

2009-12-22 Thread Francisco Reyes
A. Kretschmer writes: http://yum.pgsqlrpms.org/reporpms/repoview/letter_p.group.html Only up to 8.4.1 there. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Not finding RPMs for 8.4.2!

2009-12-22 Thread Francisco Reyes
They also used to be at ftp.postgresql.org:/pub/binary. There are only rpms up to 8.4.1 -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Not finding RPMs for 8.4.2!

2009-12-22 Thread Francisco Reyes
Devrim GÜNDÜZ writes: On Tue, 2009-12-22 at 15:10 -0500, Francisco Reyes wrote: Only up to 8.4.1 there. Which os/distro are you looking for? I hope it is not Fedora 11 - x86_64. CentOS x86_64 -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your

[GENERAL] Default listing on -Fc files

2009-09-16 Thread Francisco Reyes
Is there a way to tell to which table a DEFAULT rule belongs to? Example: Have multiple files with serial. The sequence is setup as a default. Because all tables have the same structure the DEFAULT lines on a file pg_dumped with -Fc look the same. The lines look like DEFAULT public id user

Re: [GENERAL] is a 'pairwise' possible / feasible in SQL?

2008-08-04 Thread Francisco Reyes
On 2:08 pm 08/04/08 Rajarshi Guha [EMAIL PROTECTED] wrote: paircount - - 123 456 1 667 879 2 create temp table aic_cid ( id smallint, cid smallint ); insert into aic_cid values (1,123); insert into aic_cid values (2,456); insert into aic_cid values (3,667);

Re: [GENERAL] non-WAL btree?

2008-08-01 Thread Francisco Reyes
On 4:36 pm 08/01/08 Alex Vinogradovs [EMAIL PROTECTED] wrote: It's not that I expect a lot of improvement by having non-WAL indexing Have you tried using a fill factor less than 90%? That is on my list of things to test, but have not done yet.. In particular you need to find a balance where the

Re: [GENERAL] Partitioned tables and views

2008-07-31 Thread Francisco Reyes
On 12:54 pm 07/31/08 Mike Gould [EMAIL PROTECTED] wrote: 1. Is the planner/optimizer intelligent enough to know when we are not doing a query based on location? In short yes. If the DB doesn't see the condition by which your tables are partitioned it will search all the partitions. 2. How

Re: [GENERAL] Problem running script

2008-07-30 Thread Francisco Reyes
On 9:50 am 07/30/08 Ian Lea [EMAIL PROTECTED] wrote: $ psql -whatever test.sql test.log Also psql -f filename test.log Almost the same as above.. =\i test.sql \o test That would be wrong. Say you had test.sql select * from mytable; You can change test.sql to: \o test.log select * from

Re: [GENERAL] Connecting to an existing transaction state.

2008-07-30 Thread Francisco Reyes
On 10:37 am 07/28/08 Alex Gen [EMAIL PROTECTED] wrote: 3. Using m01-s2.sql I would like to execute a certain SQL statement – BUT within the scope of the transaction begun by m01-s1.sql. I believe that is not possible. Specially if you are within a transaction on each of the scripts. As far as

Re: [GENERAL] Error 10061

2008-07-30 Thread Francisco Reyes
On 1:51 pm 07/30/08 Ivan Garro [EMAIL PROTECTED] wrote: Hola Gente, les comento lo que me pasa, Esta lista es solo para Ingles. Favor subscribirte a la lista en espan~ol en: http://archives.postgresql.org/pgsql-es-ayuda -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org)

[GENERAL] Index creation and maintenance_work_mem

2008-07-29 Thread Francisco Reyes
Besides maintenance_work_mem, what else can be changed to improve index creation? I just did two tests. One with maintenance_work_mem=128MB and another with maintenance_work_mem=1GB. Out of 3 single column index, 2 took slightly longer with the higher value and a third took almost the same. 12GB

Re: [GENERAL] Index creation and maintenance_work_mem

2008-07-29 Thread Francisco Reyes
On 2:53 pm 07/29/08 Alan Hodgson [EMAIL PROTECTED] wrote: --sar 2 30 Linux 2.6.9-42.ELsmp (trans03) 07/29/2008 12:58:09 PM CPU %user %nice %system %iowait %idle 12:58:11 PM all 12.44 0.00 0.06 0.00 87.50 12:58:13 PM all 12.44

Re: [GENERAL] Index creation and maintenance_work_mem

2008-07-29 Thread Francisco Reyes
On 3:19 pm 07/29/08 Alan Hodgson [EMAIL PROTECTED] wrote: sure you only have 4 cores? Hyper-threading? It seems Hyperthreading is on in that machine. Going to see if I can have it turned off. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your

Re: [GENERAL] Clone a database to other machine

2008-07-28 Thread Francisco Reyes
On 12:44 pm 07/28/08 Garg, Manjit [EMAIL PROTECTED] wrote: I'm stuck to an issue while cloning the pgsql database, can you please help, or give any docs to help out. What is the issue? Query - Trying to have same database on two seprate linux servers. Have you been able to pg_dump from

Re: [GENERAL] Date index not used when selecting a date range

2008-07-28 Thread Francisco Reyes
On 9:09 am 07/28/08 Poul Møller Hansen [EMAIL PROTECTED] wrote: But when selecting a date range I get this explain SELECT * FROM public.stat WHERE node = '1010101010' AND (date = '2008-06-30'::date AND date = '2008-01-01'::date) Bitmap Heap Scan on stat (cost=710.14..179319.44 rows=39174

Re: [GENERAL] Clone a database to other machine

2008-07-28 Thread Francisco Reyes
On 1:32 pm 07/28/08 Garg, Manjit [EMAIL PROTECTED] wrote: But, actually I want to keep both the Databse in Sync. I want clone db to get the data from Master in certain intervals. Sounds like you are looking for replication. Check

Re: [GENERAL] I often have to do update if exist, else insert, is my database design wrong?

2008-07-25 Thread Francisco Reyes
Is this normal or are there something else I could do so I don't have to check if it exists? I would say that it is normal. For the developers: a combined insert/update command would be nice :-) Mysql has such a beast along a some other non SQL compliant extensions. One possible approach

Re: [GENERAL] PostgreSQL vs FreeBSD 7.0 as regular user

2008-07-25 Thread Francisco Reyes
On 4:53 pm 07/25/08 Zoltan Boszormenyi [EMAIL PROTECTED] wrote: I don't know. How to determine? Running this as my own user: Is this your own machine or at an ISP? If it is your own machine, then most likely you are not in a jail. You would know if you were since you would have had to do it. If

[GENERAL] Any way to favor index scans, but not bitmap index scans?

2008-07-23 Thread Francisco Reyes
The data set I am working with has a very uneven distribution. I had to to set random_page_cost = 0.75 to get good plans. However, that first tries bitmap scans which perform very poorly. Is there a way to have the planner to favor index scans and disfavor bitmap scans? Is my best choice to just

Re: [GENERAL] Any way to favor index scans, but not bitmap index scans?

2008-07-23 Thread Francisco Reyes
On 12:40 pm 07/23/08 Tom Lane [EMAIL PROTECTED] wrote: That's fairly hard to believe. Care to offer some details? I will dig that actual project and run explain analyze. Will likely not have it till middle of next week though because of a monthly process starting out Friday. However, I do

Re: [GENERAL] A couple of newbie questions ...

2008-07-23 Thread Francisco Reyes
On 12:00 pm 07/23/08 Shane Ambler [EMAIL PROTECTED] wrote: INSERT INTO table (fld_y,fld_z) VALUES ('y','z') I believe that is the most common way of doing it. which is really doing: INSERT INTO table (fld_x, fld_y,fld_z) VALUES (DEFAULT, 'y','z') Correct. So either one should be fine.

Re: [GENERAL] Any way to favor index scans, but not bitmap index scans?

2008-07-23 Thread Francisco Reyes
On 2:23 pm 07/23/08 Scott Marlowe [EMAIL PROTECTED] wrote: However, I do have a current example where bitmap index scan was 3 times worse. What is your work_mem set to? For the examples that I posted it is work_mem = 64MB -- Sent via pgsql-general mailing list

Re: [GENERAL] Any way to favor index scans, but not bitmap index scans?

2008-07-23 Thread Francisco Reyes
On 3:37 pm 07/23/08 Tom Lane [EMAIL PROTECTED] wrote: Francisco Reyes [EMAIL PROTECTED] writes: SET ENABLE_SEQSCAN TO OFF; SET ENABLE_BITMAPSCAN TO OFF; Aggregate (cost=25665216.10..25665216.11 rows=1 width=12) (actual time=3088.894..3088.896 rows=1 loops=1) - Nested Loop

Re: [GENERAL] Any way to favor index scans, but not bitmap index scans?

2008-07-23 Thread Francisco Reyes
Table layouts: historical Column | Type |Modifiers ---+--+-- record_id | integer | not null default nextval('historical_record_id_seq'::regclass) f3| integer

Re: [GENERAL] Any way to favor index scans, but not bitmap index scans?

2008-07-23 Thread Francisco Reyes
On 4:12 pm 07/23/08 Francisco Reyes [EMAIL PROTECTED] wrote: Also, that plan is only 3 seconds. Minor update. A co-worker is using another DB.. and re-running my query after he did his work.. now the query using the index scans takes 2 minutes instead of 3 seconds. 3 seconds was likely data

Re: [GENERAL] Substitute a variable in PL/PGSQL.

2008-07-22 Thread Francisco Reyes
On 12:33 am 07/22/08 Steve Martin [EMAIL PROTECTED] wrote: Hi, I am trying to create a PL/PGSQL function to return the values of the fields in a record, e.g. 1 value per row in the output of the function. Are you trying to do a generic function that would work for any table or for just a

Re: [GENERAL] COPY between 7.4.x and 8.3.x

2008-07-22 Thread Francisco Reyes
On 6:01 pm 07/21/08 Jack Orenstein [EMAIL PROTECTED] wrote: to this: psql -h $SOURCE_HOST ... -c copy binary $SOURCE_SCHEMA.$SOURCE_T ABLE to stdout |\ psql ... -c copy binary $TARGET_SCHEMA.$TARGET_TABLE from stdin http://www.postgresql.org/docs/8.3/interactive/sql-copy.html The

Re: [GENERAL] How to remove duplicate lines but save one of the lines?

2008-07-21 Thread Francisco Reyes
On 11:33 am 07/21/08 A B [EMAIL PROTECTED] wrote: and I want it to be A 1 B 3 C 44 The slow way select distinct field1, field2 from sometable. The faster way select field1,fields2 from sometable group by field1, field2. Distinct should in theory be the same speed, but on several tests I

Re: [GENERAL] COPY between 7.4.x and 8.3.x

2008-07-21 Thread Francisco Reyes
On 4:05 pm 07/21/08 Jack Orenstein [EMAIL PROTECTED] wrote: We will now be adding 8.3.x databases to the mix, and will need to copy between 7.4.x and 8.3.x in both directions. The datatypes we use I believe it should work. Also, one feature I believe started in the 8.X line (8.2?), is the

Re: [GENERAL] Reducing memory usage of insert into select operations? [Solved]

2008-07-19 Thread Francisco Reyes
Martijn van Oosterhout wrote: Can you make them not deferred? How? I found the issue. I had the foreign key in the master table instead of the children. Deleted RI from master table and put into the inherited partitions. My whole 230 million rows merged in about an hour! And I even had two of

Re: [GENERAL] Reducing memory usage of insert into select operations? [Solved]

2008-07-19 Thread Francisco Reyes
Alvaro Herrera writes: Heh -- but are the FKs now checked? Try inserting something that violates the constraints and see if they are rejected. I knew it sounded too good to be true. 1- The trigger was not set in the master (ie nothing went to the children). 2- The master had no index and no

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 9:53 am 07/18/08 Douglas McNaught [EMAIL PROTECTED] wrote: dedicated database server you really don't ever want to have the OOM killer triggered. Found that yesterday (vm.overcommit_memory=2). Agree that this is better than OOM. I still ran out of memory last night and postgres just failed

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 8:13 am 07/18/08 Richard Huxton [EMAIL PROTECTED] wrote: Is the partition split done with triggers or rules? I have a single trigger+function combo that dynamically computes which partition the data has to go to. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 11:25 am 07/18/08 Richard Huxton [EMAIL PROTECTED] wrote: I'm wondering whether it's memory usage either for the trigger itself, or for the function (pl/pgsql?). Good point. If you're doing something like:INSERT INTO partitioned_table SELECT * FROM big_table then that's not only

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 11:25 am 07/18/08 Richard Huxton [EMAIL PROTECTED] wrote: Strace of the single/large process. Again, all the query is doing is insert into file select subquery The strace is pretty much a repetition of the lines below. semop(557057, 0x7fbfffdfb0, 1) = 0 lseek(100, 0, SEEK_END)

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 12:03 pm 07/18/08 Martijn van Oosterhout [EMAIL PROTECTED] wrote: Perhaps you can try reducing the shared_buffers, to see if that helps more? Will try. 8MB is quite small for workmem. More shared_buffers is not necessarily better. Ok, but from everything I had read shared_buffers of 1/4

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 12:23 pm 07/18/08 Alvaro Herrera [EMAIL PROTECTED] wrote: Do you have long-running transactions? (For example transactions that have been idle for a long time). No. The two inserts I was running were the only processes. I even did a restart to make sure there was absolutely nothing else

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 1:00 pm 07/18/08 Tom Lane [EMAIL PROTECTED] wrote: Are there any AFTER triggers (including foreign key constraints) I have two foreign key constraints. the table being inserted into? If so the list of pending trigger events might be your problem. I guess I can try disablign the foreign

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 3:55 pm 07/18/08 Tom Lane [EMAIL PROTECTED] wrote: AfterTriggerEvents: 10553909248 total in 1268 blocks; 20432 free (6 chunks); 1055316 used Well, that's definitely your problem ... So I need to remove the foreign constraints? HashBatchContext: 415227952 total in 59

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 3:55 pm 07/18/08 Tom Lane [EMAIL PROTECTED] wrote: AfterTriggerEvents: 10553909248 total in 1268 blocks; 20432 free (6 chunks); 1055316 used Well, that's definitely your problem ... What is the overhead for each AfterTriggerEvent? I guess I can write a program to process so many

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-18 Thread Francisco Reyes
On 4:55 pm 07/18/08 Tom Lane [EMAIL PROTECTED] wrote: The only thing I can think of is that you had a huge number of rows with all the same hash value, so that there wasn't any way to split the batch into smaller sections. What are the join keys exactly in this query, and what can you tell us

Re: [GENERAL] Backup/Restore of single table in multi TB database

2008-07-18 Thread Francisco Reyes
Simon Riggs wrote: Have a look at pg_snapclone. It's specifically designed to significantly improve dump times for very large objects. http://pgfoundry.org/projects/snapclone/ Also, in case the original poster is not aware, by default pg_dump allows to backup single tables. Just add -t

[GENERAL] Reducing memory usage of insert into select operations?

2008-07-17 Thread Francisco Reyes
Redhat 4 postgresql 8.3.3 Memory: 12GB While doing a couple of operations of the type insert into table select from some other table The OS triggered the out of memory killer (oom-killer). After some research and trial/error I found it was the inserts. I see one of the inserts is using up

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-17 Thread Francisco Reyes
Douglas McNaught writes: It does seem that reducing work_mem might help you, but others on this I reduced it from 256MB to 64MB. It seems it is helping. At 256MB the usage per DB connection instance was upwards of 12GB. At 64MB so far is around 7GB. I just reduced it further to 32MB and see

Re: [GENERAL] Reducing memory usage of insert into select operations?

2008-07-17 Thread Francisco Reyes
Douglas McNaught writes: Is this a 32-bit installation or 64-bit? 3GB of shared_buffers is way too big for a 32-bit setup. 64-bit. The machine has 12GB of RAM so shared-buffers is about 1/3. Dedicated DB server. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make

[GENERAL] Default fill factor for tables?

2008-07-11 Thread Francisco Reyes
While reading http://www.postgresql.org/docs/8.3/interactive/sql-cluster.html I discovered that tables can have a fill factor. However, the create table page doesn't mention what the default is. In the create index page it mentions the default for indexes is 90%. Is tables simmilar/same? --

[GENERAL] Benchmarking best practices?

2008-06-05 Thread Francisco Reyes
At work I am creating a standard postgresql benchmark suite based on the queries and operations that we commonly do. A couple of questions + Should I shutdown/restart the DB between runs? + How much bigger than memory should my tables be to have a good benchmark? One issue to keep in mind is

[GENERAL] Bizgrez dead?

2008-06-05 Thread Francisco Reyes
Don't see any activity in the project since 2006. Is that project dead? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Psql crashes with Segmentation fault on copy from

2008-05-28 Thread Francisco Reyes
On 6:28 pm 05/27/08 Tom Lane [EMAIL PROTECTED] wrote: Can you get us a stack trace from the crash? Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 182894175648 (LWP 4487)] 0x003cc31723e6 in memcpy () from /lib64/tls/libc.so.6 (gdb) bt #0 0x003cc31723e6 in

Re: [GENERAL] Psql crashes with Segmentation fault on copy from

2008-05-28 Thread Francisco Reyes
On 11:09 am 05/28/08 Tom Lane [EMAIL PROTECTED] wrote: I installed compat-postgresql-libs-debuginfo-3-2PGDG.rhel4.x86_64.rpm postgresql-debuginfo-8.2.7-1PGDG.rhel4.x86_64.rpm Do those *exactly* match the versions of the Postgres RPMs you're using? I got them from the same directory as

Re: [GENERAL] Psql crashes with Segmentation fault on copy from

2008-05-28 Thread Francisco Reyes
On 11:09 am 05/28/08 Tom Lane [EMAIL PROTECTED] wrote: Re-send. Didn't do reply-all before. Well, it would be if it were right; but PQsendQuery doesn't call PQunescapeBytea, so there's something wrong with the debug info. .. compat-postgresql-libs-debuginfo-3-2PGDG.rhel4.x86_64.rpm That

Re: [GENERAL] Psql crashes with Segmentation fault on copy from

2008-05-28 Thread Francisco Reyes
On 3:09 pm 05/28/08 Gregory Stark [EMAIL PROTECTED] wrote: Does it really have a COPY command at the beginning? Are you really doing \i data/usb_t_60M.sql or were you trying to do a copy from this file? Argh..That's it. When I re-organized the scripts I must have taken the copy command from the

[GENERAL] Psql crashes with Segmentation fault on copy from

2008-05-27 Thread Francisco Reyes
I am setting up a new machine and preparing some standard benchmark tests. While trying to load some data using copy from psql is crashing. OS Red Hat Enterprise Linux ES release 4 (Nahant Update 6) Postgresql 8.2.7 installed from RPMs I ran strace on psql and got: read(5, DAIRY QUEEN

[GENERAL] Seeking rows whit \

2008-05-18 Thread Francisco Reyes
Is there a way to search for rows with the character \? I tried variants of like and ~. create table mytable (f1 text); insert into mytable values ('Test row 1'); insert into mytable values (E'Test row 2 \\'); select * from mytable where f1 like E'%\\%'; -- returned nothing select * from

Re: [GENERAL] Seeking rows whit \

2008-05-18 Thread Francisco Reyes
Gurjeet Singh writes: Thus, writing a pattern that actually matches a literal backslash means writing four backslashes in the statement. Thanks. select * from mytable where f1 like '%%'; Worked. For the archives.. After reading section 4.1.2.2 Also found that these other one also

[GENERAL] Making sure \timing is on

2008-05-11 Thread Francisco Reyes
Is there a way to make certain \timing is on? I am looking for something simmilar to what one can do with the pager \pset pager always Most scripts I am working with (inheritted) don't have \timing in them so I can put timing on/off as needed. However some scripts already have it. So if I set

Re: [GENERAL] Making sure \timing is on

2008-05-11 Thread Francisco Reyes
Tom Lane writes: Not that I know of :-(. There's been discussion of fixing all of psql's toggle commands to offer \foo on and \foo off What would be a good starting point for someone interesting in looking into working on that? Started a job recently as PostgresSQL DBA.. so sooner or

Re: [GENERAL] Using Epoch to save timestamps in 4 bytes?

2008-05-10 Thread Francisco Reyes
Bruce Momjian writes: I am looking at tens of millions of rows, which is why my predecessor may have used integer to store epoch to save space. Our timestamp has a much larger range than a 4-byte time_t, docs say: entry4713 BC/entry entry294276 AD/entry Given that all

[GENERAL] Using Epoch to save timestamps in 4 bytes?

2008-05-08 Thread Francisco Reyes
While looking at a database I inheritted I noticed the database has tables with integers used to store epoch. I noticed that timestamp is 8 bytes and was wondering how come timestamp is 8 bytes and not 4. Is it to be able to support precission beyond a second? I am looking at tens of

[GENERAL] tcp_keepalive values not taking

2007-08-20 Thread Francisco Reyes
Postgresql 8.2.4 FreeBSD 6.2 According to http://www.postgresql.org/docs/current/static/runtime-config-connection.html the tcp_keepaliave variables take the OS setting if 0, or use whatever value one puts in postgresql.conf. I tried the following values in postgresql.conf:

Re: [GENERAL] Dealing with tsvector in fuctions for data split

2007-08-08 Thread Francisco Reyes
Francisco Reyes writes: ERROR: operator does not exist: text || tsvector Also, it'd be worth to show simplified version of your function, which demonstrates your problem. I did include that. The function looks like: AS $$ DECLARE v_sql TEXT; BEGIN v_sql := 'INSERT

Re: [GENERAL] Dealing with tsvector in fuctions for data split

2007-08-06 Thread Francisco Reyes
Tom Lane writes: This is not a particularly good way of accomplishing partitioning, as you'll be needing *every* part of your application to be explicitly aware of the exact partitioning scheme. I am trying to follow the presentation at PGcon2007 by Robert Treat. I created a master table,

[GENERAL] Dealing with tsvector in fuctions for data split

2007-08-05 Thread Francisco Reyes
I am trying to create a function to handle inserts in a database with partitions in a particular table. The function looks like: AS $$ DECLARE v_sql TEXT; BEGIN v_sql := 'INSERT INTO messagecatalog_'|| to_char(NEW.timestampfield,'')|| '(field1, field2) values ('

[GENERAL] Using result from current_database() into another command?

2007-07-24 Thread Francisco Reyes
select current_database(); current_database -- test2 Is there a way to use that result from current_database() into something like: alter database CURRENT_DATABASE() set search_path=myschema,public; I have some generic scripts that I am preparing for several databases. Most

[GENERAL] Feature request: Per database search_path

2007-07-18 Thread Francisco Reyes
As far as I know, currently one can set the search path globally, or on a per role bases. I was wondering if it could be possible to have a per database search_path. I believe this would be not only convenient, but will add flexibility. What got me thinking of this was that I manually ste a

Re: [GENERAL] Feature request: Per database search_path

2007-07-18 Thread Francisco Reyes
Richard Huxton writes: ALTER DATABASE leia SET search_path = public,lookups; Seems to work for me on 8.2 - you'll need to disconnect and reconnect to see it take place though. Hmm.. I must have typed something wrong when I tried.. For the archives.. If a user has a search path, that takes

Re: [GENERAL] pg_dump vs schemas

2007-07-16 Thread Francisco Reyes
Tom Lane writes: Shouldn't the public. be left out? I'm hesitant to remove the schema spec for fear we'd end up with underspecified output in some corner case or other. Totally understand. How about making it an option? Just like the --no-owner option. There are options that one may

  1   2   3   >