Re: [HACKERS] Commitfest problems
On 12/16/2014 4:32 AM, Simon Riggs wrote: On 15 December 2014 at 19:52, Josh Berkus j...@agliodbs.com wrote: On 12/15/2014 11:27 AM, Robert Haas wrote: I feel like we used to be better at encouraging people to participate in the CF even if they were not experts, and to do the best they can based on what they did know. That was a helpful dynamic. Sure, the reviews weren't perfect, but more people helped, and reviewing some of the patch well and some of it in a more cursory manner is way better than reviewing none of it at all. Well, it was strongly expressed to me by a number of senior contributors on this list and at the developer meeting that inexpert reviews were not really wanted, needed or helpful. As such, I stopped recruiting new reviewers (and, for that matter, doing them myself). I don't know if the same goes for anyone else. I don't remember saying that, hearing it or agreeing with it and I don't agree with it now. As a reviewer from long long ago, I can tell you that I wasn't sure I was even helpful. Things got busy at work, and I may not have been useful, and I annoyed some people on pg-general, so I walked away for a while. I have no knowledge of community-building so this might be a bad idea: Perhaps levels (or titles) of reviewer's would be helpful. A freshman reviewer is not expected to do anything useful, is expected to make mistakes, and is there to learn. The community votes and promotes them to junior (or whatever). They know they are on the right track. A junior review is useful but maybe not as complete as a senior reviewer. Maybe I'll aspire to work harder, and maybe not, but at least I know what I'm doing is useful. If I never get promoted, then I know, as well. Freshmen know its ok to make mistakes. They know who they can contact for help. I think I like belts better (yellow, green, red, black). I think this gives me two things: 1) permission to mess up 2) ability to measure myself -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] WARNING: pgstat waiting
I'm playing with 9.2devel, and I can fill my logs with: WARNING: pgstat waiting for 5001623 usec (494 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5011780 usec (495 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5021947 usec (496 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5032115 usec (497 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5042254 usec (498 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5052400 usec (499 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 I see its from here: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=79b2ee20c8a041a85dd230c4e787bef22edae57b So I was wondering if there is anything I can help with? Not sure if you need a reproducible case, or something you'd like me to try? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WARNING: pgstat waiting
On 09/10/2011 11:43 AM, Thom Brown wrote: On 10 September 2011 17:40, Andy Colsona...@squeakycode.net wrote: I'm playing with 9.2devel, and I can fill my logs with: WARNING: pgstat waiting for 5001623 usec (494 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5011780 usec (495 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5021947 usec (496 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5032115 usec (497 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5042254 usec (498 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 WARNING: pgstat waiting for 5052400 usec (499 loops), file timestamp 368986878169601 target timestamp 368986897680812 last errno 0 I see its from here: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=79b2ee20c8a041a85dd230c4e787bef22edae57b So I was wondering if there is anything I can help with? Not sure if you need a reproducible case, or something you'd like me to try? I've had exactly the same sort of messages, pages of them. But wasn't quite sure what I needed to recreate it. Just hammer a table with updates. I used: pgbench -i -s 75 then: pgbench -f bank1.sql -s 75 -c 5 -T -- bank1.sql \set nbranches :scale \set naccounts 10 * :scale \setrandom a 1 :naccounts \setrandom b 1 :naccounts \setrandom c 1 :naccounts begin; update pgbench_accounts set abalance = abalance + 1 where aid = :a; update pgbench_accounts set abalance = abalance + 1 where aid = :b; update pgbench_accounts set abalance = abalance + 1 where aid = :c; \sleep 30 ms end; This is on a desktop box, single sata drive, postgresql.conf: checkpoint_segments = 12 shared_buffers = 100MB work_mem = 5MB -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] REVIEW proposal: a validator for configuration files
On 09/10/2011 11:39 AM, Alexey Klyukin wrote: Hi Andy, On Sep 7, 2011, at 6:40 AM, Andy Colson wrote: Hi Alexey, I was taking a quick look at this patch, and have a question for ya. ... Where did the other warnings go? Its right though, line 570 is bad. It also seems to have killed the server. I have not gotten through the history of messages regarding this patch, but is it supposed to kill the server if there is a syntax error in the config file? Thank you for looking at this patch. v4 was more a what if concept that took a lot of time for somebody to look at. There were a lot of problems with it, but I think I've nailed down most of them. Attached is v5. It should fix both problems you've experienced with v4. As with the current code, the startup process gets interrupted on any error detected in the configuration file. Unlike the current code, the patch tries to report all of them before bailing out. The behavior during configuration reload has changed significantly. Instead of ignoring all changes after the first error, the code reports the problematic value and continues. It only skips applying new values completely on syntax errors and invalid configuration option names. In no cases should it bring the server down during reload. One problem I'm not sure how to address is the fact that we require 2 calls of set_config_option for each option, one to verify the new value and another to actually apply it. Normally, this function returns true for a valid value and false if it is unable to set the new value for whatever reason (either the value is invalid, or the value cannot be changed in the context of the caller). However, calling it once per value in the 'apply' mode during reload produces false for every unchanged value that can only be changed during startup (i.e. shared_buffers, or max_connections). If we ignore its return value, we'll lose the ability to detect whether invalid values were encountered during the reload and report this fact at the end of the function. I think the function should be changed, as described in my previous email (http://archives.postgresql.org/message-id/97a66029-9d3e-43af-95aa-46fe1b447...@commandprompt.com) and I'd like to hear other opinions on that. Meanwhile, due t o 2 calls to set_config_option, it currently reports all invalid values twice. If others will be opposed to changing the set_config_option, I'll fix this by removing the first, verification call and final 'errors were detected' warning to avoid 'false positives' on that (i.e. the WARNING you saw with the previous version for the valid .conf). I'd appreciate your further comments and suggestions. Thank you. -- Alexey Klyukinhttp://www.commandprompt.com The PostgreSQL Company – Command Prompt, Inc. After a quick two minute test, this patch seems to work well. It does just what I think it should. I'll add it to the commitfest page for ya. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha 1 for 9.2
On 09/10/2011 02:52 PM, Peter Eisentraut wrote: On tis, 2011-09-06 at 11:41 -0700, Josh Berkus wrote: I think the alphas have been extremely valuable for testing. That's not my recollection. Obviously, it's hard to measure this one way or the other, but I don't recall there being a lot of test reports from people who are not already contributors and could have used some other way to get the code. As a tester, I'll pull from git. I like a quick update from git pull. When I'm playing with patches, its a simple: git reset --hard patch ... I can't speak for others, but I find no benefit from a packaged alpha release. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [REVIEW] prepare plans of embedded sql on function start
Purpose Better test coverage of functions. On first call of a function, all sql statements will be prepared, even those not directly called. Think: create function test() returns void as $$ begin if false then select badcolumn from badtable; end if; end; $$ language plpgsql; At first I thought this patch would check sql on create, but that would create a dependency, which would be bad. Before, if you called this function, you'd get no error. With this patch, and with postgresql.conf settings enabled, you get: select * from test(); ERROR: relation badtable does not exist LINE 1: select badcolumn from badtable ^ QUERY: select badcolumn from badtable CONTEXT: PL/pgSQL function test line 4 at SQL statement The Patch = Applied ok, compile and make check ran ok. It seems to add/edit regression tests, but no documentation. I tried several different things I could think of, but it always found my bugs. Its disabled by default so wont cause unexpected changes. Its easy to enable, and to have individual functions exempt themselves. Performance === No penalty. At first I was concerned every function call would have overhead of extra preparing, but that is not the case. It's prepared on first call but not subsequent calls. But that made me worry that the prepare would exist too long and use old outdated stats, that as well is not a problem. I was able to setup a test where a bad index was chosen. I used two different psql sessions. In one I started a transaction, and selected from my function several times (and it was slow because it was using a bad index). In the other psql session I ran analyze on my table. Back in my first psql session, I just waited, running my function ever once and a while. Eventually it picked up the new stats and start running quick again. Code Review === I am not qualified Problems I like the idea of this patch. I think it will help catch more bugs in functions sooner. However, a function like: create function test5() returns integer as $$ begin create temp table junk(id integer); insert into junk(id) values(100); drop table temp; return 1; end; $$ language plpgsql; Will always throw an error because at prepare time, the temp junk table wont exist. This patch implements new syntax to disable the check: create function test5() returns integer as $$ #prepare_plans on_demand begin ... Was it Tom Lane that said, if we add new syntax, we have to support it forever? As a helpful feature I can see people (myself included) enabling this system wide. So what happens to all the plpgsql on pgxn that this happens to break? Well it needs updated, no problem, but the fix will be to add #prepare_plans on_demand in the magic spot. That breaks it for prior versions of PG. Is this the syntax we want? What if we add more compiler flags in the future: create function test5() returns integer as $$ #prepare_plans on_demand #disable_xlog #work_mem 10MB begin create temp table junk(id integer); insert into junk(id) values(100); drop table temp; return 1; end; $$ language plpgsql; I don't have an answer to that. Other sql implement via OPTION(...). One option I'd thought about, was to extended ANALYZE to support functions. It would require no additional plpgsql syntax changes, no postgresql.conf settings. If you wanted to prepare (prepare for a testing purpose, not a performance purpose) all the sql inside your function, youd: analyze test5(); I'd expect to get errors from that, because the junk table doesn't exist. I'd expect it, and just never analyze it. Summary === Its a tough one. I see benefit here. I can see myself using it. If I had to put my finger on it, I'm not 100% sold on the syntax. It is usable though, it does solve problems, so I'd use it. (I'm not 100% sure ANALYZE is better, either). I'm going to leave this patch as needs review, I think more eyes might be helpful. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: prepare plans of embedded sql on function start
Hi Pavel, I can get: ERROR: permission denied to set parameter plpgsql.prepare_plans with this script: set plpgsql.prepare_plans to on_start; create or replace function test1(a integer) returns integer as $$ begin return a+1; end; $$ language plpgsql; If test1() exists, then this script works fine: select * from test1(1); set plpgsql.prepare_plans to on_start; create or replace function test1(a integer) returns integer as $$ begin return a+1; end; $$ language plpgsql; -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] REVIEW Single pass vacuum - take 2
On 08/22/2011 01:22 AM, Pavan Deolasee wrote: Hi All, Here is a revised patch based on our earlier discussion. I implemented Robert's idea of tracking the vacuum generation number in the line pointer itself. For LP_DEAD line pointers, the lp_off/lp_len is unused (and always set to 0 for heap tuples). We use those 30 bits to store the generation number of the vacuum which would have potentially removed the corresponding index pointers, if the vacuum finished successfully. The pg_class information is used to know the status of the vacuum, whether it failed or succeeded. 30-bit numbers are large enough that we can ignore any wrap-around related issues. With this change, we don't need any additional header or special space in the page which was one of the main objection to the previous version. Other than this major change, I have added code commentary at relevant places and also fixed the item.h comments to reflect the change. I think the patch is ready for a serious review now. Thanks, Pavan Hi Pavan, I tried to apply your patch to git master (as of just now) and it failed. I assume that's what I should be checking out, right? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] REVIEW proposal: a validator for configuration files
Hi Alexey, I was taking a quick look at this patch, and have a question for ya. I have a default config from initdb, there is a new setting at the end but its commented out. root@storm: /db/pg92 # /etc/rc.d/postgresql start Starting PostgreSQL: root@storm: /db/pg92 # more serverlog LOG: database system was shut down at 2011-09-06 22:30:17 CDT LOG: database system is ready to accept connections LOG: autovacuum launcher started root@storm: /db/pg92 # /etc/rc.d/postgresql reload Reload PostgreSQL: No directory, logging in with HOME=/ root@storm: /db/pg92 # more serverlog LOG: database system was shut down at 2011-09-06 22:30:17 CDT LOG: database system is ready to accept connections LOG: autovacuum launcher started LOG: received SIGHUP, reloading configuration files WARNING: errors detected while parsing configuration files WARNING: errors detected while parsing configuration files WARNING: errors detected while parsing configuration files WARNING: errors detected while parsing configuration files WARNING: errors detected while parsing configuration files I didn't edit the config, it was fine at startup, so why does reload upset it so? Also, what line is the warning for? If I edit postgresql.conf and just add bob at the last line, then reload: LOG: received SIGHUP, reloading configuration files LOG: syntax error in file /db/pg92/postgresql.conf line 570, near end of line FATAL: errors detected while parsing configuration files Where did the other warnings go? Its right though, line 570 is bad. It also seems to have killed the server. I have not gotten through the history of messages regarding this patch, but is it supposed to kill the server if there is a syntax error in the config file? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] regular logging of checkpoint progress
Tomas, I cannot seem to see any of the patches you link here: https://commitfest.postgresql.org/action/patch_view?id=628 Looks like you need to take the out of the messageid. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] savepoint commit performance
This patch: https://commitfest.postgresql.org/action/patch_view?id=605 Seems to have been after thoughts, and back burner stuff, and forgotten about... Has it already been commit? http://archives.postgresql.org/pgsql-committers/2011-07/msg00206.php Oh, wait, nevermind, it was revoked and reworked: http://archives.postgresql.org/pgsql-hackers/2011-07/msg01041.php but that was posted Jul 19, 2011. And the Patch linked from commitfest is Jun 6, 2011. So is that an old patch? Or a new patch? I'm confused. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] regular logging of checkpoint progress
On 09/05/2011 12:17 PM, Andy Colson wrote: Tomas, I cannot seem to see any of the patches you link here: https://commitfest.postgresql.org/action/patch_view?id=628 Looks like you need to take the out of the messageid. -Andy This patch seems to solve the problem of going back in time to solve a problem. (need time stamped log files to see if things where slow because of checkpoint). Several people thought a view or some-non-log option would be better. Tomas replied but I need to go back in time to post diagnose a problem, and I saw no replies to that. Taking into account Noah's and Greg's Displaying accumulated autovacuum cost patch is also sending to logs, do we all now agree that this is proper way? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] remove useless ccache searching
This patch: https://commitfest.postgresql.org/action/patch_view?id=597 caches the cache because, I guess, the cache is slow. Simon asked, What is making the first cache so slow?. Pavel does not know, nor how to fix it, and nobody else responded. So my question is: is someone going to take a look at the cache? Should this be accepted as a short term fix (cuz someone will fix the cache later), long term fix (cuz the cache needs to say as-is), or not at all (because someone will fix cache right now now)? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove useless ccache searching
Pavel, I have not taken on your patch for review, but I was reading the history of it, and one question popped up: If you are allocating a new cache, what if the array is really big, will 1st cache + your cache get bigger than work_mem? (or are array op's not constrained by work_mem? Sorry, I have not used array's so not sure if there are memory limits on them) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Review: prepare plans of embedded sql on function start
Pavel, this patch: https://commitfest.postgresql.org/action/patch_view?id=624 It applied clean and compiled ok, but I cannot get it to work at all. $ psql Timing is on. psql (9.2devel) Type help for help. andy=# set plpgsql.prepare_plans to on_start; ERROR: unrecognized configuration parameter plpgsql.prepare_plans It was also really upset when I added it to my postgresql.conf file. I hate to split hairs, but the GUC having option on_start and on_demand seems weird. Most everything else is a yes/no. How'd you feel about renaming it to: prepare_plans_on_start = yes/no But really its not start (start might imply you call the function and it starts executing), its on create, so maybe: prepare_plans_on_create = yes/no -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: prepare plans of embedded sql on function start
On 09/05/2011 05:04 PM, Andrew Dunstan wrote: On 09/05/2011 05:03 PM, Andy Colson wrote: Pavel, this patch: https://commitfest.postgresql.org/action/patch_view?id=624 It applied clean and compiled ok, but I cannot get it to work at all. $ psql Timing is on. psql (9.2devel) Type help for help. andy=# set plpgsql.prepare_plans to on_start; ERROR: unrecognized configuration parameter plpgsql.prepare_plans Did you add plpgsql to custom_variable_classes? It looks like you might not have. (I'm not sure why plpgsql switch should require one, though, especially since we now load plpgsql by default. It might be better just to call it plpgsql_prepare_on_start.) cheers andrew Ah, yep, that was the problem, thank you. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: prepare plans of embedded sql on function start
On 09/05/2011 05:27 PM, Andy Colson wrote: On 09/05/2011 05:04 PM, Andrew Dunstan wrote: On 09/05/2011 05:03 PM, Andy Colson wrote: Pavel, this patch: https://commitfest.postgresql.org/action/patch_view?id=624 It applied clean and compiled ok, but I cannot get it to work at all. $ psql Timing is on. psql (9.2devel) Type help for help. andy=# set plpgsql.prepare_plans to on_start; ERROR: unrecognized configuration parameter plpgsql.prepare_plans Did you add plpgsql to custom_variable_classes? It looks like you might not have. (I'm not sure why plpgsql switch should require one, though, especially since we now load plpgsql by default. It might be better just to call it plpgsql_prepare_on_start.) cheers andrew Ah, yep, that was the problem, thank you. -Andy However I still cannot get it to work. andy=# set plpgsql.prepare_plans to on_start; SET Time: 0.123 ms andy=# show plpgsql.prepare_plans; plpgsql.prepare_plans --- on_start (1 row) andy=# create or replace function test1(a integer) returns integer as $$ andy$# begin andy$# return b+1; andy$# end; andy$# $$ language plpgsql; CREATE FUNCTION Time: 16.926 ms andy=# Oh... shoot, having gone back and read more closely I realize I didnt understand. I thought the sql would be checked on create. That's not the case. This is what I'd hopped it was: create table junk1 ( id serial, code1 integer, ); create or replace function test2() returns integer as $$ declare x integer; begin select bob into x from junk1 where id = 4; return x; end; $$ language plpgsql; I was thinking the create function would immediately return saying, unknown column bob, and not create the function. So now with the function above, this patch has not helped me at all. I wont get an error until I exec the function. Just like without the patch. I'm not so sure how helpful that is. What is you use the if false then ... end if trick to comment out some old code? You're sill going to check the tables and fields on every exec? Pavel, is there any way to move all that code to the create function? But, then that would create a dependency where there is not one now. So that would be bad. How about a new check function test2() type of call? I think having the tables/fields checked just once would be better than checking them over and over on ever single execute. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgbench internal contention
On 07/29/2011 04:00 PM, Robert Haas wrote:
On machines with lots of CPU cores, pgbench can start eating up a lot
of system time. Investigation reveals that the problem is with
random(), which glibc implements like this:
long int
__random ()
{
int32_t retval;
__libc_lock_lock (lock);
(void) __random_r (unsafe_state,retval);
__libc_lock_unlock (lock);
return retval;
}
weak_alias (__random, random)
Rather obviously, if you're running enough pgbench threads, you're
going to have a pretty ugly point of contention there. On the 32-core
machine provided by Nate Boley, with my usual 5-minute SELECT-only
test, lazy-vxid and sinval-fastmessages applied, and scale factor 100,
time shows that pgbench uses almost as much system time as user
time:
$ time pgbench -n -S -T 300 -c 64 -j 64
transaction type: SELECT only
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 64
duration: 300 s
number of transactions actually processed: 55319555
tps = 184396.016257 (including connections establishing)
tps = 184410.926840 (excluding connections establishing)
real5m0.019s
user21m10.100s
sys 17m45.480s
I patched it to use random_r() - the patch is attached - and here are
the (rather gratifying) results of that test:
$ time ./pgbench -n -S -T 300 -c 64 -j 64
transaction type: SELECT only
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 64
duration: 300 s
number of transactions actually processed: 71851589
tps = 239503.585813 (including connections establishing)
tps = 239521.816698 (excluding connections establishing)
real5m0.016s
user20m40.880s
sys 9m25.930s
Since a client-limited benchmark isn't very interesting, I think this
change makes sense. Thoughts? Objections? Coding style
improvements?
How much randomness do we really need for test data. What if it where changed
to more of a random starting point and then autoinc'd after that. Or if there
were two func's, a rand() and a next(). If your test really needs randomness
use rand(), otherwise use next(), it would be way faster, and you dont really
care what the number is anyway.
-Andy
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On 03/04/2011 10:22 PM, Robert Haas wrote: On Fri, Mar 4, 2011 at 10:44 PM, Robert Haasrobertmh...@gmail.com wrote: So it seems like the only thing that is an absolute must-do is write some release notes. Here's a rough attempt at filtering the post-alpha3 commit log down to approximately the set of things worth adding to the alpha4 release notes. Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On 03/05/2011 08:54 AM, Robert Haas wrote: On Sat, Mar 5, 2011 at 9:44 AM, Andy Colsona...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. I think you have this section twice: When an autovacuum worker... but it is doubly cool... so... :-) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)
On 03/05/2011 08:54 AM, Robert Haas wrote: On Sat, Mar 5, 2011 at 9:44 AM, Andy Colsona...@squeakycode.net wrote: Support unlogged tables. The contents of an unlogged table are WAL-logged; um.. are _not_ WAL-logged? Uh, yeah. It looks like I fixed that in the version I committed, but introduced another, similar mistake which I have now also fixed. Improved support for parallel make, make -k, and make -q Can we add a line saying -j still doesnt work, dont use it yet or make -j2 works great now. I admit I've never tried to use -j before... is this telling me its ok to use now? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Perl 5.12 complains about ecpg parser-hacking scripts
On 3/3/2011 6:49 AM, Michael Meskes wrote:
On Wed, Mar 02, 2011 at 01:33:35PM -0600, Andy Colson wrote:
I thought Kris was going to work on this, but saw no progress, and I
was bored the other day, so I started working on it.
Here is a parse.pl, with some major refactoring.
I named it with a 2 so I could run it beside the original and diff em:
Thanks for all the work.
I am sure there are new bugs. I have not run it on anything but
9.0.1. Are there other .y files you might feed it? (something other
than backend/parser/gram.y?)
I ran it against several versions and it always gave the right output. So i
decided to just commit it to the archive so we can see if it breaks anything.
The old script is still in there so in case of a major problem that I cannot
foresee we can simply change the Makefile back to using parse.pl.
Michael
And here is check_rules. Much less change, but I did run it through
perl tidy, so I'll bet most of the lines changed.
-Andy
#!/usr/bin/perl
# $PostgreSQL: pgsql/src/interfaces/ecpg/preproc/check_rules.pl,v 1.2
2010/01/02 16:58:11 momjian Exp $
# test parser generater for ecpg
# call with backend parser as stdin
#
# Copyright (c) 2009-2010, PostgreSQL Global Development Group
#
# Written by Michael Meskes mes...@postgresql.org
#
# Placed under the same license as PostgreSQL.
#
# Command line: [-v] [path only to ecpg.addons] [full filename of gram.y]
# -v enables verbose mode... show's some stats... thought it might be
interesting
#
# This script loads rule names from gram.y and sets $found{rule} = 1 for each.
# Then it checks to make sure each rule in ecpg.addons was found in gram.y
use strict;
use warnings;
no warnings 'uninitialized';
my $verbose = 0;
if ($ARGV[0] eq '-v')
{
$verbose = shift;
}
my $path = shift || '.';
my $parser = shift || '../../../backend/parser/gram.y';
my $filename = $path . /ecpg.addons;
if ($verbose)
{
print parser: $parser\n;
print addons: $filename\n;
}
my %replace_line = (
'ExecuteStmtEXECUTEnameexecute_param_clause' =
'EXECUTE prepared_name execute_param_clause execute_rest',
'ExecuteStmtCREATEOptTempTABLEcreate_as_targetASEXECUTEnameexecute_param_clause'
=
'CREATE OptTemp TABLE create_as_target AS EXECUTE prepared_name
execute_param_clause',
'PrepareStmtPREPAREnameprep_type_clauseASPreparableStmt' =
'PREPARE prepared_name prep_type_clause AS PreparableStmt'
);
my $block= '';
my $yaccmode = 0;
my $brace_indent = 0;
my (@arr, %found);
my $comment = 0;
my $non_term_id = '';
my $cc = 0;
open GRAM, $parser or die $!;
while (GRAM)
{
if (/^%%/)
{
$yaccmode++;
}
if ( $yaccmode != 1 )
{
next;
}
chomp;# strip record separator
next if ($_ eq '');
# Make sure any braces are split
s/{/ { /g;
s/}/ } /g;
# Any comments are split
s|\/\*| /* |g;
s|\*\/| */ |g;
# Now split the line into individual fields
my $n = ( @arr = split( ' ' ) );
# Go through each field in turn
for ( my $fieldIndexer = 0 ; $fieldIndexer $n ; $fieldIndexer++ )
{
if ( $arr[$fieldIndexer] eq '*/' $comment )
{
$comment = 0;
next;
}
elsif ($comment)
{
next;
}
elsif ( $arr[$fieldIndexer] eq '/*' )
{
# start of a multiline comment
$comment = 1;
next;
}
elsif ( $arr[$fieldIndexer] eq '//' )
{
next;
}
elsif ( $arr[$fieldIndexer] eq '}' )
{
$brace_indent--;
next;
}
elsif ( $arr[$fieldIndexer] eq '{' )
{
$brace_indent++;
next;
}
if ( $brace_indent 0 )
{
next;
}
if ( $arr[$fieldIndexer] eq ';' || $arr[$fieldIndexer] eq '|' )
{
$block = $non_term_id . $block;
if ( $replace_line{$block} )
{
$block = $non_term_id . $replace_line{$block};
$block =~ tr/ |//d;
}
$found{$block} = 1;
$cc++;
$block = '';
}
elsif ( ( $arr[$fieldIndexer] =~ '[A-Za-z0-9]+:' )
|| $arr[ $fieldIndexer + 1 ] eq
Re: [HACKERS] Perl 5.12 complains about ecpg parser-hacking scripts
On 1/23/2011 5:11 AM, Michael Meskes wrote: On Sat, Jan 22, 2011 at 08:40:13PM -0500, Andrew Dunstan wrote: I think these really need to be rewritten from scratch. They look like they were written by someone who never heard of Perl 5 (it's only about 16 years old). You might remember that we had this discussion before. The script was written in awk and then translated to perl by a2p. We knew that this code was less than optimal, but at least it works. As I already said when the script was introduced, I would love to have a real perl solution, but I'm not a perl programmer by any means. Michael I thought Kris was going to work on this, but saw no progress, and I was bored the other day, so I started working on it. Here is a parse.pl, with some major refactoring. I named it with a 2 so I could run it beside the original and diff em: time perl parse.pl . ../../../backend/parser/gram.y preproc.y real0m10.636s user0m8.793s sys 0m0.050s time perl parse2.pl . ../../../backend/parser/gram.y preproc2.y real0m0.289s user0m0.214s sys 0m0.009s diff preproc.y preproc2.y I am sure there are new bugs. I have not run it on anything but 9.0.1. Are there other .y files you might feed it? (something other than backend/parser/gram.y?) I touched pretty much everything, and I broke a few things and had to go back and fix 'em, so it would not surprise me at all if there were still a few bugs. Anyway, feedback, suggestions, etc are welcomed. (oh, yeah, I probably need to go add some documentation) -Andy #!/usr/bin/perl # src/interfaces/ecpg/preproc/parse.pl # parser generater for ecpg # call with backend parser as stdin # # Copyright (c) 2007-2010, PostgreSQL Global Development Group # # Written by Mike Aubury mike.aub...@aubit.com #Michael Meskes mes...@postgresql.org # # Placed under the same license as PostgreSQL. # use strict; use warnings; no warnings 'uninitialized'; my $path = shift @ARGV; $path = . unless $path; my $copymode = 0; my $brace_indent = 0; my $yaccmode = 0; my $header_included = 0; my $feature_not_supported = 0; my $tokenmode = 0; my(%buff, $infield, $comment, %tokens, %addons ); my($stmt_mode, @fields); my($line, $non_term_id); # some token have to be replaced by other symbols # either in the rule my %replace_token = ( 'BCONST' = 'ecpg_bconst', 'FCONST' = 'ecpg_fconst', 'Sconst' = 'ecpg_sconst', 'IDENT' = 'ecpg_ident', 'PARAM' = 'ecpg_param', ); # or in the block my %replace_string = ( 'WITH_TIME'= 'with time', 'NULLS_FIRST' = 'nulls first', 'NULLS_LAST' = 'nulls last', 'TYPECAST' = '::', 'DOT_DOT' = '..', 'COLON_EQUALS' = ':=', ); # specific replace_types for specific non-terminals - never include the ':' # ECPG-only replace_types are defined in ecpg-replace_types my %replace_types = ( 'PrepareStmt' = 'prep', 'opt_array_bounds' = 'index', # ignore means: do not create type and rules for this non-term-id 'stmtblock' = 'ignore', 'stmtmulti' = 'ignore', 'CreateAsStmt' = 'ignore', 'DeallocateStmt' = 'ignore', 'ColId' = 'ignore', 'type_function_name' = 'ignore', 'ColLabel' = 'ignore', 'Sconst' = 'ignore', ); # these replace_line commands excise certain keywords from the core keyword # lists. Be sure to account for these in ColLabel and related productions. my %replace_line = ( 'unreserved_keywordCONNECTION' = 'ignore', 'unreserved_keywordCURRENT_P' = 'ignore', 'unreserved_keywordDAY_P' = 'ignore', 'unreserved_keywordHOUR_P' = 'ignore', 'unreserved_keywordINPUT_P'= 'ignore', 'unreserved_keywordMINUTE_P' = 'ignore', 'unreserved_keywordMONTH_P'= 'ignore', 'unreserved_keywordSECOND_P' = 'ignore', 'unreserved_keywordYEAR_P' = 'ignore', 'col_name_keywordCHAR_P' = 'ignore', 'col_name_keywordINT_P'= 'ignore', 'col_name_keywordVALUES' = 'ignore', 'reserved_keywordTO' = 'ignore', 'reserved_keywordUNION'= 'ignore', # some other production rules have to be ignored or replaced 'fetch_argsFORWARDopt_from_incursor_name' = 'ignore', 'fetch_argsBACKWARDopt_from_incursor_name' = 'ignore', opt_array_boundsopt_array_bounds'['Iconst']' = 'ignore', 'VariableShowStmtSHOWvar_name' = 'SHOW var_name ecpg_into', 'VariableShowStmtSHOWTIMEZONE' = 'SHOW TIME ZONE ecpg_into', 'VariableShowStmtSHOWTRANSACTIONISOLATIONLEVEL' = 'SHOW TRANSACTION ISOLATION LEVEL ecpg_into', 'VariableShowStmtSHOWSESSIONAUTHORIZATION' = 'SHOW SESSION AUTHORIZATION ecpg_into',
Re: [HACKERS] Perl 5.12 complains about ecpg parser-hacking scripts
On 01/22/2011 09:28 PM, k...@shannon.id.au wrote: On 23 January 2011 13:14, Andrew Dunstanand...@dunslane.net wrote: But there are quite a few perlheads around. ISTR Matt Trout was muttering about these scripts on IRC recently. A quick cleanup of the check_rules.pl... It's a starting point at least. Oh! Perl is my favorite. Kris, if you're not going to, I'd love to work on this. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Perl 5.12 complains about ecpg parser-hacking scripts
On 01/23/2011 08:29 AM, Andy Colson wrote: On 01/22/2011 09:28 PM, k...@shannon.id.au wrote: On 23 January 2011 13:14, Andrew Dunstanand...@dunslane.net wrote: But there are quite a few perlheads around. ISTR Matt Trout was muttering about these scripts on IRC recently. Ok, so I've figured out what its purpose is. Are there other problems with this script? Does it not pull out the rule names correct all the time or something? What problem was Matt having with it? I think rewriting from scratch is overkill, unless this script is just failing. The code to pull the rule names out is a bit complex, and it seems to work, so I'd rather not touch it. Are there other things you wished this script did? (reports, counts, etc) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Perl 5.12 complains about ecpg parser-hacking scripts
On 01/23/2011 10:06 AM, Andrew Dunstan wrote: On 01/23/2011 10:16 AM, Andy Colson wrote: On 01/23/2011 08:29 AM, Andy Colson wrote: On 01/22/2011 09:28 PM, k...@shannon.id.au wrote: On 23 January 2011 13:14, Andrew Dunstanand...@dunslane.net wrote: But there are quite a few perlheads around. ISTR Matt Trout was muttering about these scripts on IRC recently. Ok, so I've figured out what its purpose is. Are there other problems with this script? Does it not pull out the rule names correct all the time or something? What problem was Matt having with it? I think rewriting from scratch is overkill, unless this script is just failing. The code to pull the rule names out is a bit complex, and it seems to work, so I'd rather not touch it. Are there other things you wished this script did? (reports, counts, etc) It's doing the right thing. But it's really spaghetti code, generated by a2p. Matt was just (rightly) offended by the $[ setting, IIRC. The point is that it's close to being totally unmaintainable. But if you think you can remedy it without rewriting it, go for it. I think minimum requirements would be: * avoid setting $[ * use strict; * comments saying what it's actually doing We want to be in a situation where of it ever does break because of some external change, we're not left having to wade through the crap to find out how to fix it. cheers andrew ** OOPS, forenote: I only noticed check_rules.pl. That's the only file I looked at. I'll go look at parse.pl right now. Below refers to check_rules.pl only. I can comment the code. $[ can safely be removed (from check_rules.pl) because we are not even using arrays. I disagree (check_rules.pl) is not spaghetti code. There are two loops, one thru gram.y to pick out the rule names, and one through ecpg.addons to make sure they are in use. Unmaintainable is another story. Parsing gram.y for rule names, then combining them to make up the same name in ecpg.addons is a little complex. But then I think it would be a little complex in any language. A little change to the syntax/layout of either file and it could go bad, but you'd have that same problem with any other language too. Is there anyway to make bison/yacc/gcc/etc spit out the rule names? I'm not sure rule name is the proper words. In gram.y we have: stmt : AlterDatabaseStmt | AlterDatabaseSetStmt We pull out and create: stmtAlterDatabaseStmt and stmtAlterDatabaseSetStmt -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] texteq/byteaeq: avoid detoast [REVIEW]
This is a review of:
https://commitfest.postgresql.org/action/patch_view?id=468
Purpose:
Equal and not-equal _may_ be quickly determined if their lengths are different.
This _may_ be a huge speed up if we dont have to detoat.
The Patch:
==
I was able to read and understand the patch, its a simple change and looked
correct to me (a non PG hacker).
It applies clean to git head, compiles and runs fine with debug enabled.
make check passes
Usability:
==
I used _may_ above. The benchmark included with the patch, showing huge speedups, is
really contrived. It uses a where clause with a thousand character constant: (where c =
'long...long...long...long...ConstantText...etc'). In my opinion this is very uncommon
(the author does note this is a best case). If you have a field large enough
to be toasted you are not going to be using that to search on, you are going to have an
ID field that is indexed. (select c where id = 7)
This also only touches = and .and like wont be touched. So I think the
scope of this is limited.
THAT being said, the patch is simple, and if you do happen to hit the code, it
will speed things up. As a user of PG I'd like to have this included. Its a
corner case, but a big corner, and its a small, simple change, and it wont slow
anything else down.
Performance:
I created myself a more real world test, with a table with indexes and id's and
a large toasted field.
create table junk(id serial primary key, xgroup integer, c text);
create index junk_group on junk(xgroup);
I filled it full of junk:
do $$
declare i integer;
declare j integer;
begin
for i in 1..100 loop
for j in 1..500 loop
insert into junk(xgroup, c) values (j, 'c'||i);
insert into junk (xgroup, c) select j, repeat('abc',
2000)|| n from generate_series(1, 5) n;
end loop;
end loop;
end$$;
This will make about 600 records within the same xgroup. As well as a simple 'c15' type
of value in c we can search for. My thinking is you may not know the exact unique id,
but you do know what group its in, so that'll cut out 90% of the records, and then you'll
have to add and c = 'c15' to get the exact one you want.
I still saw a nice performance boost.
Old PG:
$ psql bench3.sql
Timing is on.
DO
Time: 2010.412 ms
Patched:
$ psql bench3.sql
Timing is on.
DO
Time: 184.602 ms
bench3.sql:
do $$
declare i integer;
begin
for i in 1..400 loop
perform count(*) from junk where xgroup = i and c like 'c' || i;
end loop;
end$$;
Summary:
Performance speed-up: Oh yeah! If you just happen to hit it, and if you do
hit it, you might want to re-think your layout a little bit.
Do I want it? Yes please.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] reviewers needed!
I reviewed a couple patched, and I added my review to the commitfest page. If I find a problem, its obvious I should mark the patch as returned with feedback. But what if I'm happy with it? I'm not a hacker so cannot do C code review, should I leave it alone? Mark it as ready for committer? I marked my two reviews as ready for committer, but I feel like I've overstepped my bounds. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plperlu problem with utf8 [REVIEW]
On 01/16/2011 07:14 PM, Alex Hunsaker wrote: On Sat, Jan 15, 2011 at 14:20, Andy Colsona...@squeakycode.net wrote: This is a review of plperl encoding issues https://commitfest.postgresql.org/action/patch_view?id=452 Thanks for taking the time to review! [...] The Patch: == Applies clean to git head as of January 15 2011. PG built with --enable-cassert and --enable-debug seems to run fine with no errors. I don't think regression tests cover plperl, so understandable there are no tests in the patch. FWI there are plperl tests, you can do 'make installcheck' from the plperl dir or installcheck-world from the top. However I did not add any as AFAIK there is not a way to handle multiple locales with them (at least for the automated case). oh, cool. I'd kinda thought 'make check' was the one to run. I'll have to checkout 'make check' vs 'make installcheck'. There is no manual updates in the patch either, and I think there should be. I think it should be made clear that data (varchar, text, etc. but not bytea) will be passed to perl as UTF-8, regardless of database encoding I don't disagree, but I dont see where to put it either. Maybe its only release note material? I think this page: http://www.postgresql.org/docs/current/static/plperl-funcs.html Right after: Arguments and results are handled as in any other Perl subroutine: arguments are passed in @_, and a result value is returned with return or as the last expression evaluated in the function. Add: Arguments will be converted from the databases encoding to UTF-8 for use inside plperl, and then converted from UTF-8 back to the database encoding upon return. OR, that same sentence could be added to the next page: http://www.postgresql.org/docs/current/static/plperl-data.html However, this patch brings back DWIM to plperl. It should just work without having to worry about it. I'd be ok either way. Also that use utf8; is always loaded and in use. Sorry, I probably mis-worded that in my original description. Its that we always do the 'utf8fix' for plperl. Not that utf8 is loaded and in use. This fix basically makes sure the unicode database and associated modules are loaded. This is needed because perl will try to dynamically load these when you need them. As we restrict 'require' in the plperl case, things that depended on that would fail. Previously we only did the utf8fix when we were a PG_UTF8 database. I don't really think its worth documenting, its more a bug fix than anything else. Agreed. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] plperlu problem with utf8 [REVIEW]
This is a review of plperl encoding issues
https://commitfest.postgresql.org/action/patch_view?id=452
Purpose:
Your database uses one encoding, and passes data to perl in the same encoding,
which perl is not prepared for (it assumes UTF-8). This patch makes sure data
is encoded into UTF-8 before its passed to plperl then converts the response
from UTF-8 back to the database encoding for storage.
My test:
ptest2=# create database ptest2 encoding 'EUC_JP' template template0;
I created a simple perl function that reverses the string. I don't know Japanese so I
found a tattoo website that had sayings in Japanese... I picked: I am awesome.
create or replace function preverse(x text) returns text as $$
my $tmp = reverse($_[0]);
return $tmp;
$$ LANGUAGE plperl;
Before the patch:
ptest2=#select preverse('私はよだれを垂らす');
preverse
垢蕕眇鬚譴世茲呂篁
(1 row)
It is also possible to generate invalid characters. This function pulls off
the last character in the string... assuming its UTF-8
create or replace function plastchar(x text) returns text as $$
my $tmp = substr($_[0], -1);
return $tmp;
$$ LANGUAGE plperl;
ptest2=# select plastchar('私はよだれを垂らす');
ERROR: invalid byte sequence for encoding EUC_JP: 0xb9
CONTEXT: PL/Perl function plastchar
Because the string was not UTF-8, perl got confused and returned an invalid
character.
After the patch:
The exact same plperl functions work fine:
ptest2=# select preverse('私はよだれを垂らす');
preverse
すら垂をれだよは私
(1 row)
ptest2=# select plastchar('私はよだれを垂らす');
plastchar
---
す
(1 row)
Performance:
This is a bug fix, not for performance, however, as noted by the author, many
encodings will be very UTF-8'ish and the overhead will be very small. For
those encodings that would need converted, you'd need to do the same convert
inside your perl function anyway before you could use the data. The processing
has just moved from inside your perl func to inside PG.
The Patch:
==
Applies clean to git head as of January 15 2011. PG built with
--enable-cassert and --enable-debug seems to run fine with no errors.
I don't think regression tests cover plperl, so understandable there are no
tests in the patch.
There is no manual updates in the patch either, and I think there should be. I
think it should be made clear
that data (varchar, text, etc. but not bytea) will be passed to perl as UTF-8,
regardless of database encoding. Also that use utf8; is always loaded and in
use.
Code Review:
I am not qualified. Looking through the patch, I'm reminded of the old saying: Any
sufficently advanced perl XS code is indistinguishable from magic :-)
Other Remarks:
==
- Yes I know... it was a joke.
- I sure hope this posts to the news group ok
- My terminal (konsole) had a hard time displaying Japanese, so I used psql's
\i and \o to read/write files that kwrite show'd/encoded correctly via EUC_JP
Summary:
Looks good. Looks needed. Needs manual updates.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] page compression
I know its been discussed before, and one big problem is license and patent problems. Would this project be a problem: http://oldhome.schmorp.de/marc/liblzf.html -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Given the foregoing discussion, I see only two possible paths forward here. 1. Just decide that that unlogged tables can't have GIST indexes, at least until someone figures out a way to make it work. That's sort of an annoying limitation, but I think we could live with it. +1 In the small subset of situations that need unlogged tables, I would think the subset of those that need gist is exceedingly small. Unless someone can come up with a use case that needs both unlogged and gist, I'd vote not to spend time on it. (And, if ever someone does come along with a really good use, then time can be put toward it). -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
2nd) I can get the data to stick around after restart. Though not reliably. In general: create and fill a table, vacuum it (not sure if its important, I do it because thats what I'd done in my pgbench testing where I noticed the data stuck around), wait an hour (I usually left it for 12-24 hours, but recreated it with as little as a half hour), then restart pg. Sometimes the data is there... sometimes not. I also filled my table with more data than memory would hold so it would spill to disk, again, because it recreates my pgbench setup. I'm still working on finding the exact steps, but I wanted to get you #1 above. -Andy Ok, forget the time thing. Has nothing to do with it. (Which everyone already assumed I imagine). Its truncate. Create unloged table, fill it, truncate it, fill it again, restart pg, and the data will still be there. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
On 11/30/2010 10:27 PM, Robert Haas wrote: This appears as though you've somehow gotten a normal table connected to an unlogged index. That certainly sounds like a bug, but there's not enough details here to figure out what series of steps I should perform to recreate the problem. There is \h help: +1 but I can find no way of determining the tempness/unloggedness of a table via \d* It's clearly displayed in the \d output. Unlogged Table public.test Column | Type | Modifiers +-+--- a | integer | not null Indexes: test_pkey PRIMARY KEY, btree (a) Jeez... Were it a snake it'd a bit me! Ok. I blew away my database and programs, re-gitted, re-patched (they work), re-compiled (ok), and re-ran initdb. I have these non-standard settings: shared_buffers = 512MB work_mem = 5MB checkpoint_segments = 7 1st) I can recreate some warning messages from vacuum: WARNING: relation ulone page 0 is uninitialized --- fixing WARNING: relation pg_toast_16433 page 0 is uninitialized --- fixing you create an unlogged table, fill it, restart pg (and it clears the table), then fill it again, and vacuum complains. Here is a log: andy=# drop table ulone; DROP TABLE Time: 40.532 ms andy=# create unlogged table ulone(id serial, a integer, b integer, c text); NOTICE: CREATE TABLE will create implicit sequence ulone_id_seq for serial column ulone.id CREATE TABLE Time: 151.968 ms andy=# insert into ulone(a, b, c) select x, 1, 'bbb' from generate_series(1, 1000) x; INSERT 0 1000 Time: 80401.505 ms andy=# \q $ vacuumdb -az vacuumdb: vacuuming database andy vacuumdb: vacuuming database postgres vacuumdb: vacuuming database template1 $ sudo /etc/rc.d/postgresql stop Stopping PostgreSQL: No directory, logging in with HOME=/ $ sudo /etc/rc.d/postgresql start Starting PostgreSQL: $ psql Timing is on. psql (9.1devel) Type help for help. andy=# select count(*) from ulone; count --- 0 (1 row) Time: 1.164 ms andy=# insert into ulone(a, b, c) select x, 1, 'bbb' from generate_series(1, 1000) x; INSERT 0 1000 Time: 75312.753 ms andy=# \q $ vacuumdb -az vacuumdb: vacuuming database andy WARNING: relation ulone page 0 is uninitialized --- fixing WARNING: relation pg_toast_16478 page 0 is uninitialized --- fixing vacuumdb: vacuuming database postgres vacuumdb: vacuuming database template1 2nd) I can get the data to stick around after restart. Though not reliably. In general: create and fill a table, vacuum it (not sure if its important, I do it because thats what I'd done in my pgbench testing where I noticed the data stuck around), wait an hour (I usually left it for 12-24 hours, but recreated it with as little as a half hour), then restart pg. Sometimes the data is there... sometimes not. I also filled my table with more data than memory would hold so it would spill to disk, again, because it recreates my pgbench setup. I'm still working on finding the exact steps, but I wanted to get you #1 above. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] unlogged tables
I have played around a little more, and think I found a problem.
If given enough time, an unlogged table makes it to disk, and a restart wont
clear the data. If I insert a bunch of stuff, commit, and quickly restart PG,
it table is cleared. If I let it sit for a while, it stays.
Based on that, I have a pgbench_accounts table (unlogged) that after a restart
has data in it.
andy=# select aid, bid, abalance from pgbench_accounts where abalance = 3305;
aid | bid | abalance
-+-+--
3790226 | 38 | 3305
274130 | 3 | 3305
2169892 | 22 | 3305
705321 | 8 | 3305
4463145 | 45 | 3305
I dropped the index, and added a new one, then restart PG. Now it seems the
index is empty/unusable.
andy=# select aid, bid, abalance from pgbench_accounts where aid = 3790226;
aid | bid | abalance
-+-+--
(0 rows)
andy=# select pg_indexes_size('pgbench_accounts');
pg_indexes_size
-
16384
Lets recreate it:
andy=# drop index bob;
DROP INDEX
Time: 13.829 ms
andy=# create index bob on pgbench_accounts(aid, bid);
CREATE INDEX
Time: 17215.859 ms
andy=# select aid, bid, abalance from pgbench_accounts where aid = 3790226;
aid | bid | abalance
-+-+--
3790226 | 38 | 3305
(1 row)
Time: 0.712 ms
andy=# select pg_indexes_size('pgbench_accounts');
pg_indexes_size
-
179716096
I also did kill -9 on all the postgres* processes, while they were busy
inserting records, to try to corrupt the database. But could not seem to.
Setting fsync off also did not give me errors, but I assume because I was using
unlogged tables, and they were all getting cleared anyway, I never saw them.
With fsync off and normal tables, I got bad looking things in my logs and
vacuum:
LOG: unexpected pageaddr 1/AB1D6000 in log file 1, segment 187, offset 1925120
WARNING: relation access page 28184 is uninitialized --- fixing
etc...
AND last, I tried to update my git repo and see if the patches still work. They
do not.
There was much discussion on the syntax:
create unlogged table vs create temp xxx table vs something else.
There was much discussion on how persistent the tables should be. And some on
backups.
At this point, though, I find myself at an end, not sure what else to do until
the dust settles.
Oh, also, I wanted to add:
There is \h help: +1
but I can find no way of determining the tempness/unloggedness of a table
via \d*
The only way I found was to pg_dump -s
I will attempt to link this to the website, and mark it as returned to author.
-Andy
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
I have done a bunch of benchmarking. It was not easy to find consistent numbers, so I picked a job and ran the same thing over and over. I'm running Slackware 13.1 on a desktop computer. Linux storm 2.6.35.7-smp #1 SMP Sun Oct 10 21:43:07 CDT 2010 i686 AMD Athlon(tm) 7850 Dual-Core Processor AuthenticAMD GNU/Linux Database on: /dev/sda2 on /pub type ext4 (rw,noatime) I started with stock, unpatched, pg 9.1, and ran pg_bench. I used several scale's and always set the # connections at half the scale. (so scale 20 used 10 connections). I ran all tests for 180 seconds. autovacuum was always off, and I ran vacuum -z between each pg_bench. each block of numbers has these columns: scale, test 1, test 2, test 3, avg So the first line below: 6, 96, 105, 102, 101 means: pg_becnh -i -s 6 pg_bench -c 3 -T 180 vacuum -z pg_bench -c 3 -T 180 vacuum -z pg_bench -c 3 -T 180 result times for the three runs 96, 105 and 102 seconds, with average 101 seconds. The LOGS test is importing 61+ million rows of apache logs. Its a perl script, uses COPY over many many files. Each file is commit separate. checkpoint_segments = 7 shared_buffers = 512MB effective_cache_size = 1024MB autovacuum off fsync on synchronous_commit on full_page_writes on bgwriter_lru_maxpages 100 180 second tests scale, test 1, test 2, test 3, avg 6, 96, 105, 102, 101 20, 120, 82, 76, 93 40, 73, 42, 43, 53 80, 50, 29, 35, 38 synchronous_commit off 6, 239, 676, 614, 510 20, 78, 47, 56, 60 40, 59, 35, 41, 45 80, 53, 30, 35, 39 LOGS: ~ 3,900 ins/sec (I didnt record this well, its sort of a guess) synchronous_commit off full_page_writes off 6, 1273, 1344, 1287, 1301 20, 1323, 1307, 1313, 1314 40, 1051, 872, 702, 875 80, 551, 206, 245, 334 LOGS (got impatient and killed it) Total rows: 20,719,095 Total Seconds: 5,279.74 Total ins/sec: 3,924.25 fsync off synchronous_commit off full_page_writes off bgwriter_lru_maxpages 0 6, 3622, 2940, 2879, 3147 20, 2860, 2952, 2939, 2917 40, 2204, 2143, 2349, 2232 80, 1394, 1043, 1085, 1174 LOG (this is a full import) Total rows: 61,467,489 Total Seconds: 1,240.93 Total ins/sec: 49,533.37 --- Apply unlogged patches and recompile, re-initdb --- I patched pg_bench to run with either normal or unlogged tables fsync on synchronous_commit on full_page_writes on bgwriter_lru_maxpages 100 180 second tests normal tables 6, 101, 102, 108, 103 20, 110, 71, 90, 90 40, 83, 45, 49, 59 80, 50, 34, 30, 38 LOGS (partial import) Total rows: 24,754,871 Total Seconds: 6,058.03 Total ins/sec: 4,086.28 unlogged tables 6, 2966, 3047, 3007, 3006 20, 2767, 2515, 2708, 2663 40, 1933, 1311, 1464, 1569 80, 837, 552, 579, 656 LOGS (full import) Total rows: 61,467,489 Total Seconds: 1,126.75 Total ins/sec: 54,552.60 After all this... there are too many numbers for me. I have no idea what this means. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
I was able to apply and compile and run ok, creating unlogged tables seems to work as well. I patched up pgbench to optionally create unlogged tables, and ran it both ways. I get ~80tps normally, and ~1,500tps with unlogged. (Thats from memory, was playing with it last night at home) I also have a real world test I can try (import apache logs and run a few stats). What other things would be good to test: indexes? analyze/stats/plans? dump/restore? Is create temp unlogged table stuff(...) an option? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] unlogged tables
I am attempting to test this https://commitfest.postgresql.org/action/patch_view?id=424 but I'm not sure which version of PG this should be applied to. (it would be really neat, on here: https://commitfest.postgresql.org/action/commitfest_view?id=8 if there was a note that said, this test this stuff against git tag X or branch Y or whatever) I got the git: git clone git://git.postgresql.org/git/postgresql.git downloaded the patches, and applied them ok. then did ./configure and make after much spewage I got: bufmgr.c: In function 'PrefetchBuffer': bufmgr.c:126:10: error: 'struct RelationData' has no member named 'rd_istemp' make[4]: *** [bufmgr.o] Error 1 Just to make sure everything was ok with the original, I reset: git reset --hard HEAD^ ./configure make and all was well. so I tried again: make clean make maintainer-clean patch -p1 relpersistence-v1.patch .. ok .. but then... $ patch -p1 unlogged-tables-v1.patch patching file doc/src/sgml/indexam.sgml patching file doc/src/sgml/ref/create_table.sgml patching file doc/src/sgml/ref/create_table_as.sgml patching file src/backend/access/gin/gininsert.c patching file src/backend/access/gist/gist.c patching file src/backend/access/hash/hash.c patching file src/backend/access/nbtree/nbtree.c patching file src/backend/access/transam/xlog.c patching file src/backend/catalog/catalog.c patching file src/backend/catalog/heap.c patching file src/backend/catalog/index.c patching file src/backend/catalog/storage.c patching file src/backend/parser/gram.y patching file src/backend/storage/file/Makefile patching file src/backend/storage/file/copydir.c patching file src/backend/storage/file/fd.c The next patch would create the file src/backend/storage/file/reinit.c, which already exists! Assume -R? [n] That didnt happen the first time... I'm almost positive. Not sure what I should do now. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Another try at reducing repeated detoast work for PostGIS
Tom Lane wrote: Mark Cave-Ayland mark.cave-ayl...@siriusit.co.uk writes: Thanks for the patch. Fortunately enough I was able to find the dataset from the original report above, and so I've tested your patch against PostgreSQL 8.4. Unfortunately in the original test case, it doesn't seem to give the same performance improvement for me that Paul was seeing :( Huh. As far as I can see this example should traverse the same code path. I was about to ask for the dataset, but I think you might have already sent it to me once --- does this look familiar? $ tar tvfj geography.tar.bz2 -rw-r--r-- shade/shade 6444737 2008-06-06 13:33 geography.dbf -rw-r--r-- shade/shade 37179008 2008-06-06 13:33 geography.shp -rw-r--r-- shade/shade 263140 2008-06-06 13:33 geography.shx If so, what do I do with it exactly --- the file extensions convey nothing to my mind at the moment ... regards, tom lane You'll need the postgis stuff I think. use the shp2pgsql tool, like this: shp2pgsql -D -S geography geography geography.sql -D write dump format (ie COPY) -S creates simple geom's, if you get an error, remove the -S. USAGE: shp2pgsql [options] shapefile [schema.]table If you wanna see the data right from the shapefiles, you can use a tool like qgis. .dbf is regular dbase file .shp is a shapefile (esri shapefile) .shx is an index -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multicolumn index corruption on 8.4 beta 2
Floris Bos / Maxnet wrote:
The following settings differ from the defaults:
--
shared_buffers=3500MB
maintenance_work_mem = 128MB
fsync = off
synchronous_commit = off
checkpoint_segments = 25
--
==
Table layout
==
--
Table public.posts_index
Column | Type | Modifiers
++---
cid| integer| not null default
nextval('posts_index_cid
_seq'::regclass)
groupid| integer| not null
startdate | integer| not null
poster | character varying(64) | not null
basefile | character varying(64) | not null
subject| character varying(255) | not null
size | real |
nfo| boolean|
c | boolean|
parts | integer|
totalparts | integer|
imdb | integer|
ng1| boolean| default false
g2 | integer| default 0
g3 | integer| default 0
data | bytea |
Indexes:
posts_index5_pkey PRIMARY KEY, btree (cid) CLUSTER
gr_idx btree (groupid, (- cid))
pgb_idx btree (poster, groupid, basefile)
--
Only noticed problems with the pgb_idx index so far.
I have been trying to reproduce the problem but no success so far. I
made myself a table that matches yours, then I wrote a little perl
script to fill it with random data. (The script also writes out a text
file I can use to re-query things).
I fill the db, then add the indexes. Then I test lookup every record I
added, and find them all.
So, a few questions:
1) did you dump/restore into 8.4beta1 first and then upgrade the
program? Or did you dump/restore into 8.4beta2?
2) did you use any of the concurrent restore options?
3) do you do any updates or deletes to the table after you restore it?
4) do you do any other operations on the table (vacuum, cluster, etc..)?
5) got any triggers or stored procs?
6) To the -hackers: I write the records and then refind them in the
exact same order, would it be a better test to search for records in a
more random order? would it make a difference? Would searching for
some but not all make a difference?
-Andy
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Multicolumn index corruption on 8.4 beta 2
Josh Berkus wrote: Andy, 6) To the -hackers: I write the records and then refind them in the exact same order, would it be a better test to search for records in a more random order? would it make a difference? Would searching for some but not all make a difference? Are you on OpenSolaris? Can you give your script to Zdenek Jignesh to test in their enviroments? I am not, and yes I can. Hopefully I can attach a .tar.bz2 The bigtest.sh is the one to run (it runs all the parts). You'll need to edit fill.pl and test.pl and set the dbname and maybe give a username/password. In the fill.pl there is a $max variable that's used to set the number of records to insert. (its set to 10 million right now) Oh, this .tar wont create a subdirectory -Andy test.tar.bz2 Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sun blade 1000 donation
Greg Smith wrote: On Wed, 27 May 2009, andy wrote: I have a Sun blade 1000 that's just collecting dust now days...It weighs a ton. Bah, I know I picked one of those up myself once, which means it's far from being what I'd consider a heavy server as Sun hardware goes. Specs say it's 70 pounds and pulls 670W. It's a tower form factor through, right? That would make it hard to install some places. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD Yeah, when it shipped I think it was about 75 pounds. It is a tower, yes, and an impressively large box (my experience with servers is limited, this is the first I've ever gotten to play with, so it may not be out of the ordinary). I think my kill-a-watt said, at idle, it was near 300W. (Though it's been a while, I may not be remembering that correctly, and I don't recall looking at it under load) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sun blade 1000 donation
Jignesh K. Shah wrote: On 05/27/09 22:00, Josh Berkus wrote: Andy, I have a Sun blade 1000 that's just collecting dust now days. I was wondering if there were any pg-hackers that could find use for it. Its dual UltraSPARC III 750 (I think) and has two 36? gig fiber channel scsi disks. It weighs a ton. I'd be happy to donate it to a good cause. Feh, as much as we need more servers, we're really limited in our ability to accept stuff which is large high power consumption. Now, if we had a DSL line we could hook it to, I could see using it for the buildfarm; it would be interesting old HW / old Solaris for us. Actually I think you can use cutting edge OpenSolaris 2009.06 release (which will happen in less than a week) for SPARC on that hardware. I haven't tried it out on Sun Blade 1000/2000 yet but in theory you can. Refer to the following thread http://mail.opensolaris.org/pipermail/indiana-discuss/2009-February/014134.html Though you will need an Automated Installer setup to install OpenSolaris on SPARC http://dlc.sun.com/osol/docs/content/dev/AIinstall/index.html Regards, Jignesh Well that could be fun to play with. I have snv_99 on there now, so I'm not too outdated. The two drives are in a zfs mirror and as long as you use both processors its a pretty snappy box. (gmake vs gmake -j 4 is noticeably faster) But still.. I'm buying a new computer and need to clear out some of the old one's first. (I took a count, and I have about 11 computers, counting anything I can ssh to or run apache on as a computer (so my gf's iTouch counts as a computer)) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sun blade 1000 donation
Greg Smith wrote: On Thu, 28 May 2009, Andy Colson wrote: Yeah, when it shipped I think it was about 75 pounds. It is a tower, yes, and an impressively large box (my experience with servers is limited, this is the first I've ever gotten to play with, so it may not be out of the ordinary). To give you a better idea of the scale people were thinking with your original comment, the last Sun server I installed was 170 pounds and you had to provision a dedicated power outlet for it. The Blade 1000 would be considered a medium sized server. A small server is one that fits in 1 to 3 rack units. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD Sweet. That sounds fun to play on. So yeah, as I was saying before, its a 75lb box, nothing huge.. ya know... average... :-) -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sun blade 1000 donation
Josh Berkus wrote: Andy, Yeah, when it shipped I think it was about 75 pounds. It is a tower, yes, and an impressively large box (my experience with servers is limited, this is the first I've ever gotten to play with, so it may not be out of the ordinary). I think my kill-a-watt said, at idle, it was near 300W. (Though it's been a while, I may not be remembering that correctly, and I don't recall looking at it under load) Ok, that's not as bad as the spec sheet online looked. The machine is still too slow/old for benchmarking though, and we couldn't rack it (our donated rack space is limited). Does someone have a home for this machine? And would we use it for buildfarm, or for something else? I'll plug it in tonight when I get home and verify those numbers. It looks like there are a bunch of sparc build farm members, would another really be helpful? -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] benchmark farm
Jaime Casanova wrote: On 1/21/08, Andrew Dunstan [EMAIL PROTECTED] wrote: It's on my (very long) TODO list to add benchmarking as an option on the buildfarm. If you're interested in working on it then contact me offline and we will work on how to move forward. any move in this? Not much on my side. It seemed really complicated to plug into the existing run_bench.pl, which I didnt think was going to let me play with different postgres.conf settings easily. And they wanted to start with just running pg_bench, which wasn't what I was really interested in. I did write a few perl scripts to play around with inserts and updates and compared a few timings on xfs vs ext2... and then kinda lost interest. -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: Fwd: [HACKERS] PostgreSQL 8.4 development plan
Robert Treat wrote: On Saturday 09 February 2008 22:51, Christopher Browne wrote: On Feb 9, 2008 4:58 PM, Jan Wieck [EMAIL PROTECTED] wrote: I wonder if the efforts to provide mirrors for many different systems can hurt later down the road. It is pretty obvious that amost every current system has options to convert from or to mirror a CVS repository. But what if we someday really want to use something else as the master repository? Are we ready to accept losing unsupported mirrors at that time, or will that actually influence the choice (I think that it should not ... but I can hear the outcry already). The primary reason for a hue and cry to happen would require several prerequisites: 0. An SCM would be chosen to replace CVS. Let us identify it as SCM1 1. The ones hueing and crying would have chosen an SCM, SCM2, that was different from SCM1, and, furthermore, one where there isn't any tailor[1] available to permit translation of patches between them. (I'm not sure that any of the options that people are thinking about *aren't* on tailor's supported list...) 2. There is a further requirement for this lead to a hue and cry that needs to be listened to, namely that some complex and non-migratable processes have been set up that depend on SCM2. I think we can avoid this by declaring up front that its a Really Dumb Idea to set up complex processes that depend on a particular alternative SCM without the nice big fat caveat that The PGDG has not committed to migrating to any particular SCM at this time. Depend on such at your peril! Would a pre-requisite for any new SCM to be anointed as *the* new SCM that the buildfarm can be reconfigured to run with it? Unless there is an SCM2CVS option available I suppose... how many SCM's support such a thing? I dont think the buildfarm needs to require CVS. The code can be changed in the buildfarm to just run 'svn up' or 'git up and go' (sorry, never used git so I had to guess at the command :-) ) right? -Andy ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] full text search in 8.3
Hi All, You knew it was coming I have an 8.2 database that has full text searching. I tried to backup/restore it to 8.3 but got lots of errors: snip ERROR: could not access file $libdir/tsearch2: No such file or directory ERROR: function public.gtsq_in(cstring) does not exist ERROR: could not access file $libdir/tsearch2: No such file or directory ERROR: function public.gtsq_out(gtsq) does not exist ERROR: function gtsq_in(cstring) does not exist snip ERROR: type tsvector is only a shell ERROR: type public.tsdebug does not exist snip etc... I didn't really expect it to totally work, but I'm not sure how to move my db. Any pointers would be appreciated. I used the 8.3 pg_dump on my laptop to backup the 8.2 db from the server, and tried to restore on my laptop. I tried both pg_dump -Fc, and just a pg_dump. Am I going to need to backup the the data, and nothing else (pg_dump --data-only ). Will the tsvector column be ok? I tried doing a pg_dump --schema-only and restoring just that, but still got a bunch of errors (those above). If I clean that up of all the old text search stuff, and then run it, then do the data, will that work ok? One more dumb question: I don't have to enable or install anything, do I? Thanks, -Andy ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
