Re: [HACKERS] Serializable Snapshot Isolation
On 17/09/10 01:35, Kevin Grittner wrote: Heikki Linnakangas wrote: The functions are well commented, but an overview at the top of the file of all the hash tables and other data structures would be nice. What is stored in each, when are they updated, etc. I moved all the structures from predicate.h and predicate.c to a new predicate_internal.h file and added comments. You can view its current contents here: http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git;a=blob;f=src/include/storage/predicate_internal.h;h=7cdb5af6eebdc148dd5ed5030847ca50d7df4fe8;hb=7f05b21bc4d846ad22ae8c160b1bf495e254 Does this work for you? Yes, thank you, that helps a lot. So, the purpose of SerializableXidHash is to provide quick access to the SERIALIZABLEXACT struct of a top-level transaction, when you know its transaction id or any of its subtransaction ids. To implement the "or any of its subtransaction ids" part, you need to have a SERIALIZABLEXID struct for each subtransaction in shared memory. That sounds like it can eat through your shared memory very quickly if you have a lot of subtransactions. Why not use SubTransGetTopmostTransaction() ? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Use a latch to make startup process wake up and replay
On Thu, Sep 16, 2010 at 4:18 AM, Simon Riggs wrote: > We definitely have the time, so the question is, what are the best > ideas? Before advancing the review of each patch, we must determine what should be committed in 9.1, and what's in this CF. "Synchronization level on per-transaction" feature is included in Simon's patch, but not in mine. This is most important difference, which would have wide-reaching impact on the implementation, e.g., protocol between walsender and walreceiver. So, at first we should determine whether we'll commit the feature in 9.1. Then we need to determine how far we should implement in this CF. Thought? Each patch provides "synchronization level on per-standby" feature. In Simon's patch, that level is specified in the standbys's recovery.conf. In mine, it's in the master's standbys.conf. I think that the former is simpler. But if we support the capability to register the standbys, the latter would be required. Which is the best? Simon's patch seems to include simple quorum commit feature (correct me if I'm wrong). That is, when there are multiple synchronous standbys, the master waits until ACK has arrived from at least one standby. OTOH, in my patch, the master waits until ACK has arrived from all the synchronous standbys. Which should we choose? I think that we should commit my straightforward approach first, and enable the quorum commit on that. Thought? Simon proposes to invoke walwriter in the standby. This is not included in my patch, but looks good idea. ISTM that this is not essential feature for synchronous replication, so how about detachmenting of the walwriter part from the patch and reviewing it independently? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autonomous transactions
Robert Haas wrote: On Thu, Sep 16, 2010 at 5:19 AM, Dimitri Fontaine wrote: I think they call that dynamic scope, in advanced programming language. I guess that's calling for a quote of Greenspun's Tenth Rule: Any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. So the name of the game could be to find out a way to implement (a limited form of) dynamic scoping in PostgreSQL, in C, then find out all and any backend local variable that needs that to support autonomous transactions, then make it happen… Right? Interestingly, PostgreSQL was originally written in LISP, and there are remnants of that in the code today; for example, our heavy use of List nodes. But I don't think that has much to do with this project. I plan to reserve judgment on the best way of managing the relevant state until such time as someone has gone to the trouble of identifying what state that is. It would probably do Pg some good to try and recapture its functional language roots where reasonably possible. I believe that, design-wise, functional languages really are the best way to do object-relational databases, given that pure functions and immutable data structures are typically the best way to express anything one would do with them. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bad variable subst after "AS"
Thank you to the 4 people who replied. Heikki Linnakangas wrote: It's a known misfeature, PL/pgSQL isn't very smart about replacing variables with parameter markers. The good news is that this has been completely rewritten in 9.0. The above will work in 9.0. If you can't upgrade to 9.0, you'll have to rename the variable or use a different alias in the AS clause. Good to hear that this is fixed in 9.0; and I do intend to upgrade any week now. Andrew Dunstan wrote: Remove the AS clause. You don't need it here at all. Okay, that seems to be the best workaround while running under 8.4; or I would use something like _a1 instead for documentation purposes or for referencing. Alvaro Herrera wrote: Meanwhile, what is the best way to write f to work around this misbehavior? Give the column a different alias, one not colliding with a variable name. In this circumstance, the whole point of the AS clause is, because I was declaring in the function signature that it exported a table with a column named a1, I used the AS clause to make sure the selected column was named a1, else conceptually there would be a type mismatch between declared and actual result. Making them exactly the same is the whole point of the exercise. I'm in the school of thought that a table column's name is the only proper way to identify it, rather than the ordinal position being significant for identity, so even though SQL supports the latter, I consider it a misfeature of SQL that leads to error-prone code, and try to not rely on it when I can help it. The fact that Pg would make things work when the result column name is different than the declared name points squarely to ordinal position as identity, as that's the only way it could work. Considering that behavior, I agree that using a different name is reasonable under 8.4 to make this work. Pavel Stehule said: It's not a bug - just you cannot use a variable there. Table name, column names are specified in planner time, and cannot be parametrized. p.s. you can use a dynamic SQL - EXECUTE statement - RETURN QUERY EXECUTE - but it doesn't help you, because you cannot overwrite a function definition. I think you misunderstood what I was trying to do. In contrast to what you said, I was *not* expecting the a1 in "AS a1" to be treated as a variable. But no worries; Heikki/Andrew/Alvaro understood what I meant. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] security label support, revised
2010/9/14 KaiGai Kohei : > The attached patch is a revised version, but a bit difference from what > I introduced yesterday. I am working through this patch and fixing a variety of things things that seem to need fixing. Please hang tight and don't send any new versions for now. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Heartbeat between Primary and Standby replicas
On Fri, Sep 17, 2010 at 6:49 AM, fazool mein wrote: > I am designing a heartbeat system between replicas to know when a replica > goes down so that necessary measures can be taken. As I see, there are two > ways of doing it: > > 1) Creating a separate heartbeat process on replicas. > 2) Creating a heartbeat message, and sending it over the connection that is > already established between walsender and walreceiver. > > With 2, sending heartbeat from walsender to walreceiver seems trivial. > Sending a heartbeat from walreceiver to walsender seems tricky. Going > through the code, it seems that the walreceiver is always in the > PGASYNC_COPY_OUT mode (except in the beginning when handshaking is done). > > Can you recommend the right way of doing this? The existing keepalive feature doesn't help? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: SQL/MED(FDW) DDL
2010/9/15 SAKAMOTO Masahiko : > This is a proposal patch for SQL/MED for 9.1. > (1) foreign table DDL support (this proposal): > - support for foreign table DDL syntax (CREATE/ALTER FOREIGN TABLE) > - Definition of FDW routine interface and system catalogs for it. I checked the patch. It includes changes for DDL, system catalogs, information schema, and basic psql support. The patch itself have no useful works, but we need the parts anyway to support the SQL standard. I have a couples of comments: * There are unused types in the patch. They might be used by additional patches based on the patch, but should be removed for now. - enum GenericOptionFlags.ForeignTableOpt - typedef struct FSConnection FSConnection; - typedef struct FdwRoutine FdwRoutine; - typedef struct FdwReply FdwReply; * Needs an error check to SELECT FROM foreign table. It might be replaced to actual FDW routines soon, but the current error message is not ideal. postgres=# SELECT * FROM ft1; ERROR: could not open file "base/11908/16391": No such file or directory * Type checks between TABLE and FOREIGN TABLE are a bit inconsistent. For example, "ALTER TABLE ADD COLUMN" can add a column to a foreign tables but "DROP TABLE" cannot remove foreign tables. IMHO, however, we can allow such looseness because operations actually forbidden will end with ERRORs without problems. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
On 09/16/2010 08:50 PM, Tom Lane wrote: Andrew Dunstan writes: Looks like we're green on 9.0 for both MinGW and MSVC. Would you kick brown_bat too so we can check the cygwin case? Done. Looks fine. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] top-level DML under CTEs
2010/9/15 Hitoshi Harada : > 2010/9/15 Tom Lane : >> >> Well, I would think that the no-duplication rule applies to each WITH >> list separately, not both together. If you do something like >> >> with t1 as (select * from foo) >> select * from >> (with t2 as (select * from foo) >> select * from t1, t2) ss; >> > > Well, I didn't know it is allowed. That would look like the way to go. I made changes to the previous version, so that it avoids to resolve CTE name duplication. regression=# with t as (select 1 as i) insert into z with t as(select 2 as i )values ((select * from t)); INSERT 0 1 Time: 1.656 ms regression=# table z; f3 2 (1 row) Also, the sample Marko gave is OK. > CREATE TABLE foo(a int); > > WITH t AS (SELECT * FROM foo) > INSERT INTO bar > WITH RECURSIVE foo (SELECT 1 AS a) > SELECT * FROM t; > Hope this covers all the cases. Regards, -- Hitoshi Harada toplevel-dml-cte.20100917.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
Andrew Dunstan writes: > Looks like we're green on 9.0 for both MinGW and MSVC. Would you kick brown_bat too so we can check the cygwin case? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
On 09/16/2010 05:29 PM, Andrew Dunstan wrote: On 09/16/2010 04:37 PM, Magnus Hagander wrote: On Thu, Sep 16, 2010 at 19:30, Tom Lane wrote: Magnus Hagander writes: So, it's been tested by at leasdt one EDB customer with success. Do we want to sneak this in before we release 9.0.0? I think we had consensus on applying the simple fix as far back as we have the deadman switch code. If you can get it done in the next few hours, go ahead. Done. Anybody with a win32 buildfarm member - if you can give it a kick to make sure it does a run ASAP, please do so. OK, I have started MSVC/9.0 (red_bat) running. Looks like we're green on 9.0 for both MinGW and MSVC. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Server crash during simple c-language function
2010/9/16 Tomáš Kovářík : > Thanks Robert for the test on MacOS X. I wanted to be sure that it is > not only my installation issue, so I tried the same on Windows Server > 2003 with PostgreSQL 8.3; the same crash. Is there anybody who has > written a C-function on Windows platform? Maybe you should look at some of the examples in contrib, and then modify them... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Progress indication prototype
On Thu, Sep 16, 2010 at 4:57 PM, Peter Eisentraut wrote: > On tor, 2010-09-16 at 15:56 -0400, Robert Haas wrote: >> I reiterate my earlier criticism of this whole approach: it seems to >> assume that computing query progress is something inexpensive enough >> that we can afford to do it regardless of whether anyone is looking. > > That assumption appears to hold so far. It seems unlikely to hold in the general case, though, particularly if you want to do it to be accurate. The problems with database-wide VACUUM seem likely to be the tip of the iceberg. > Anyway, do you have an alternative suggestion? I think that there should be a function which returns just this one piece of data and is not automatically called as part of select * from pg_stat_activity. Then we could eventually decide to give backends a way to know if that function had been invoked on them and how recently. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Snapshot Isolation
Alvaro Herrera wrote: > Now that I look at your new patch, I noticed that I was actually > confusing relcache.h with rel.h. The latter includes a big chunk > of our headers, but relcache.h is pretty thin. Including > relcache.h in another header is not much of a problem. OK, thanks for the clarification. With the structures all brought back together in a logical order, and the new comments in front of the structure declarations, do you think a summary at the top of the file is still needed in that header file? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Snapshot Isolation
Excerpts from Kevin Grittner's message of mié sep 15 14:52:36 -0400 2010: > Alvaro Herrera wrote: > > > I think that would also solve a concern that I had, which is that > > we were starting to include relcache.h (and perhaps other headers > > as well, but that's the one that triggered it for me) a bit too > > liberally, so +1 from me. > > Unfortunately, what I proposed doesn't solve that for relcache.h, > although it does eliminate lock.h from almost everywhere and htup.h > from everywhere. Now that I look at your new patch, I noticed that I was actually confusing relcache.h with rel.h. The latter includes a big chunk of our headers, but relcache.h is pretty thin. Including relcache.h in another header is not much of a problem. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RelationCreateStorage can orphan files
Bruce Momjian writes: > Tom Lane wrote: >> This is essentially the same reason why CREATE DATABASE and related >> commands xlog directory copy operations only after completing them. >> That potentially wastes much more than a few blocks; but it's still >> non-dangerous, and far safer than the alternative. > Is this documented in a C comment somewhere? Obviously not in a place > Robert found. I had thought it was documented in the discussion of WAL logging rules in access/transam/README, but it isn't. I'll see about adding something. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RelationCreateStorage can orphan files
Tom Lane wrote: > Robert Haas writes: > > I notice that RelationCreateStorage() creates the main fork on disk > > before writing (let alone flushing) WAL. So if PG gets killed at that > > point, we end up with an orphaned file on disk. I think that we could > > even extend the relation a few times before WAL gets written, so I > > don't even think it's necessarily a zero-size file. We could perhaps > > avoid this by writing and flushing a WAL record that includes the > > creating XID before touching the disk; when we replay the record, we > > create the file but then delete it if the XID fails to commit before > > recovery ends. But I guess maybe our feeling is that it's just not > > worth taking a performance hit for this? > > That design is intentional. If the file create fails, and you've > already written a WAL record that says you created it, you are flat > out screwed. You can't even PANIC --- if you do, then the replay of > the WAL record will likely fail and PANIC again, leaving the database > dead in the water. > > Orphaned files, in contrast, are completely non-dangerous --- the worst > they can do is waste a little bit of disk space. That's a cheap price > to pay for not having an unrecoverable database after a create failure. > > This is essentially the same reason why CREATE DATABASE and related > commands xlog directory copy operations only after completing them. > That potentially wastes much more than a few blocks; but it's still > non-dangerous, and far safer than the alternative. Is this documented in a C comment somewhere? Obviously not in a place Robert found. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Snapshot Isolation
Heikki Linnakangas wrote: > The functions are well commented, but an overview at the top of > the file of all the hash tables and other data structures would be > nice. What is stored in each, when are they updated, etc. I moved all the structures from predicate.h and predicate.c to a new predicate_internal.h file and added comments. You can view its current contents here: http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git;a=blob;f=src/include/storage/predicate_internal.h;h=7cdb5af6eebdc148dd5ed5030847ca50d7df4fe8;hb=7f05b21bc4d846ad22ae8c160b1bf495e254 Does this work for you? That leaves the predicate.h file with just this: http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git;a=blob;f=src/include/storage/predicate.h;h=7dcc2af7628b860f9cec9ded6b78f55163b58934;hb=7f05b21bc4d846ad22ae8c160b1bf495e254 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git diff --patience
Kevin Grittner wrote: > I just discovered the --patience flag on the git diff command, and > I'd like to suggest that we encourage people to use it when possible > for building patches. I just looked at output with and without it > (and for good measure, before and after filterdiff --format=context > for both), and the results were much better with this switch. > > Here's a reference to the algorithm: > > http://bramcohen.livejournal.com/73318.html > > I think that page understates the benefits, though. I have seen the bracket example shown and the --patience output is clearly nicer. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] can we publish a aset interface?
On Thu, Sep 16, 2010 at 08:43:37PM +0200, Pavel Stehule wrote: > 2010/9/16 Peter Eisentraut : > > On tis, 2010-09-07 at 20:35 +0200, Pavel Stehule wrote: > >> I don't plan to try to move this module to core. And it's useless > >> - other languages has not our problems. > > > > I don't know the details of what you're struggling with, but it's > > a bit hard to believe that there is a problem that is absolutely > > unique to the Czech language. > > I think so people uses a steamer dictionary - because ispell > dictionary should be slow for any language. But there are not > available steamer for Czech language. People who need fast > processing just use a simple dictionary - and probably there are not > any pg hacker from Poland or Slovakia. I know of at least one in Poland, and I'd be amazed if there were none from Slovakia. Cheers, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Heartbeat between Primary and Standby replicas
Hello everyone, I am designing a heartbeat system between replicas to know when a replica goes down so that necessary measures can be taken. As I see, there are two ways of doing it: 1) Creating a separate heartbeat process on replicas. 2) Creating a heartbeat message, and sending it over the connection that is already established between walsender and walreceiver. With 2, sending heartbeat from walsender to walreceiver seems trivial. Sending a heartbeat from walreceiver to walsender seems tricky. Going through the code, it seems that the walreceiver is always in the PGASYNC_COPY_OUT mode (except in the beginning when handshaking is done). Can you recommend the right way of doing this? Thank you. Regards --- Postgres version = 9.0 beta-4
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
On 09/16/2010 04:37 PM, Magnus Hagander wrote: On Thu, Sep 16, 2010 at 19:30, Tom Lane wrote: Magnus Hagander writes: So, it's been tested by at leasdt one EDB customer with success. Do we want to sneak this in before we release 9.0.0? I think we had consensus on applying the simple fix as far back as we have the deadman switch code. If you can get it done in the next few hours, go ahead. Done. Anybody with a win32 buildfarm member - if you can give it a kick to make sure it does a run ASAP, please do so. OK, I have started MSVC/9.0 (red_bat) running. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Progress indication prototype
On tor, 2010-09-16 at 15:56 -0400, Robert Haas wrote: > I reiterate my earlier criticism of this whole approach: it seems to > assume that computing query progress is something inexpensive enough > that we can afford to do it regardless of whether anyone is looking. That assumption appears to hold so far. Anyway, do you have an alternative suggestion? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
On Thu, Sep 16, 2010 at 19:30, Tom Lane wrote: > Magnus Hagander writes: >> So, it's been tested by at leasdt one EDB customer with success. > >> Do we want to sneak this in before we release 9.0.0? > > I think we had consensus on applying the simple fix as far back as we > have the deadman switch code. If you can get it done in the next > few hours, go ahead. Done. Anybody with a win32 buildfarm member - if you can give it a kick to make sure it does a run ASAP, please do so. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Progress indication prototype
On Thu, Sep 16, 2010 at 2:52 PM, Peter Eisentraut wrote: >> > a very simple query. >> SELECT * FROM tbl; >> can report reasonable progress, but >> SELECT count(*) FROM tbl; >> cannot, because planned_tuple_count of the aggregation is 1. >> I hope better solutions for the grouping case because they are used >> in complex queries, where the progress counter is eagerly wanted. > > I think that's a problem for a later day. Once we have the interfaces > to report the progress, someone (else) can investigate how to track > progress of arbitrary queries. I reiterate my earlier criticism of this whole approach: it seems to assume that computing query progress is something inexpensive enough that we can afford to do it regardless of whether anyone is looking. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Progress indication prototype
On tor, 2010-09-16 at 15:47 +0900, Itagaki Takahiro wrote: > On Tue, Aug 17, 2010 at 2:19 PM, Peter Eisentraut wrote: > > VACUUM (lazy) (also autovacuum), table-rewriting ALTER TABLE > We could also support VACUUM FULL, CLUSTER, CREATE INDEX and REINDEX. Well, yeah, but those are a lot harder to do. ;-) > > a very simple query. > SELECT * FROM tbl; > can report reasonable progress, but > SELECT count(*) FROM tbl; > cannot, because planned_tuple_count of the aggregation is 1. > I hope better solutions for the grouping case because they are used > in complex queries, where the progress counter is eagerly wanted. I think that's a problem for a later day. Once we have the interfaces to report the progress, someone (else) can investigate how to track progress of arbitrary queries. > > - Are the interfaces OK? > > I like the new column in pg_stat_activity to "pull" the progress. > In addition, as previously discussed, we could also have "push" > notifications; Ex. GUC parameter "notice_per_progress" (0.0-1.0), > or periodical NOTIFY messages. That's a three-line change in pgstat_report_progress() in the simplest case. Maybe also something to consider later. > > - How to handle commands that process multiple tables? For example, > > lazy VACUUM on a single table is pretty easy to track (count the block > > numbers), but what about a database-wide lazy VACUUM? > > Not only a database-wide lazy VACUUM but also some of maintenance > commands have non-linear progress -- Progress of index scans in VACUUM > is not linear. ALTER TABLE could have REINDEX after table rewrites. > > We might need to have arbitrary knowledges for the non-uniform commands; > For example, "CREATE INDEX assigns 75% of the progress for table scan, > and 25% for the final merging of tapes". Maybe another approach is to forget about presenting progress numerically. Instead, make it a string that saying something like, for example for database-wide VACUUM, 'table 1/14 block 5/32'. That way you can cover anything you want, and you give the user the most accurate information available, but then you can't do things like sort pg_stat_activitiy by expected end time, or display a progress bar. Or of course we could do numerically and string, but that might be a bit too much clutter. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] can we publish a aset interface?
2010/9/16 Peter Eisentraut : > On tis, 2010-09-07 at 20:35 +0200, Pavel Stehule wrote: >> I don't plan to try to move this module to core. And it's useless - >> other languages has not our problems. > > I don't know the details of what you're struggling with, but it's a bit > hard to believe that there is a problem that is absolutely unique to the > Czech language. > I think so people uses a steamer dictionary - because ispell dictionary should be slow for any language. But there are not available steamer for Czech language. People who need fast processing just use a simple dictionary - and probably there are not any pg hacker from Poland or Slovakia. Regards Pavel > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Introducing Myself
Thank you. They did actually help me much. :) On Thu, Sep 16, 2010 at 7:14 PM, Kevin Grittner wrote: > Vaibhav Kaushal wrote: > > > please show me the way, at least to the terms which you people > > use when developing. > > You should probably start with this page, and follow its links: > > http://www.postgresql.org/developer/ > > > I think its time I move on to the development section of the 9 > > series of the DB. Am I right? > > There's really no such things as a "9 series". You need to > understand the version numbering better: > > http://www.postgresql.org/support/versioning > > > what happens from the moment I hit enter on PSQL after a command > > to the results being displayed > > http://developer.postgresql.org/pgdocs/postgres/index.html > http://www.postgresql.org/developer/ext.backend.html > > > where are the functions located in the source tree. > > http://www.postgresql.org/developer/ext.backend_dirs.html > > Those links should get you started > > -Kevin >
Re: [HACKERS] can we publish a aset interface?
On tis, 2010-09-07 at 20:35 +0200, Pavel Stehule wrote: > I don't plan to try to move this module to core. And it's useless - > other languages has not our problems. I don't know the details of what you're struggling with, but it's a bit hard to believe that there is a problem that is absolutely unique to the Czech language. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Basic JSON support
> I think that if we make a habit of rewriting the contributions of > first-time contributors in toto, we will have fewer second-time > contributors. I think it would have been a good idea to discuss this > on the list before you went and did it. To be fair to Itagaki-san, he DID ask about the status of the SOC project, and didn't get much of an answer. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
On Thu, Sep 16, 2010 at 1:20 PM, Markus Wanner wrote: > On 09/16/2010 04:26 PM, Robert Haas wrote: >> >> I agree. I've already said my piece on how I think that stuff would >> need to be reworked to be acceptable, so we might have to agree to >> disagree on those, especially if your goal is to get something >> committed that doesn't involve a major rewrite on your end. > > Just for clarification: you are referring to imessages and dynshmem here, > right? I agree that dynshmem needs to be reworked and rethought. And > imessages simply depends on dynshmem. Yes, I was referring to imessages and dynshmem. > If you are referring to the bgworker stuff, I'm not quite clear about what I > could do to make bgworker more acceptable. (Except perhaps for removing the > dependency on imessages). I'm not sure, either. It would be nice if there were a way to create a general facility here that we could then build various applications on, but I'm not sure whether that's the case. We had some back-and-forth about what is best for replication vs. what is best for vacuum vs. what is best for parallel query. If we could somehow conceive of a system that could serve all of those needs without introducing any more configuration complexity than what we have now, that would of course be very interesting. But judging by your comments I'm not very sure such a thing is feasible, so perhaps wait-and-see is the best approach. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
Bruce Momjian writes: > Can we assume all the mutexes will be cleaned up from a 128-exit? In the deadman-switch case I think we're safe enough. I'm not convinced at the moment that ignoring the error would be safe without that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
Magnus Hagander writes: > So, it's been tested by at leasdt one EDB customer with success. > Do we want to sneak this in before we release 9.0.0? I think we had consensus on applying the simple fix as far back as we have the deadman switch code. If you can get it done in the next few hours, go ahead. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
Dave Page wrote: > On Fri, Sep 10, 2010 at 1:45 PM, Bruce Momjian wrote: > > > > I am not sure how clear it is on Win32 that 128 is a special return > > code. > > I asked Microsoft platform support (roughly) that question. Here's the > response: I assume we are going to summarize this in a C comment when we ignore 128 return codes. Can we assume all the mutexes will be cleaned up from a 128-exit? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
Morning, On 09/16/2010 04:26 PM, Robert Haas wrote: I agree. I've already said my piece on how I think that stuff would need to be reworked to be acceptable, so we might have to agree to disagree on those, especially if your goal is to get something committed that doesn't involve a major rewrite on your end. Just for clarification: you are referring to imessages and dynshmem here, right? I agree that dynshmem needs to be reworked and rethought. And imessages simply depends on dynshmem. If you are referring to the bgworker stuff, I'm not quite clear about what I could do to make bgworker more acceptable. (Except perhaps for removing the dependency on imessages). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Server crash during simple c-language function
Thanks Robert for the test on MacOS X. I wanted to be sure that it is not only my installation issue, so I tried the same on Windows Server 2003 with PostgreSQL 8.3; the same crash. Is there anybody who has written a C-function on Windows platform? Tomas 2010/9/15 Robert Haas : > 2010/9/15 Tomáš Kovářík : >> I am trying to create a simple c-language function for "PostgreSQL >> 8.4.4, compiled by Visual C++ build 1400, 32-bit" running on Windows 7 >> (32-bit). It works, until I use a SPI. >> >> 1) CRASH: I successfully execute a simple query using SPI_exec(), but >> when getting the result, it crashes: >> SPI_getvalue(SPI_tuptable->vals[0], SPI_tuptable->tupdesc, 1); >> I tried to look wherever I could, but I have no idea what could go wrong. >> >> 2) NOT COMPILE: When I want to read value of global SPI_result, I >> cannot compile - unresolved external symbol "_SPI_result". I am >> currently linking with 'postgres.lib" and I haven't found anything >> else what to link with additionally. Compiling using Visual Studio >> 2008, C/C++ project set to compile for "C". >> >> Can anybody point out, what I am doing wrong? >> The complete code and project are below. > > I tried your example on MacOS X and it worked OK, so it's probably > some bit of Windows-specific weirdness, but unfortunately my > Windows-fu is pretty poor, so I can't speculate... > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise Postgres Company > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #5305: Postgres service stops when closing Windows session
On Wed, Sep 15, 2010 at 19:25, Robert Haas wrote: > On Wed, Sep 15, 2010 at 4:03 AM, Dave Page wrote: >> Therefore I cannot give you specific areas where this will happen. Of >> course when systems are low on resources or they are completely >> depleted (100% CPU) things will stop working > > Of course. As we all know, degrading gracefully under load is an > unachievable goal. > > Anyway, this more or less confirms what I was kind of suspecting all > along: it's hopeless to try to avoid these exit(128) events, so we > just need to look for ways to minimize the impact as much as possible > (i.e. avoid a database PANIC where possible). So, it's been tested by at leasdt one EDB customer with success. Do we want to sneak this in before we release 9.0.0? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bad variable subst after "AS"
Excerpts from Darren Duncan's message of jue sep 16 02:33:37 -0400 2010: > My impression of this is that Pg is treating the "a1" after the "AS" like it > was > a variable reference and so substituted it for $1. Yes. > Now that just seems wrong to me. I can understand either "a0" or "rv" > getting a > substitution, but something following an "AS" being substituted is just wrong. > > Is that a bug and if not then what is the rationale for working that way, and > can it be changed? Fixed in 9.0. > Meanwhile, what is the best way to write f to work around this misbehavior? Give the column a different alias, one not colliding with a variable name. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autonomous transactions
Robert Haas writes: > I plan to reserve judgment on the best way of managing the relevant > state until such time as someone has gone to the trouble of > identifying what state that is. The really fundamental problem here is that you never will be able to identify all such state. Even assuming that you successfully completed the herculean task of fixing the core backend, what of add-on code? (This is also why I'm quite unimpressed with the idea of trying to get backends to switch to a different database after startup.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
On Thu, Sep 16, 2010 at 4:47 AM, Markus Wanner wrote: > > BTW, that'd be what I call a huge patch: > > bgworkers, excluding dynshmem and imessages: > 34 files changed, 2910 insertions(+), 1421 deletions(-) > > from there to Postgres-R: > 98 files changed, 14856 insertions(+), 230 deletions(-) > Yeah, that's huge. :-) >> That >> conversation probably needs to start from the other end - is the >> overall architecture correct for us? - before we get down to specific >> patches. On the other hand, I'm very interested in laying the >> groundwork for parallel query > > Cool. Maybe we should take another look at bgworkers, as soon as a parallel > querying feature gets planned? Well, that will obviously depend somewhat on the wishes of whoever decides to work on parallel query, but it seems reasonable to me. It would be nice to get some pieces of this committed incrementally but as I say I fear there is too much dependency on what might happen later, at least the way things are structured now. >> and I think there are probably a number >> of bits of architecture both from this project and Postgres-XC, that >> could be valuable contributions to PostgreSQL; > > (...note that Postgres-R is license compatible, as opposed to the GPL'ed > Postgres-XC project...) Yeah. +1 for license compatibility. > As you can certainly imagine, it's important for me that any modification to > such a patch from Postgres-R would still be compatible to what I use it for > in Postgres-R and not cripple any functionality there, because that'd > probably create more work for me than not getting the patch accepted > upstream at all. That's an understandable goal, but it may be difficult to achieve. > As long as you don't consider imessages and dynshmem a part of Postgres-R, > they are independent of the rest of Postgres-R in the technical sense. > > And for any kind of parallel querying feature, imessages and dynshmem might > be of help as well. So I currently don't see where I could de-couple these > patches any further. I agree. I've already said my piece on how I think that stuff would need to be reworked to be acceptable, so we might have to agree to disagree on those, especially if your goal is to get something committed that doesn't involve a major rewrite on your end. > I didn't really expect them to get accepted to Postgres core at the moment. > But the Postgres team normally asks for sharing concepts and ideas as early > as possible... Absolutely. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autonomous transactions
On Thu, Sep 16, 2010 at 5:19 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> One thing that strikes me (maybe this is obvious) is that the >> execution of the main transaction and the autonomous transaction are >> not interleaved: it's a stack. So in terms of globals and stuff, >> assuming you knew which things needed to be updated, you could push >> all that stuff off to the side, do whatever with the new transaction, >> and then restore all the context afterwards. > > I think they call that dynamic scope, in advanced programming > language. I guess that's calling for a quote of Greenspun's Tenth Rule: > > Any sufficiently complicated C or Fortran program contains an ad hoc > informally-specified bug-ridden slow implementation of half of Common > Lisp. > > So the name of the game could be to find out a way to implement (a > limited form of) dynamic scoping in PostgreSQL, in C, then find out all > and any backend local variable that needs that to support autonomous > transactions, then make it happen… Right? Interestingly, PostgreSQL was originally written in LISP, and there are remnants of that in the code today; for example, our heavy use of List nodes. But I don't think that has much to do with this project. I plan to reserve judgment on the best way of managing the relevant state until such time as someone has gone to the trouble of identifying what state that is. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Basic JSON support
On Wed, Sep 15, 2010 at 9:53 PM, Itagaki Takahiro wrote: > On Thu, Sep 16, 2010 at 9:44 AM, Robert Haas wrote: >> Did you extract this from his work, or is this completely independent? >> I'm a bit disinclined to say we should just toss overboard all the >> work that's already been done. Why did you write a new patch? > > I read his patch and am inspired by the work, but the code is almost > rewritten. As my previous comment, > http://archives.postgresql.org/pgsql-hackers/2010-08/msg01773.php > I think it's not a good idea to develop JSON datatype based on the > complex node links. The point of my patch is to simplify it. > > I'd like to encourage use of the simplified structure to implement > his other works, including JSONPath. I think that if we make a habit of rewriting the contributions of first-time contributors in toto, we will have fewer second-time contributors. I think it would have been a good idea to discuss this on the list before you went and did it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: SQL/MED(FDW) DDL
On Wed, Sep 15, 2010 at 10:05:00PM -0400, Tom Lane wrote: > Mark Kirkwood writes: > > On 16/09/10 13:22, Tom Lane wrote: > >> What exactly do those get you that an ordinary index, or at worst an > >> index-organized table, doesn't get you? > > > It is pretty rare to see key value stores vs relational engines > > discussed without a descent into total foolishiness, but this Wikipedia > > page looks like a reasonable summary: > > http://en.wikipedia.org/wiki/NoSQL > > That doesn't do anything at all to answer my question. I don't want > to debate NoSQL versus traditional RDBMS here. What I asked was: > given that PG is a traditional RDBMS, what exactly are you hoping > to accomplish by putting a key-value storage mechanism in it? And > if you did, how would that be different from an index-organized table? One thing it would get is integration with existing infrastructure that makes up many critical apps. Many horizontal apps use things like memcached or redis to provide a distributed data layer for developing their applications. Sometimes that becomes the middle layer for an enterprise. Being able to hook into that middle layer is very handy. Shared login or session information is a good example of data that one might want put in a KVS. Also, many enterprises are full of different departments, orgs, teams, systems, etc ... KVS are simple and limited enough they might make a good choice for standardizing on how to share data in some places. Isn't this what SQL/Med is about? Garick > > regards, tom lane > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Introducing Myself
Vaibhav Kaushal wrote: > please show me the way, at least to the terms which you people > use when developing. You should probably start with this page, and follow its links: http://www.postgresql.org/developer/ > I think its time I move on to the development section of the 9 > series of the DB. Am I right? There's really no such things as a "9 series". You need to understand the version numbering better: http://www.postgresql.org/support/versioning > what happens from the moment I hit enter on PSQL after a command > to the results being displayed http://developer.postgresql.org/pgdocs/postgres/index.html http://www.postgresql.org/developer/ext.backend.html > where are the functions located in the source tree. http://www.postgresql.org/developer/ext.backend_dirs.html Those links should get you started -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Make CLUSTER VERBOSE more verbose
On Thu, Sep 16, 2010 at 4:49 PM, Abhijit Menon-Sen wrote: > I looked at the patch and it seems quite reasonable, but two hunks of > the changes to src/backend/commands/cluster.c don't apply cleanly. Please read the thread. The patch is intended to be applied after "sequence scan + sort for CLUSTER" patch. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autonomous transactions
Robert Haas writes: > One thing that strikes me (maybe this is obvious) is that the > execution of the main transaction and the autonomous transaction are > not interleaved: it's a stack. So in terms of globals and stuff, > assuming you knew which things needed to be updated, you could push > all that stuff off to the side, do whatever with the new transaction, > and then restore all the context afterwards. I think they call that dynamic scope, in advanced programming language. I guess that's calling for a quote of Greenspun's Tenth Rule: Any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. So the name of the game could be to find out a way to implement (a limited form of) dynamic scoping in PostgreSQL, in C, then find out all and any backend local variable that needs that to support autonomous transactions, then make it happen… Right? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO note
Hi, On 09/15/2010 07:30 PM, Robert Haas wrote: One problem with autonomous transactions is that you have to figure out where to store all the state associated with the autonomous transaction and its subtransactions. Another is that you have to avoid an unacceptable slowdown in the tuple-visibility checks in the process. It just occurs to me that this is the other potential use case for bgworkers: autonomous transactions. Simply store any kind of state in the bgworker and use one per autonomous transaction. What's left to be done: implement communication between the controlling backend (with the client connection) and the bgworker (imessages), drop the bgworker's session to user privileges (and re-raise to superuser after the job) and implement better error handling, as those would have to be propagated back to the controlling backend. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bg worker: patch 1 of 6 - permanent process
Hi, On 09/15/2010 08:54 PM, Robert Haas wrote: I think that the bar for committing to another in-core replication solution right now is probably fairly high. I'm not trying to convince you to accept the Postgres-R patch.. at least not now. BTW, that'd be what I call a huge patch: bgworkers, excluding dynshmem and imessages: 34 files changed, 2910 insertions(+), 1421 deletions(-) from there to Postgres-R: 98 files changed, 14856 insertions(+), 230 deletions(-) I am pretty doubtful that our current architecture is going to get us to the full feature set we'd eventually like to have - multi-master, partial replication, etc. Would be hard to do, due to the (physical) format of WAL, yes. That's why Postgres-R uses its own (logical) wire format. But we're not ever going to have ten replication solutions in core, so we need to think pretty carefully about what we accept. That's very understandable. That conversation probably needs to start from the other end - is the overall architecture correct for us? - before we get down to specific patches. On the other hand, I'm very interested in laying the groundwork for parallel query Cool. Maybe we should take another look at bgworkers, as soon as a parallel querying feature gets planned? and I think there are probably a number of bits of architecture both from this project and Postgres-XC, that could be valuable contributions to PostgreSQL; (...note that Postgres-R is license compatible, as opposed to the GPL'ed Postgres-XC project...) however, in neither case do I expect them to be accepted without significant modification. Sure, that's understandable as well. I've published this part of the infrastructure to get some feedback as early as possible on that part of Postgres-R. As you can certainly imagine, it's important for me that any modification to such a patch from Postgres-R would still be compatible to what I use it for in Postgres-R and not cripple any functionality there, because that'd probably create more work for me than not getting the patch accepted upstream at all. I'm saying it's hard to think about committing any of them because they aren't really independent of each other or of other parts of Postgres-R. As long as you don't consider imessages and dynshmem a part of Postgres-R, they are independent of the rest of Postgres-R in the technical sense. And for any kind of parallel querying feature, imessages and dynshmem might be of help as well. So I currently don't see where I could de-couple these patches any further. If you have a specific requirement, please don't hesitate to ask. I feel like there is an antagonistic thread to this conversation, and some others that we've had. I hope I'm misreading that, because it's not my intent to piss you off. I'm just offering my honest feedback. Your mileage may vary; others may feel differently; none of it is personal. That's absolutely fine. I'm thankful for your feedback. Also note that I initially didn't even want to add the bgworker patches to the commit fest. I've de-coupled and published these separate from Postgres-R with a) the hope to get feedback (more than for the overall Postgres-R patch) and b) to show others that such a facility exists and is ready to be reused. I didn't really expect them to get accepted to Postgres core at the moment. But the Postgres team normally asks for sharing concepts and ideas as early as possible... OK, I think I understand what you're trying to say now. I guess I feel like the ideal architecture for any sort of solution that needs a pool of workers would be to keep around the workers that most recently proved to be useful. Upon needing a new worker, you look for one that's available and already bound to the correct database. If you find one, you assign him to the new task. That's mostly how bgworkers are designed, yes. The min/max idle background worker GUCs allow a loose control over how many spare processes you want to allow hanging around doing nothing. If not, you find the one that's been idle longest and either (a) kill him off and start a new one that is bound to the correct database or, even better, (b) tell him to flush his caches and rebind to the correct database. Hm.. sorry if I didn't express this more clearly. What I'm trying to say is that (b) isn't worth implementing, because it doesn't offer enough of an improvement over (a). The only saving would be the fork() and some basic process initialization. Being able to re-use a bgworker connected to the correct database already gives you most of the benefit, namely not having to fork() *and* re-connect to the database for every job. Back at the technical issues, let me try to summarize the feedback and what I do with it. In general, there's not much use for bgworkers for just autovacuum as the only background job. I agree. Tom raised the 'lots of databases' issue. I agree that the bgworker infrastructur
[HACKERS] [REVIEW] Re: I: About "Our CLUSTER implementation is pessimal" patch
(Sorry for the broken threading. I didn't have a convenient copy of the original message to reply to.) I looked at the patch and it seems quite reasonable, but two hunks of the changes to src/backend/commands/cluster.c don't apply cleanly. I'm not sure what version the patch was generated against, but the code in copy_heap_data() seems to have changed quite a bit. I don't think it would be too much trouble to adapt the changes, though. -- ams -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Introducing Myself
Hello all, May be this is notneeded, but I thought I could just introduce myself to the list and why am I here. My name is Vaibhav Kaushal, a good leaner and a really bad student. I bunk classes, don't attend exams and love to develop. I just completed my engineering course in Information science but with a few subjects still on the list. I am not much of a developer yet but understand the basics. I use both Windows and Linux; Linux being my more preferred OS. I was trying to develop a small web application and selected MySQL because "you know its the most popular OSS DB on the Internet" and has a more friendly name (although almost crappy ways of work with awkward SQL syntax, no foreign key support by default and rules allowing silent ignores on the commands, specially the 'check' rule). But then I thought I could look for options; and I got postgresql and i had to change the whole architecture of the application! It is just too much feature packed and powerful and fast enough considering what it does. I am here to learn how it works (it has become my favorite DB almost instantly), and how to make it work even better. I do not understand everything what the documents say but I want to know. So if anyone can,*please show me the way, at least to the terms which you people use when developing*. I want to change a few things in the inner core of it. May be my additions would be beneficial to the community. But for that I need to start. Also, I think its time I move on to the development section of the 9 series of the DB. Am I right? I am new here and need some guidance. I would first like to know what it does (*what happens from the moment I hit enter on PSQL after a command to the results being displayed*) and where are the functions located in the source tree. Thanks a lot. Vaibhav (*_*)
Re: [HACKERS] bad variable subst after "AS"
On 09/16/2010 02:33 AM, Darren Duncan wrote: I don't know if this is a bug or not, but if not, it looks like a misfeature ... When executing the following in Pg 8.4.4: CREATE OR REPLACE FUNCTION f () RETURNS TABLE(a1 INTEGER) AS $BODY$ BEGIN RETURN QUERY SELECT a0 AS a1 FROM rv; RETURN; END; $BODY$ LANGUAGE plpgsql; ... I get this error: ERROR: syntax error at or near "$1" SQL state: 42601 My impression of this is that Pg is treating the "a1" after the "AS" like it was a variable reference and so substituted it for $1. Now that just seems wrong to me. I can understand either "a0" or "rv" getting a substitution, but something following an "AS" being substituted is just wrong. Is that a bug and if not then what is the rationale for working that way, and can it be changed? Meanwhile, what is the best way to write f to work around this misbehavior? Thank you. Remove the AS clause. You don't need it here at all. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
