[HACKERS] about error handling mechanism
Hi! I was looking into the postgres error handling mechanism, and the documentation states that the present mechanism is primitive. I quote whenever the parser, planner/optimizer or executor decide that a statement cannot be processed any longer, the whole transaction gets aborted and the system jumps back into the main loop to get the next command from the client application. Also, it states it is possible that the database server is in an inconsistent state at this point so returning to the upper executor or issuing more commands might corrupt the whole database Since postgres aborts the transaction compeletely, why would it be ever left in an inconsistant state in such an event? Also , I want to get someone's help about the mechanism about how it works. Could someone help me? thanks -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] patch: remove redundant code from pl_exec.c
Hello
This patch remove redundant rows from PL/pgSQL executor (-89 lines).
Doesn't change a functionality.
Regards
Pavel Stehule
*** ./src/pl/plpgsql/src/pl_exec.c.orig 2010-12-16 10:25:37.0 +0100
--- ./src/pl/plpgsql/src/pl_exec.c 2010-12-17 10:50:31.793623763 +0100
***
*** 204,209
--- 204,211
PLpgSQL_expr *dynquery, List *params,
const char *portalname, int cursorOptions);
+ static bool can_leave_loop(PLpgSQL_execstate *estate,
+ PLpgSQL_any_loop *stmt, int *rc);
/* --
* plpgsql_exec_function Called by the call handler for
***
*** 1566,1571
--- 1568,1637
return exec_stmts(estate, stmt-else_stmts);
}
+ /*
+ * function returns true, when outer cycle should be leaved
+ */
+ static bool
+ can_leave_loop(PLpgSQL_execstate *estate, PLpgSQL_any_loop *stmt, int *rc)
+ {
+ switch (*rc)
+ {
+ case PLPGSQL_RC_OK:
+ return false;
+
+ case PLPGSQL_RC_RETURN:
+ return true;
+
+ case PLPGSQL_RC_EXIT:
+ if (estate-exitlabel == NULL)
+ {
+ /* unlabelled exit, finish the current loop */
+ *rc = PLPGSQL_RC_OK;
+ }
+ if (stmt-label != NULL strcmp(stmt-label, estate-exitlabel) == 0)
+ {
+ /* labelled exit, matches the current stmt's label */
+ estate-exitlabel = NULL;
+ *rc = PLPGSQL_RC_OK;
+ }
+
+ /*
+ * otherwise, this is a labelled exit that does not match the
+ * current statement's label, if any: return RC_EXIT so that the
+ * EXIT continues to propagate up the stack.
+ */
+ return true;
+
+ case PLPGSQL_RC_CONTINUE:
+ if (estate-exitlabel == NULL)
+ {
+ /* anonymous continue, so re-run the loop */
+ *rc = PLPGSQL_RC_OK;
+ }
+ else if (stmt-label != NULL
+ strcmp(stmt-label, estate-exitlabel) == 0)
+ {
+ /* label matches named continue, so re-run loop */
+ estate-exitlabel = NULL;
+ *rc = PLPGSQL_RC_OK;
+ }
+ else
+ {
+ /*
+ * otherwise, this is a named continue that does not match the
+ * current statement's label, if any: return RC_CONTINUE so
+ * that the CONTINUE will propagate up the stack.
+ */
+ return true;
+ }
+
+ return false;
+
+ default:
+ elog(ERROR, unrecognized rc: %d, *rc);
+ return false; /* be compiler quiet */
+ }
+ }
/* --
* exec_stmt_loop Loop over statements until
***
*** 1575,1621
static int
exec_stmt_loop(PLpgSQL_execstate *estate, PLpgSQL_stmt_loop *stmt)
{
for (;;)
{
! int rc = exec_stmts(estate, stmt-body);
!
! switch (rc)
! {
! case PLPGSQL_RC_OK:
! break;
!
! case PLPGSQL_RC_EXIT:
! if (estate-exitlabel == NULL)
! return PLPGSQL_RC_OK;
! if (stmt-label == NULL)
! return PLPGSQL_RC_EXIT;
! if (strcmp(stmt-label, estate-exitlabel) != 0)
! return PLPGSQL_RC_EXIT;
! estate-exitlabel = NULL;
! return PLPGSQL_RC_OK;
!
! case PLPGSQL_RC_CONTINUE:
! if (estate-exitlabel == NULL)
! /* anonymous continue, so re-run the loop */
! break;
! else if (stmt-label != NULL
! strcmp(stmt-label, estate-exitlabel) == 0)
! /* label matches named continue, so re-run loop */
! estate-exitlabel = NULL;
! else
! /* label doesn't match named continue, so propagate upward */
! return PLPGSQL_RC_CONTINUE;
! break;
!
! case PLPGSQL_RC_RETURN:
! return rc;
! default:
! elog(ERROR, unrecognized rc: %d, rc);
! }
}
! return PLPGSQL_RC_OK;
}
--- 1641,1657
static int
exec_stmt_loop(PLpgSQL_execstate *estate, PLpgSQL_stmt_loop *stmt)
{
+ int rc;
+
for (;;)
{
! rc = exec_stmts(estate, stmt-body);
! if (can_leave_loop(estate, (PLpgSQL_any_loop *) stmt, rc))
! break;
}
! return rc;
}
***
*** 1628,1636
static int
exec_stmt_while(PLpgSQL_execstate *estate, PLpgSQL_stmt_while *stmt)
{
for (;;)
{
- int rc;
bool value;
bool isnull;
--- 1664,1673
static int
exec_stmt_while(PLpgSQL_execstate *estate, PLpgSQL_stmt_while *stmt)
{
+ int rc;
+
for (;;)
{
bool value;
bool isnull;
***
*** 1642,1684
rc = exec_stmts(estate, stmt-body);
! switch (rc)
! {
! case PLPGSQL_RC_OK:
! break;
!
! case PLPGSQL_RC_EXIT:
! if (estate-exitlabel == NULL)
! return PLPGSQL_RC_OK;
! if (stmt-label == NULL)
! return PLPGSQL_RC_EXIT;
! if (strcmp(stmt-label, estate-exitlabel) != 0)
! return PLPGSQL_RC_EXIT;
! estate-exitlabel = NULL;
! return PLPGSQL_RC_OK;
!
! case PLPGSQL_RC_CONTINUE:
! if (estate-exitlabel == NULL)
! /* anonymous continue, so re-run loop */
! break;
! else if (stmt-label != NULL
! strcmp(stmt-label, estate-exitlabel) == 0)
! /* label matches named continue, so re-run loop */
! estate-exitlabel = NULL;
! else
! /* label
Re: [HACKERS] clang and LLVM
Firebird http://mapopa.blogspot.com/2010/10/clang-compiling-successful-experiments.html Regards Alexandre Riveira Tom Lane escreveu: Gevik Babakhani pg...@xs4all.nl writes: I was wondering if there has been anyone experimenting to compile PG using LLVM/clang compiler tools. There is (or was, not sure if it's up right now) a buildfarm machine using LLVM. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
On 17/12/2010 3:46 PM, Magnus Hagander wrote: On Dec 17, 2010 8:02 AM, Craig Ringer cr...@postnewspapers.com.au mailto:cr...@postnewspapers.com.au wrote: On 16/12/10 21:01, Magnus Hagander wrote: Found another problem in it: when running with an older version of dbghelp.dll (which I was), it simply didn't work. We need to grab the version of dbghelp.dll at runtime and pick which things we're going to dump based on that. I was about to suggest dropping the fallback loading of the system dbghelp.dll, and just bundle a suitable version with Pg, then I saw how big the DLL is. It's 800kb (!!) of code that'll hopefully never get used on any given system. With a footprint like that, bundling it seems rather less attractive. I think Magnus is right: test the dbghelp.dll version and fall back to supported features - as far back as XP, anyway; who cares about anything earlier than that. An updated version can always be put in place by the user if the built-in one can't produce enough detail. Did you get a chance to test that it still produced a full dump on your system? I've just tested that and it looks ok so far. The dumps produced are smaller - 1.3MB instead of ~4MB for a dump produced by calling a simple crashme() function from SQL, the source for which follows at the end of this post. However, I'm getting reasonable stack traces, as shown by the windb.exe output below. That said, the dump appears to exclude the backend's private memory. I can't resolve *fcinfo from PG_FUNCTION_ARGS; it looks like the direct values of arguments are available as they're on the stack, as are function local variables, but the process heap isn't dumped so you can't see the contents of any pointers-to-struct. That'll make it harder to track down many kinds of error - but, on the other hand, means that dumps of processes with huge work_mem etc won't be massively bloated. That might be a safer starting point than including the private process memory, really. I'd like to offer a flag to turn on dumping of private process memory but I'm not sure how best to go about it, given that we'd want to avoid accessing GUCs etc if possible. later, I think; this is better for now. I'm happy with the patch as it stands, with the one exception that consideration of mingw is required before it can be committed. Symbol search path is: C:\Users\Craig\Developer\postgres\Release\postgres;SRV*c:\localsymbols*http://msdl.microsoft.com/download/symbols;C:\Program Files\PostgreSQL\9.0\symbols Executable search path is: Windows 7 Version 7600 MP (2 procs) Free x86 compatible Product: WinNt, suite: SingleUserTS Machine Name: Debug session time: Fri Dec 17 17:37:19.000 2010 (UTC + 8:00) System Uptime: not available Process Uptime: 0 days 0:01:09.000 This dump file has an exception of interest stored in it. The stored exception information can be accessed via .ecxr. (16b4.cd8): Access violation - code c005 (first/second chance not available) eax=02e8 ebx=03210aa8 ecx=0055e700 edx=03210a68 esi=03210a68 edi=0055ea14 eip=771464f4 esp=0055e6d4 ebp=0055e6e4 iopl=0 nv up ei pl zr na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs= efl=0246 ntdll!KiFastSystemCallRet: 771464f4 c3 ret 0:000 .ecxr eax= ebx= ecx=0103fa88 edx=7001100f esi=0103fa78 edi=0103fab4 eip=70011052 esp=0055f62c ebp=0103fa78 iopl=0 nv up ei pl zr na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs= efl=00010246 crashme!crashdump_crashme+0x2: 70011052 c7000100mov dword ptr [eax],1ds:0023:= 0:000 ~ . 0 Id: 16b4.cd8 Suspend: 0 Teb: 7ffdf000 Unfrozen 1 Id: 16b4.1798 Suspend: 0 Teb: 7ffde000 Unfrozen 2 Id: 16b4.6f4 Suspend: 0 Teb: 7ffdd000 Unfrozen 0:000 k *** Stack trace for last set context - .thread/.cxr resets it ChildEBP RetAddr 0055f628 0142c797 crashme!crashdump_crashme+0x2 [c:\users\craig\developer\postgres\contrib\crashme\crashme.c @ 14] 0055f68c 0142c804 postgres!ExecMakeFunctionResult+0x427 [c:\users\craig\developer\postgres\src\backend\executor\execqual.c @ 1824] 0055f6b4 0142b760 postgres!ExecEvalFunc+0x34 [c:\users\craig\developer\postgres\src\backend\executor\execqual.c @ 2260] 0055f6e0 0142ba83 postgres!ExecTargetList+0x70 [c:\users\craig\developer\postgres\src\backend\executor\execqual.c @ 5095] 0055f720 0143f074 postgres!ExecProject+0x173 [c:\users\craig\developer\postgres\src\backend\executor\execqual.c @ 5312] 0055f734 01427e07 postgres!ExecResult+0x94 [c:\users\craig\developer\postgres\src\backend\executor\noderesult.c @ 157] 0055f744 01425ccd postgres!ExecProcNode+0x67 [c:\users\craig\developer\postgres\src\backend\executor\execprocnode.c @ 361] 0055f758 01426ace postgres!ExecutePlan+0x2d [c:\users\craig\developer\postgres\src\backend\executor\execmain.c @ 1236] 0055f788 0152ec7d postgres!standard_ExecutorRun+0x8e
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
On Fri, Dec 17, 2010 at 12:08, Craig Ringer cr...@postnewspapers.com.au wrote: On 17/12/2010 3:46 PM, Magnus Hagander wrote: On Dec 17, 2010 8:02 AM, Craig Ringer cr...@postnewspapers.com.au mailto:cr...@postnewspapers.com.au wrote: On 16/12/10 21:01, Magnus Hagander wrote: Found another problem in it: when running with an older version of dbghelp.dll (which I was), it simply didn't work. We need to grab the version of dbghelp.dll at runtime and pick which things we're going to dump based on that. I was about to suggest dropping the fallback loading of the system dbghelp.dll, and just bundle a suitable version with Pg, then I saw how big the DLL is. It's 800kb (!!) of code that'll hopefully never get used on any given system. With a footprint like that, bundling it seems rather less attractive. I think Magnus is right: test the dbghelp.dll version and fall back to supported features - as far back as XP, anyway; who cares about anything earlier than that. An updated version can always be put in place by the user if the built-in one can't produce enough detail. Did you get a chance to test that it still produced a full dump on your system? I've just tested that and it looks ok so far. The dumps produced are smaller - 1.3MB instead of ~4MB for a dump produced by calling a simple crashme() function from SQL, the source for which follows at the end of this post. However, I'm getting reasonable stack traces, as shown by the windb.exe output below. That said, the dump appears to exclude the backend's private memory. I can't resolve *fcinfo from PG_FUNCTION_ARGS; it looks like the direct values of arguments are available as they're on the stack, as are function local variables, but the process heap isn't dumped so you can't see the contents of any pointers-to-struct. That'll make it harder to track down many kinds of error - but, on the other hand, means that dumps of processes with huge work_mem etc won't be massively bloated. What version of dbghelp do you have? And what does the API report for it? The code is supposed to add the private memory if it finds version 6 or higher, perhaps that check is incorrect? That might be a safer starting point than including the private process memory, really. I'd like to offer a flag to turn on dumping of private process memory but I'm not sure how best to go about it, given that we'd want to avoid accessing GUCs etc if possible. later, I think; this is better for now. I'm not too happy with having a bunch of switches for it. People likely to tweak those can just install the debugger tools and use those, I think. So I'd be happy to have it enabled - given that we want a default. However, what I'm most interested in right now is finding out why it's not being included anymore in the new patch, because it's supposed to be.. I'm happy with the patch as it stands, with the one exception that consideration of mingw is required before it can be committed. What do you mean? That it has to be tested on mingw specifically, or something else? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Unnecessary limit on max_standby_streaming_delay
The limit on max_standby_streaming_delay is currently 35 minutes (around) - or you have to set it to unlimited. This is because the GUC is limited to MAX_INT/1000, unit milliseconds. Is there a reason for the /1000, or is it just an oversight thinking the unit was in seconds? If we can get rid of the /1000, it would make the limit over three weeks, which seems much more reasonable. Or if we made it a 64-bit counter it would go away completely. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unnecessary limit on max_standby_streaming_delay
Magnus Hagander wrote: The limit on max_standby_streaming_delay is currently 35 minutes (around) - or you have to set it to unlimited. This is because the GUC is limited to MAX_INT/1000, unit milliseconds. Is there a reason for the /1000, or is it just an oversight thinking the unit was in seconds? If we can get rid of the /1000, it would make the limit over three weeks, which seems much more reasonable. Or if we made it a 64-bit counter it would go away completely. The code for this uses TimestampTzPlusMilliseconds to determine its deadline: return TimestampTzPlusMilliseconds(rtime, max_standby_streaming_delay); Which is implemented like this: #define TimestampTzPlusMilliseconds(tz,ms) ((tz) + ((ms) * (int64) 1000)) My guess is that the range was limited at some point to avoid concerns of integer overflow in that multiplication, which I don't think actually is a risk due the int64 cast there. I confirmed the upper limit on this thing is the 36 minutes that Magnus suggests. This is a terrible limitation to have slipped through. For the pg_dump from the standby use case, the appropriate setting for most people is in the multiple hours range, but not unlimited (lest an out of control backup linger to overlap with what happens the next day). Whichever approach is taken to make this better, I think it deserves a backport too. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services and Supportwww.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
On Tue, Dec 14, 2010 at 10:54 PM, Fujii Masao masao.fu...@gmail.com wrote: I found a bug which always prevents SignalSomeChildren with BACKEND_TYPE_WALSND from sending a signal to walsender. Though currently SignalSomeChildren with BACKEND_TYPE_WALSND has not been called anywhere, it's not hard to believe that will be called in the future. So we should apply the following change. I think the attached might be a little tidier. Thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company signal-some-children.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PS display and standby query conflict
On Sun, Dec 12, 2010 at 10:06 PM, Fujii Masao masao.fu...@gmail.com wrote: But if it annoys you, it seems OK to change it. Don't see a reason to backpatch though? I think that It's worth backpatch to prevent users who observe the occurrence of the query conflicts carefully for testing 9.0 from getting confusing. I don't think this is important enough to back-patch, but it does seem like a good idea in general, so committed, but just to the master branch. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Excerpts from Robert Haas's message of vie dic 17 10:08:04 -0300 2010: On Tue, Dec 14, 2010 at 10:54 PM, Fujii Masao masao.fu...@gmail.com wrote: I found a bug which always prevents SignalSomeChildren with BACKEND_TYPE_WALSND from sending a signal to walsender. Though currently SignalSomeChildren with BACKEND_TYPE_WALSND has not been called anywhere, it's not hard to believe that will be called in the future. So we should apply the following change. I think the attached might be a little tidier. Thoughts? Yeah, that looks more readable to me. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] about error handling mechanism
On 17.12.2010 11:18, fanng yuan wrote: Hi! I was looking into the postgres error handling mechanism, and the documentation states that the present mechanism is primitive. I quote whenever the parser, planner/optimizer or executor decide that a statement cannot be processed any longer, the whole transaction gets aborted and the system jumps back into the main loop to get the next command from the client application. That quote is from the 7.4 manual. You should look at something more recent. Also, it states it is possible that the database server is in an inconsistent state at this point so returning to the upper executor or issuing more commands might corrupt the whole database Since postgres aborts the transaction compeletely, why would it be ever left in an inconsistant state in such an event? What it tries to say is that the system might be in an inconsistent state, which is the reason the transaction has to be aborted. The abort cleans it up, so you're not left in an inconsistent state. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang and LLVM
On 12/17/2010 04:50 AM, Alexandre Riveira wrote: Firebird http://mapopa.blogspot.com/2010/10/clang-compiling-successful-experiments.html In addition to top-posting this has nothing to do with the original question, and is seriously off-topic for a Postgres developers mailing list. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: remove redundant code from pl_exec.c
Pavel Stehule pavel.steh...@gmail.com writes: This patch remove redundant rows from PL/pgSQL executor (-89 lines). Doesn't change a functionality. I'm not really impressed with this idea: there's no a priori reason that all those loop types would necessarily have exactly the same control logic. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] ps_status on fastpath
I noticed that the fastpath code doesn't update ps_status, which would be harmless except that it leads to idle in transaction being logged in log_line_prefix for the command tag. Are there objections to applying this? -- Álvaro Herrera alvhe...@alvh.no-ip.org fastpath-ps.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: remove redundant code from pl_exec.c
Excerpts from Pavel Stehule's message of vie dic 17 07:02:00 -0300 2010: Hello This patch remove redundant rows from PL/pgSQL executor (-89 lines). Doesn't change a functionality. Hmm I'm not sure but I think the new code has some of the result values inverted. Did you test this thoroughly? I think this would be a nice simplification because the repetitive coding is ugly and confusing, but I'm nervous about the unstated assumption that all loop structs are castable to the new struct. Seems like it could be easily broken in the future. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
Magnus Hagander mag...@hagander.net writes: On Fri, Dec 17, 2010 at 12:08, Craig Ringer cr...@postnewspapers.com.au wrote: That might be a safer starting point than including the private process memory, really. I'm not too happy with having a bunch of switches for it. People likely to tweak those can just install the debugger tools and use those, I think. Yeah. So I'd be happy to have it enabled - given that we want a default. There is absolutely nothing more frustrating than having a crash dump that hasn't got the information you need. Please turn it on by default. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unnecessary limit on max_standby_streaming_delay
Greg Smith g...@2ndquadrant.com writes: Magnus Hagander wrote: The limit on max_standby_streaming_delay is currently 35 minutes (around) - or you have to set it to unlimited. This is because the GUC is limited to MAX_INT/1000, unit milliseconds. Is there a reason for the /1000, or is it just an oversight thinking the unit was in seconds? My guess is that the range was limited at some point to avoid concerns of integer overflow in that multiplication, which I don't think actually is a risk due the int64 cast there. Yes, it's certainly there on the thought that somebody might try to convert the value to microseconds in integer arithmetic. If you run through all the uses of the variable and confirm that that never happens, maybe it'd be safe to enlarge the limit. Check the units-aware GUC printing code in particular. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Excerpts from Pavel Stehule's message of jue dic 16 16:19:17 -0300 2010: The most performance issue of access to a untoasted array is solved with other patch. Was the other patch applied? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: remove redundant code from pl_exec.c
2010/12/17 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: This patch remove redundant rows from PL/pgSQL executor (-89 lines). Doesn't change a functionality. I'm not really impressed with this idea: there's no a priori reason that all those loop types would necessarily have exactly the same control logic. this code processes a rc from EXIT, CONTINUE and RETURN statement. All these statements are defined independent to outer loops, so there are no reason why this code has be different. And actually removed code was almost same. There was different a process for FOR statement, because there isn't possible direct return from cycle, because is necessary to release a allocated memory. There is no reason why the processing should be same, but actually is same. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Robert Haas robertmh...@gmail.com writes: I think the attached might be a little tidier. Thoughts? I'm not really thrilled at the idea of calling IsPostmasterChildWalSender for every child whether or not it will have any impact on the decision. That involves touching shared memory which can be rather expensive (see previous discussions about shared cache lines and so forth). The existing coding is pretty ugly, I agree. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plperlu problem with utf8
On Thu, Dec 16, 2010 at 07:24:46PM -0800, David Wheeler wrote: On Dec 16, 2010, at 6:39 PM, Alex Hunsaker wrote: Grr that should error out with Invalid server encoding, or worst case should return a length of 3 (it utf8 encoded 128 into 2 bytes instead of leaving it as 1). In this case the 333 causes perl store it internally as utf8. Well with SQL_ASCII anything goes, no? Anything except a byte that's all 0s :( Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang and LLVM
On tor, 2010-12-16 at 22:31 +0100, Gevik Babakhani wrote: I was wondering if there has been anyone experimenting to compile PG using LLVM/clang compiler tools. This has been dealt with a number of times. Search the archives. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: remove redundant code from pl_exec.c
2010/12/17 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Pavel Stehule's message of vie dic 17 07:02:00 -0300 2010: Hello This patch remove redundant rows from PL/pgSQL executor (-89 lines). Doesn't change a functionality. Hmm I'm not sure but I think the new code has some of the result values inverted. Did you test this thoroughly? I think this would be a nice simplification because the repetitive coding is ugly and confusing, but I'm nervous about the unstated assumption that all loop structs are castable to the new struct. Seems like it could be easily broken in the future. All regress tests was successful. A common structure isn't a new. There is same for FOR loops, there is some similar in parser yylval, and it is only explicit description of used construction for stmt structures. I should not to modify any other structure. But I am not strong in this. A interface can be changed and enhanced about pointer to label. Just I am not satisfied from current state, where same things are implemented with minimal difference. Pavel -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Pavel Stehule's message of jue dic 16 16:19:17 -0300 2010: The most performance issue of access to a untoasted array is solved with other patch. Was the other patch applied? no, it's in queue for next commitfest https://commitfest.postgresql.org/action/patch_view?id=440 Regards Pavel -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: remove redundant code from pl_exec.c
Pavel Stehule pavel.steh...@gmail.com writes: 2010/12/17 Tom Lane t...@sss.pgh.pa.us: I'm not really impressed with this idea: there's no a priori reason that all those loop types would necessarily have exactly the same control logic. There is no reason why the processing should be same, but actually is same. Yes, and it might need to be different in future, whereupon this patch would make life extremely difficult. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ps_status on fastpath
Alvaro Herrera alvhe...@alvh.no-ip.org writes: I noticed that the fastpath code doesn't update ps_status, which would be harmless except that it leads to idle in transaction being logged in log_line_prefix for the command tag. Are there objections to applying this? Hm, what about pgstat_report_activity()? Is anyone concerned about the added overhead for fastpath calls? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: remove redundant code from pl_exec.c
2010/12/17 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2010/12/17 Tom Lane t...@sss.pgh.pa.us: I'm not really impressed with this idea: there's no a priori reason that all those loop types would necessarily have exactly the same control logic. There is no reason why the processing should be same, but actually is same. Yes, and it might need to be different in future, whereupon this patch would make life extremely difficult. Do you know about some possible change? I can't to argument with this argument. But I can use a same argument. Isn't be more practical to have a centralized management for return code? There is same probability to be some features in future that will need a modify this fragment - for example a new return code value. Without centralized management, you will have to modify four fragments instead one. Regards Pavel Stehule regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Pavel Stehule pavel.steh...@gmail.com writes: I am resending a redesigned proposal about special plpgsql statement that support iteration over an array. OK ... == Iteration over multidimensional arrays == Its designed to reduce one dimension from source array. It can remove a slicing and simplify code: This seems like a really bad, confusing idea. I think it should throw a type-mismatch error in this case. If there is any use-case for such a thing, which I'm quite unconvinced of, it ought to use a separate syntax rather than overloading the element-by-element syntax. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Serializable lock consistency (was Re: CommitFest wrap-up)
On 15.12.2010 16:20, Florian Pflug wrote:
On Dec14, 2010, at 15:01 , Robert Haas wrote:
On Tue, Dec 14, 2010 at 7:51 AM, Florian Pflugf...@phlo.org wrote:
- serializable lock consistency - I am fairly certain this needs
rebasing. I don't have time to deal with it right away. That sucks,
because I think this is a really important change.
I can try to find some time to update the patch if it suffers from bit-rot.
Would that help?
Yes!
I've rebased the patch to the current HEAD, and re-run my FK concurrency test
suite,
available from https://github.com/fgp/fk_concurrency, to verify that things
still work.
I've also asserts to the callers of heap_{update,delete,lock_tuple} to verify
(and document)
that update_xmax may only be InvalidTransactionId if a lockcheck_snapshot is
passed to
heap_{update,delete,lock_tuple}.
Finally, I've improved the explanation in src/backend/executor/README of how
row locks and
REPEATABLE READ transactions interact, and tried to state the guarantees
provided by
FOR SHARE and FOR UPDATE locks more precisely.
I've published my work to
https://github.com/fgp/postgres/tree/serializable_lock_consistency,
and attached an updated patch. I'd be happy to give write access to that GIT
repository
to anyone who wants to help getting this committed.
Here's some typo style fixes for that, also available at
git://git.postgresql.org/git/users/heikki/postgres.git.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a397bad..3b12aca 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2034,8 +2034,8 @@ simple_heap_insert(Relation relation, HeapTuple tup)
* cid - delete command ID (used for visibility test, and stored into
* cmax if successful)
* wait - true if should wait for any conflicting update to commit/abort
- * lockcheck_snapshot - if not InvalidSnapshot, report the tuple as updated if it
- * was once locked by a transaction not visible under this snapshot
+ * lockcheck_snapshot - if not InvalidSnapshot, report the tuple as updated
+ * if it was once locked by a transaction not visible under this snapshot
*
* Normal, successful return value is HeapTupleMayBeUpdated, which
* actually means we did delete it. Failure return codes are
@@ -2046,14 +2046,14 @@ simple_heap_insert(Relation relation, HeapTuple tup)
* in ctid and update_xmax.
* If ctid is the same as t_self and update_xmax a valid transaction id,
* the tuple was deleted.
- * If ctid differes from t_self, the tuple was updated, ctid is the location
- * of the replacement tupe and update_xmax is the updating transaction's xid.
+ * If ctid differs from t_self, the tuple was updated. ctid is the location
+ * of the replacement tuple and update_xmax is the updating transaction's xid.
* update_xmax must in this case be used to verify that the replacement tuple
* matched.
* Otherwise, if ctid is the same as t_self and update_xmax is
- * InvalidTranactionId, the tuple was neither replaced nor deleted, but locked
- * by a transaction invisible to lockcheck_snapshot. This case can thus only
- * arise if lockcheck_snapshot is a valid snapshot.
+ * InvalidTransactionId, the tuple was neither replaced nor deleted, but
+ * locked by a transaction invisible to lockcheck_snapshot. This case can
+ * thus only arise if lockcheck_snapshot is a valid snapshot.
*/
HTSU_Result
heap_delete(Relation relation, ItemPointer tid,
@@ -2180,7 +2180,8 @@ l1:
result = HeapTupleUpdated;
}
- /* If the tuple was updated, we report the updating transaction's
+ /*
+ * If the tuple was updated, we report the updating transaction's
* xid in update_xmax. Otherwise, we must check that it wasn't
* locked by a transaction invisible to lockcheck_snapshot before
* continuing.
@@ -2368,9 +2369,9 @@ simple_heap_delete(Relation relation, ItemPointer tid)
* update_xmax - output parameter, used only for failure case (see below)
* cid - update command ID (used for visibility test, and stored into
* cmax/cmin if successful)
+ * wait - true if should wait for any conflicting update to commit/abort
* lockcheck_snapshot - if not InvalidSnapshot, report the tuple as updated
* if it was once locked by a transaction not visible under this snapshot
- * wait - true if should wait for any conflicting update to commit/abort
*
* Normal, successful return value is HeapTupleMayBeUpdated, which
* actually means we *did* update it. Failure return codes are
@@ -2387,14 +2388,14 @@ simple_heap_delete(Relation relation, ItemPointer tid)
* in ctid and update_xmax.
* If ctid is the same as t_self and update_xmax a valid transaction id,
* the tuple was deleted.
- * If ctid differes from t_self, the tuple was updated, ctid is the location
- * of the replacement tupe and update_xmax is the updating transaction's xid.
+ * If ctid differs from t_self, the tuple was
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Fri, Dec 17, 2010 at 10:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Pavel Stehule pavel.steh...@gmail.com writes: I am resending a redesigned proposal about special plpgsql statement that support iteration over an array. OK ... == Iteration over multidimensional arrays == Its designed to reduce one dimension from source array. It can remove a slicing and simplify code: This seems like a really bad, confusing idea. I think it should throw a type-mismatch error in this case. If there is any use-case for such a thing, which I'm quite unconvinced of, it ought to use a separate syntax rather than overloading the element-by-element syntax. I don't agree at all -- iterating arrays by slice is a frequently requested feature (you can kinda sorta do it by slice notation, but arr[n] giving null is a -general FAQ. This is how people think arrays should work. I suppose that having this functionality reserved in a tiny corner of plpgsql is not so good, but I think foreach... would become the preferred way to iterate arrays always. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I am resending a redesigned proposal about special plpgsql statement that support iteration over an array. OK ... == Iteration over multidimensional arrays == Its designed to reduce one dimension from source array. It can remove a slicing and simplify code: This seems like a really bad, confusing idea. I think it should throw a type-mismatch error in this case. If there is any use-case for such a thing, which I'm quite unconvinced of, it ought to use a separate syntax rather than overloading the element-by-element syntax. Without this feature any iteration over 2d and more dimensional array is not practical. If I have a 2D array, then I would to get a vector. Access to individual values can be to limiting, because I need a more cycles to get a complete vector. Usually I can use a array of row instead a 2d array, but still and in feature there is problem with iteration over row. So sometime is more practical to use a 2d array. Actually It raise a type mismatch error, when a user used a scalar variable and data is a vector (array) Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
On Fri, Dec 17, 2010 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I think the attached might be a little tidier. Thoughts? I'm not really thrilled at the idea of calling IsPostmasterChildWalSender for every child whether or not it will have any impact on the decision. That involves touching shared memory which can be rather expensive (see previous discussions about shared cache lines and so forth). The existing code already does that, unless I'm missing something. We could improve on my proposed patch a bit by doing the is_autovacuum test first and the walsender test second. I'm not sure how to improve on it beyond that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Merlin Moncure mmonc...@gmail.com writes: On Fri, Dec 17, 2010 at 10:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: This seems like a really bad, confusing idea. I think it should throw a type-mismatch error in this case. If there is any use-case for such a thing, which I'm quite unconvinced of, it ought to use a separate syntax rather than overloading the element-by-element syntax. I don't agree at all -- iterating arrays by slice is a frequently requested feature (you can kinda sorta do it by slice notation, but arr[n] giving null is a -general FAQ. This is how people think arrays should work. I suppose that having this functionality reserved in a tiny corner of plpgsql is not so good, but I think foreach... would become the preferred way to iterate arrays always. Well, okay, if it's useful we can have it, but I still say it needs to be a separate syntax. The example Pavel gives looks like nothing so much as a beginner's error, ie putting [] on the target variable when he shouldn't have. Furthermore, it's underspecified: who's to say how many dimensions of the array are supposed to get sliced off? There's no reasonable place to extend this syntax to specify that. It will also be inconsistent for foreach scalar in array to iterate element-by-element no matter how many dimensions array has, while foreach array in array does something different from that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Excerpts from Tom Lane's message of vie dic 17 12:27:30 -0300 2010: Robert Haas robertmh...@gmail.com writes: I think the attached might be a little tidier. Thoughts? I'm not really thrilled at the idea of calling IsPostmasterChildWalSender for every child whether or not it will have any impact on the decision. That involves touching shared memory which can be rather expensive (see previous discussions about shared cache lines and so forth). Is it possible to save the is walsender flag in the Backend struct? That would make it possible to solve the problem very easily. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not really thrilled at the idea of calling IsPostmasterChildWalSender for every child whether or not it will have any impact on the decision. That involves touching shared memory which can be rather expensive (see previous discussions about shared cache lines and so forth). The existing code already does that, unless I'm missing something. No, it only makes that call when it's actually relevant to the decision and it's exhausted all other possible tests. In particular, the call is never made for target = ALL which I think is the common case. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Reset 'ps' display just once when resolving VXID conflicts.
On Fri, Dec 17, 2010 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas rh...@postgresql.org writes: Reset 'ps' display just once when resolving VXID conflicts. This prevents the word waiting from briefly disappearing from the ps status line when ResolveRecoveryConflictWithVirtualXIDs begins a new iteration of the outer loop. I imagine the reason for the original coding was to avoid a useless gettimeofday kernel call in the common case that there are no conflicting xacts to wait for. Could we restore that behavior? Something like the attached? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company resolve-recovery-conflict-tweak.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Alvaro Herrera alvhe...@commandprompt.com writes: Is it possible to save the is walsender flag in the Backend struct? That would make it possible to solve the problem very easily. Yeah, I was wondering about that too, but the problem is that the postmaster doesn't know that at the time it forks the child. The flag in shared memory will get set later, but it's hard to tell how much later. Of course, that observation also means that anyplace the postmaster tries to distinguish walsenders from other children is fundamentally broken anyhow: a walsender that hasn't set the flag yet will get treated like a regular backend. I think what we ought to be looking to do is get rid of the distinction, so that the postmaster treats walsenders the same as other children. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Reset 'ps' display just once when resolving VXID conflicts.
Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: I imagine the reason for the original coding was to avoid a useless gettimeofday kernel call in the common case that there are no conflicting xacts to wait for. Could we restore that behavior? Something like the attached? I would just add the return-at-the-top. Changing the loop logic is not necessary --- the loop test is cheap. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Tom Lane t...@sss.pgh.pa.us: Merlin Moncure mmonc...@gmail.com writes: On Fri, Dec 17, 2010 at 10:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: This seems like a really bad, confusing idea. I think it should throw a type-mismatch error in this case. If there is any use-case for such a thing, which I'm quite unconvinced of, it ought to use a separate syntax rather than overloading the element-by-element syntax. I don't agree at all -- iterating arrays by slice is a frequently requested feature (you can kinda sorta do it by slice notation, but arr[n] giving null is a -general FAQ. This is how people think arrays should work. I suppose that having this functionality reserved in a tiny corner of plpgsql is not so good, but I think foreach... would become the preferred way to iterate arrays always. Well, okay, if it's useful we can have it, but I still say it needs to be a separate syntax. The example Pavel gives looks like nothing so much as a beginner's error, ie putting [] on the target variable when he shouldn't have. Now the message is unclean - but it can be enhanced. We can a diagnose situation when result is multidimensional array and target isn't array, and the we can to throw user friendly message. Furthermore, it's underspecified: who's to say how many dimensions of the array are supposed to get sliced off? There's no reasonable place to extend this syntax to specify that. It will also be inconsistent for foreach scalar in array to iterate element-by-element no matter how many dimensions array has, while foreach array in array does something different from that. it reduce just one dimension. Now I expect, and I think so it is correct, so user knows a used dimension. Just doesn't know a data. So user can to decide and fill correct type. The design strictly remove any U.I. from design. So using a incorect type is bug. Because a FOREACH syntax is new, we can to enhance it to possible direction: FOREACH VALUE var IN ARRAY expr LOOP END LOOP and then it will iterate per one field without a dimension reduction. So this possibility is available and I think so could be implemented too. Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
On Fri, Dec 17, 2010 at 11:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think what we ought to be looking to do is get rid of the distinction, so that the postmaster treats walsenders the same as other children. It's not apparent to me that the existing places where postmaster.c makes that distinction are in fact correct. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Excerpts from Fujii Masao's message of mié dic 15 00:54:39 -0300 2010: Hi, I found a bug which always prevents SignalSomeChildren with BACKEND_TYPE_WALSND from sending a signal to walsender. Though currently SignalSomeChildren with BACKEND_TYPE_WALSND has not been called anywhere, it's not hard to believe that will be called in the future. So we should apply the following change. BTW doesn't CountChildren have the same problem? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
On 17/12/2010 7:17 PM, Magnus Hagander wrote: What version of dbghelp do you have? 6.1.7600.16385 by default, as shipped in Windows 7 32-bit, and what I was testing with. 6.12.0002.633 is what came with my copy of Debugging Tools for windows, from the windows SDK. The same version comes with Visual Studio 10 Express. 6.9.0003.113 is what came with Visual Studio 9 Express. I've now re-tested with the latest, 6.12.0002.633, and have the same result. And what does the API report for it? This: version = (*pApiVersion)(); write_stderr(version: %hu.%hu.%hu\n, version-MajorVersion, version-MinorVersion, version-Revision); outputs version: 4.0.5 when loading dbghelp.dll 6.12.0002.633 from c:\postgres91\bin\dbghelp.dll as verified by a write_stderr() just before LoadLibrary and a test verifying the LoadLibrary worked. Shouldn't dbghelp.dll just ignore any flags it doesn't understand? I'm surprised we can't just pass flags a particular version doesn't know about and have it deal with them. Still, I guess it's not wise to assume such things. The code is supposed to add the private memory if it finds version 6 or higher, perhaps that check is incorrect? It begins to look like the version check might be lying. I'll dig further in a bit; it's 12:20am now and I need to sleep. I'm going on holiday in about 48 hours, and will be away from Windows (phew!) unless I get a VM set up during that time. I'll see what I can do. I'm happy with the patch as it stands, with the one exception that consideration of mingw is required before it can be committed. What do you mean? That it has to be tested on mingw specifically, or something else? I'm not even sure it'll compile or link with MinGW, and if it does, I doubt it'll be useful. It should only be compiled and called for native VC++ builds. I'm not sure if backend\port\win32 is built for all win32, or only for VC++. Testing for defined(_MSC_VER) would do the trick. I'm not sure testing defined(WIN32_ONLY_COMPILER) is quite right, since I doubt dbghelp.dll would be much use for a Pg compiled with Borland or the like either if that were ever supported. -- Craig Ringer Tech-related writing at http://soapyfrogs.blogspot.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Excerpts from Tom Lane's message of vie dic 17 13:18:35 -0300 2010: Alvaro Herrera alvhe...@commandprompt.com writes: Is it possible to save the is walsender flag in the Backend struct? That would make it possible to solve the problem very easily. Yeah, I was wondering about that too, but the problem is that the postmaster doesn't know that at the time it forks the child. The flag in shared memory will get set later, but it's hard to tell how much later. Yeah, I arrived at the same conclusion. I was wondering if we could cache the result of the shared mem lookup the first time it was done, but as you say there would still be a race condition. I think what we ought to be looking to do is get rid of the distinction, so that the postmaster treats walsenders the same as other children. I think the problem with this is that walsenders are treated in a very special way during shutdown -- they need to stay up until after bgwriter is gone. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unnecessary limit on max_standby_streaming_delay
On fre, 2010-12-17 at 12:57 +0100, Magnus Hagander wrote: The limit on max_standby_streaming_delay is currently 35 minutes (around) - or you have to set it to unlimited. This is because the GUC is limited to MAX_INT/1000, unit milliseconds. Is there a reason for the /1000, or is it just an oversight thinking the unit was in seconds? Yeah, I actually noticed this week that log_min_duration_statement is limited to the same 35 minutes for the same reason. Might be good to address that, too. Note that statement_timeout does not have that limit. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Reset 'ps' display just once when resolving VXID conflicts.
On Fri, Dec 17, 2010 at 11:20 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: I imagine the reason for the original coding was to avoid a useless gettimeofday kernel call in the common case that there are no conflicting xacts to wait for. Could we restore that behavior? Something like the attached? I would just add the return-at-the-top. Changing the loop logic is not necessary --- the loop test is cheap. Hmm... that's a fine bit of hairsplitting, but OK. Done that way, then. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Pavel Stehule pavel.steh...@gmail.com writes: 2010/12/17 Tom Lane t...@sss.pgh.pa.us: Furthermore, it's underspecified: who's to say how many dimensions of the array are supposed to get sliced off? Â There's no reasonable place to extend this syntax to specify that. Â It will also be inconsistent for foreach scalar in array to iterate element-by-element no matter how many dimensions array has, while foreach array in array does something different from that. it reduce just one dimension. Now I expect, and I think so it is correct, so user knows a used dimension. Just doesn't know a data. So user can to decide and fill correct type. The design strictly remove any U.I. from design. So using a incorect type is bug. In other words, your proposal is error-prone to use, restricted in what it can do, and incapable of being extended later without breaking things. If there is some redeeming social value to set against those problems, I'm not seeing it. What I think we should have is FOREACH scalar-variable IN ARRAY array-expression which iterates element by element regardless of how many dimensions the array has. Then there should be some other syntax for iterating over slices, and we should give some thought to being able to specify how deep the slice is. I can definitely think of use cases for pulling off either 1 dimension at a time (so you get vectors) or N-1 dimensions at a time, and it's not out of the realm of reason to want intermediate cases. Maybe FOR_EACH scalar-variable IN ARRAY array-expression FOR_SLICE array-variable [DEPTH n] IN ARRAY array-expression Or I guess you could use the same leading keyword if you make the depth specification mandatory for the slice case: FOREACH scalar-variable IN ARRAY array-expression FOREACH array-variable SLICE n IN ARRAY array-expression That might be a better idea since it avoids the inevitable argument over whether the default slice depth should be 1 dimension or N-1 dimensions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie dic 17 13:18:35 -0300 2010: I think what we ought to be looking to do is get rid of the distinction, so that the postmaster treats walsenders the same as other children. I think the problem with this is that walsenders are treated in a very special way during shutdown -- they need to stay up until after bgwriter is gone. Why do they need to survive the bgwriter? And more to the point, why does that logic need to be implemented on the postmaster side? Since only the child process really knows reliably whether it's a walsender, it'd be far safer if the behavioral difference could be handled on the child side. I haven't looked at the details, but I'm wondering if we couldn't make this go by having walsender children react differently to the same signals the postmaster sends other children. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable lock consistency (was Re: CommitFest wrap-up)
On Dec17, 2010, at 16:49 , Heikki Linnakangas wrote:
On 15.12.2010 16:20, Florian Pflug wrote:
On Dec14, 2010, at 15:01 , Robert Haas wrote:
On Tue, Dec 14, 2010 at 7:51 AM, Florian Pflugf...@phlo.org wrote:
- serializable lock consistency - I am fairly certain this needs
rebasing. I don't have time to deal with it right away. That sucks,
because I think this is a really important change.
I can try to find some time to update the patch if it suffers from
bit-rot. Would that help?
Yes!
I've rebased the patch to the current HEAD, and re-run my FK concurrency
test suite,
available from https://github.com/fgp/fk_concurrency, to verify that things
still work.
I've also asserts to the callers of heap_{update,delete,lock_tuple} to
verify (and document)
that update_xmax may only be InvalidTransactionId if a lockcheck_snapshot is
passed to
heap_{update,delete,lock_tuple}.
Finally, I've improved the explanation in src/backend/executor/README of how
row locks and
REPEATABLE READ transactions interact, and tried to state the guarantees
provided by
FOR SHARE and FOR UPDATE locks more precisely.
I've published my work to
https://github.com/fgp/postgres/tree/serializable_lock_consistency,
and attached an updated patch. I'd be happy to give write access to that GIT
repository
to anyone who wants to help getting this committed.
Here's some typo style fixes for that, also available at
git://git.postgresql.org/git/users/heikki/postgres.git.
Thanks! FYI, I've pulled these into
https://github.com/fgp/postgres/tree/serializable_lock_consistency
best regards,
Florian Pflug
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] clang and LLVM
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I was wondering if there has been anyone experimenting to compile PG using LLVM/clang compiler tools. I got it working on Linux but it required a Postgres src file change to work properly (see previous thread by me). Supposedly the clang bug that caused this was fixed in llvm's HEAD, but HEAD will not compile for me yet, so I cannot verify it yet. There's a separate bug concerning usage of plperl, but that's for another day. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201012171144 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAk0Lk4UACgkQvJuQZxSWSsigEwCdG9aNk50eK1EzelewzqGMhgyO 8fYAoPt+emmdaxEd7lmeYidYgpqIdfjK =6ksq -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
On Fri, Dec 17, 2010 at 17:24, Craig Ringer cr...@postnewspapers.com.au wrote: On 17/12/2010 7:17 PM, Magnus Hagander wrote: What version of dbghelp do you have? 6.1.7600.16385 by default, as shipped in Windows 7 32-bit, and what I was testing with. 6.12.0002.633 is what came with my copy of Debugging Tools for windows, from the windows SDK. The same version comes with Visual Studio 10 Express. 6.9.0003.113 is what came with Visual Studio 9 Express. I've now re-tested with the latest, 6.12.0002.633, and have the same result. And what does the API report for it? This: version = (*pApiVersion)(); write_stderr(version: %hu.%hu.%hu\n, version-MajorVersion, version-MinorVersion, version-Revision); outputs version: 4.0.5 when loading dbghelp.dll 6.12.0002.633 from c:\postgres91\bin\dbghelp.dll as verified by a write_stderr() just before LoadLibrary and a test verifying the LoadLibrary worked. Now, that's annoying. So clearly we can't use that function to determine which version we're on. Seems it only works for image help api, and not the general thing. According to http://msdn.microsoft.com/en-us/library/ms679294(v=vs.85).aspx, we could look for: SysEnumLines - if present, we have at least 6.1. However, I don't see any function that appeared in 6.0 only.. We're probably better off looking at the VERSIONINFO structure. Which has a not-so-nice API, but at least it's documented :) Shouldn't dbghelp.dll just ignore any flags it doesn't understand? I'm surprised we can't just pass flags a particular version doesn't know about and have it deal with them. Still, I guess it's not wise to assume such things. You'd think so, but if I include all the flags and run it with the default one on Windows 2003, it comes back with the infamous Parameter Is Incorrect error. The code is supposed to add the private memory if it finds version 6 or higher, perhaps that check is incorrect? It begins to look like the version check might be lying. I'll dig further in a bit; it's 12:20am now and I need to sleep. I'm going on holiday in about 48 hours, and will be away from Windows (phew!) unless I get a VM set up during that time. I'll see what I can do. I'll try to put together a version that uses the VERSIONINFO to determine it, for you to test. I'm happy with the patch as it stands, with the one exception that consideration of mingw is required before it can be committed. What do you mean? That it has to be tested on mingw specifically, or something else? I'm not even sure it'll compile or link with MinGW, and if it does, I doubt it'll be useful. It should only be compiled and called for native VC++ builds. I'm pretty sure it could be useful with mingw as well. not *as* useful, but still useful. You'd get proper stack traces and such for anything in msvcrt, and the actual reason for the exception etc. I'm not sure if backend\port\win32 is built for all win32, or only for VC++. It's for all win32. Testing for defined(_MSC_VER) would do the trick. I'm not sure testing defined(WIN32_ONLY_COMPILER) is quite right, since I doubt dbghelp.dll would be much use for a Pg compiled with Borland or the like either if that were ever supported. We don't support building the backend with borland - only libpq and psql. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Fri, Dec 17, 2010 at 11:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Pavel Stehule pavel.steh...@gmail.com writes: 2010/12/17 Tom Lane t...@sss.pgh.pa.us: Furthermore, it's underspecified: who's to say how many dimensions of the array are supposed to get sliced off? There's no reasonable place to extend this syntax to specify that. It will also be inconsistent for foreach scalar in array to iterate element-by-element no matter how many dimensions array has, while foreach array in array does something different from that. it reduce just one dimension. Now I expect, and I think so it is correct, so user knows a used dimension. Just doesn't know a data. So user can to decide and fill correct type. The design strictly remove any U.I. from design. So using a incorect type is bug. In other words, your proposal is error-prone to use, restricted in what it can do, and incapable of being extended later without breaking things. If there is some redeeming social value to set against those problems, I'm not seeing it. What I think we should have is FOREACH scalar-variable IN ARRAY array-expression which iterates element by element regardless of how many dimensions the array has. Then there should be some other syntax for iterating over slices, and we should give some thought to being able to specify how deep the slice is. I can definitely think of use cases for pulling off either 1 dimension at a time (so you get vectors) or N-1 dimensions at a time, and it's not out of the realm of reason to want intermediate cases. Maybe FOR_EACH scalar-variable IN ARRAY array-expression FOR_SLICE array-variable [DEPTH n] IN ARRAY array-expression Or I guess you could use the same leading keyword if you make the depth specification mandatory for the slice case: FOREACH scalar-variable IN ARRAY array-expression FOREACH array-variable SLICE n IN ARRAY array-expression That might be a better idea since it avoids the inevitable argument over whether the default slice depth should be 1 dimension or N-1 dimensions. another way: FOREACH scalar IN ARRAY arr_exp DIMS in dim_var dim_var being int[], or possibly text, of length #dimensions, giving per dimesion index. I like this because it would fit well with alternate form of unnest, should it ever be written: create function unnest(anyarray, dims out int[], elem out anyelement) returns setof... SLICE notation is still good though, and it's probably faster since you have less work to do in iteration step? It's certainly easier, but very plpgsql specific. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug in SignalSomeChildren
On Fri, Dec 17, 2010 at 11:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie dic 17 13:18:35 -0300 2010: I think what we ought to be looking to do is get rid of the distinction, so that the postmaster treats walsenders the same as other children. I think the problem with this is that walsenders are treated in a very special way during shutdown -- they need to stay up until after bgwriter is gone. Why do they need to survive the bgwriter? And more to the point, why does that logic need to be implemented on the postmaster side? Since only the child process really knows reliably whether it's a walsender, it'd be far safer if the behavioral difference could be handled on the child side. I haven't looked at the details, but I'm wondering if we couldn't make this go by having walsender children react differently to the same signals the postmaster sends other children. I'm not too sure we're shutting down the WAL senders right now. I think they may just be exiting on their own when the postmaster goes away. /* * Emergency bailout if postmaster has died. This is to avoid the * necessity for manual cleanup of all postmaster children. */ if (!PostmasterIsAlive(true)) exit(1); I'm having a bit of trouble confirming this on MacOS X, though. Attaching gdb to either the startup process or a WAL sender causes PostmasterIsAlive to return false, resulting in a near-immediate exit. Seems pretty stupid for attaching gdb to change the return value of getppid() but it seems like that must be what's happening. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2010/12/17 Tom Lane t...@sss.pgh.pa.us: Furthermore, it's underspecified: who's to say how many dimensions of the array are supposed to get sliced off? There's no reasonable place to extend this syntax to specify that. It will also be inconsistent for foreach scalar in array to iterate element-by-element no matter how many dimensions array has, while foreach array in array does something different from that. it reduce just one dimension. Now I expect, and I think so it is correct, so user knows a used dimension. Just doesn't know a data. So user can to decide and fill correct type. The design strictly remove any U.I. from design. So using a incorect type is bug. In other words, your proposal is error-prone to use, restricted in what it can do, and incapable of being extended later without breaking things. If there is some redeeming social value to set against those problems, I'm not seeing it. What I think we should have is FOREACH scalar-variable IN ARRAY array-expression which iterates element by element regardless of how many dimensions the array has. Then there should be some other syntax for iterating over slices, and we should give some thought to being able to specify how deep the slice is. I can definitely think of use cases for pulling off either 1 dimension at a time (so you get vectors) or N-1 dimensions at a time, and it's not out of the realm of reason to want intermediate cases. I am not against Maybe FOR_EACH scalar-variable IN ARRAY array-expression FOR_SLICE array-variable [DEPTH n] IN ARRAY array-expression Or I guess you could use the same leading keyword if you make the depth specification mandatory for the slice case: FOREACH scalar-variable IN ARRAY array-expression FOREACH array-variable SLICE n IN ARRAY array-expression I prefer FOREACH keyword. The syntax can be enhanced and I like a talk about it. I am not sure if SLICE is good keyword for this, but I don't know better - hope so native speakers can select well. I could to use maybe DIMENSIONS ? Regards Pavel That might be a better idea since it avoids the inevitable argument over whether the default slice depth should be 1 dimension or N-1 dimensions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Sat, Dec 18, 2010 at 02:03, Merlin Moncure mmonc...@gmail.com wrote: FOREACH scalar-variable IN ARRAY array-expression FOR_EACH scalar-variable IN ARRAY array-expression FOR_SLICE array-variable [DEPTH n] IN ARRAY array-expression FOREACH scalar-variable IN ARRAY array-expression FOREACH array-variable SLICE n IN ARRAY array-expression FOREACH scalar IN ARRAY arr_exp DIMS in dim_var It should be not a main subject, but I remember there was a discussion that IN ARRAY array-expression looks redundant for a literal array: IN ARRAY ARRAY[1, 3, 5] Are there any improvement for the issue? -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Merlin Moncure mmonc...@gmail.com On Fri, Dec 17, 2010 at 11:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Pavel Stehule pavel.steh...@gmail.com writes: 2010/12/17 Tom Lane t...@sss.pgh.pa.us: Furthermore, it's underspecified: who's to say how many dimensions of the array are supposed to get sliced off? There's no reasonable place to extend this syntax to specify that. It will also be inconsistent for foreach scalar in array to iterate element-by-element no matter how many dimensions array has, while foreach array in array does something different from that. it reduce just one dimension. Now I expect, and I think so it is correct, so user knows a used dimension. Just doesn't know a data. So user can to decide and fill correct type. The design strictly remove any U.I. from design. So using a incorect type is bug. In other words, your proposal is error-prone to use, restricted in what it can do, and incapable of being extended later without breaking things. If there is some redeeming social value to set against those problems, I'm not seeing it. What I think we should have is FOREACH scalar-variable IN ARRAY array-expression which iterates element by element regardless of how many dimensions the array has. Then there should be some other syntax for iterating over slices, and we should give some thought to being able to specify how deep the slice is. I can definitely think of use cases for pulling off either 1 dimension at a time (so you get vectors) or N-1 dimensions at a time, and it's not out of the realm of reason to want intermediate cases. Maybe FOR_EACH scalar-variable IN ARRAY array-expression FOR_SLICE array-variable [DEPTH n] IN ARRAY array-expression Or I guess you could use the same leading keyword if you make the depth specification mandatory for the slice case: FOREACH scalar-variable IN ARRAY array-expression FOREACH array-variable SLICE n IN ARRAY array-expression That might be a better idea since it avoids the inevitable argument over whether the default slice depth should be 1 dimension or N-1 dimensions. another way: FOREACH scalar IN ARRAY arr_exp DIMS in dim_var dim_var being int[], or possibly text, of length #dimensions, giving per dimesion index. If dim_var contains length it is need to be renamed: FOREACH scalar IN ARRAY arr_exp SIZES IN sizes_var. I like this because it would fit well with alternate form of unnest, should it ever be written: create function unnest(anyarray, dims out int[], elem out anyelement) returns setof... SLICE notation is still good though, and it's probably faster since you have less work to do in iteration step? It's certainly easier, but very plpgsql specific. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- // Dmitriy.
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
On Fri, Dec 17, 2010 at 17:42, Magnus Hagander mag...@hagander.net wrote: On Fri, Dec 17, 2010 at 17:24, Craig Ringer cr...@postnewspapers.com.au wrote: On 17/12/2010 7:17 PM, Magnus Hagander wrote: Now, that's annoying. So clearly we can't use that function to determine which version we're on. Seems it only works for image help api, and not the general thing. According to http://msdn.microsoft.com/en-us/library/ms679294(v=vs.85).aspx, we could look for: SysEnumLines - if present, we have at least 6.1. However, I don't see any function that appeared in 6.0 only.. Actually, I'm wrong - there are functions enough to determine the version. So here's a patch that tries that. (BTW, the document is actually incorrect - partially because the version that's shipped with Windows 2003 which is 5.2, is more or less undocumented) Let me know how that one works for you, and if it doesn't, whether it does actually detect any of those higher versions by searching for the functions. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml index 0eeaa25..a480a8d 100644 --- a/doc/src/sgml/installation.sgml +++ b/doc/src/sgml/installation.sgml @@ -2734,6 +2734,22 @@ cc-1020 cc: ERROR File = pqcomm.c, Line = 427 under commandCMD.EXE/command, as the MSYS console has buffering issues. /para + + sect3 +titleCollecting crash dumps on Windows/title + +para + If PostgreSQL on Windows crashes, it has the ability to generate + productnameminidumps/ that can be used to track down the cause + for the crash, similar to core dumps on Unix. These dumps can be + read using the productnameWindows Debugger Tools/ or using + productnameVisual Studio/. To enable the generation of dumps + on Windows, create a subdirectory named filenamecrashdumps/filename + inside the cluster data directory. The dumps will then be written + into this directory with a unique name based on the identifier of + the crashing process and the current time of the crash. +/para + /sect3 /sect2 sect2 id=installation-notes-sco diff --git a/src/backend/main/main.c b/src/backend/main/main.c index 6cb70f2..5a56c25 100644 --- a/src/backend/main/main.c +++ b/src/backend/main/main.c @@ -79,6 +79,14 @@ main(int argc, char *argv[]) argv = save_ps_display_args(argc, argv); /* + * If supported on the current platform, set up a handler to be called if + * the backend/postmaster crashes with a fatal signal or exception. + */ +#ifdef WIN32 + pgwin32_install_crashdump_handler(); +#endif + + /* * Set up locale information from environment. Note that LC_CTYPE and * LC_COLLATE will be overridden later from pg_control if we are in an * already-initialized database. We set them here so that they will be diff --git a/src/backend/port/win32/Makefile b/src/backend/port/win32/Makefile index 8bf9f74..d00c334 100644 --- a/src/backend/port/win32/Makefile +++ b/src/backend/port/win32/Makefile @@ -12,6 +12,6 @@ subdir = src/backend/port/win32 top_builddir = ../../../.. include $(top_builddir)/src/Makefile.global -OBJS = timer.o socket.o signal.o security.o mingwcompat.o +OBJS = timer.o socket.o signal.o security.o mingwcompat.o crashdump.o include $(top_srcdir)/src/backend/common.mk diff --git a/src/backend/port/win32/crashdump.c b/src/backend/port/win32/crashdump.c new file mode 100644 index 000..82fb692 --- /dev/null +++ b/src/backend/port/win32/crashdump.c @@ -0,0 +1,222 @@ +/*- + * + * win32_crashdump.c + * Automatic crash dump creation for PostgreSQL on Windows + * + * The crashdump feature traps unhandled win32 exceptions produced by the + * backend, and tries to produce a Windows MiniDump crash + * dump for later debugging and analysis. The machine performing the dump + * doesn't need any special debugging tools; the user only needs to send + * the dump to somebody who has the same version of PostgreSQL and has debugging + * tools. + * + * crashdump module originally by Craig Ringer ring...@ringerc.id.au + * + * LIMITATIONS + * === + * This *won't* work in hard OOM situations or stack overflows. + * + * For those, it'd be necessary to take a much more complicated approach where + * the handler switches to a new stack (if it can) and forks a helper process + * to debug its self. + * + * POSSIBLE FUTURE WORK + * + * For bonus points, the crash dump format permits embedding of user-supplied data. + * If there's anything else (postgresql.conf? Last few lines of a log file?) that + * should always be supplied with a crash dump, it could potentially be added, + * though at the cost of a greater chance of the crash dump failing. + * + * + * Portions Copyright (c) 1996-2010, PostgreSQL Global Development Group + * + * IDENTIFICATION + *
Re: [HACKERS] bug in SignalSomeChildren
On 17.12.2010 19:08, Robert Haas wrote: On Fri, Dec 17, 2010 at 11:43 AM, Tom Lanet...@sss.pgh.pa.us wrote: Alvaro Herreraalvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie dic 17 13:18:35 -0300 2010: I think what we ought to be looking to do is get rid of the distinction, so that the postmaster treats walsenders the same as other children. I think the problem with this is that walsenders are treated in a very special way during shutdown -- they need to stay up until after bgwriter is gone. Why do they need to survive the bgwriter? Because we want the shutdown checkpoint record that bgwriter writes just before it dies to be replicated to the standbys. And more to the point, why does that logic need to be implemented on the postmaster side? Since only the child process really knows reliably whether it's a walsender, it'd be far safer if the behavioral difference could be handled on the child side. I haven't looked at the details, but I'm wondering if we couldn't make this go by having walsender children react differently to the same signals the postmaster sends other children. I'm not too sure we're shutting down the WAL senders right now. Sure we do. postmaster sends walsenders SIGUSR2 when bgwriter dies. When a walsender receives SIGUSR2, it tries to send all pending WAL, and terminates after that. The postmaster goes into PM_SHUTDOWN_2 state, waiting for all the walsenders and the archiver process to die. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Merlin Moncure mmonc...@gmail.com writes: another way: FOREACH scalar IN ARRAY arr_exp DIMS in dim_var dim_var being int[], or possibly text, of length #dimensions, giving per dimesion index. [ scratches head... ] I don't follow what you envision this doing, exactly? I'm not thrilled with that specific syntax because it'd require making DIMS a reserved word, but right at the moment I'm more concerned about what semantics you have in mind. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Itagaki Takahiro itagaki.takah...@gmail.com: On Sat, Dec 18, 2010 at 02:03, Merlin Moncure mmonc...@gmail.com wrote: FOREACH scalar-variable IN ARRAY array-expression FOR_EACH scalar-variable IN ARRAY array-expression FOR_SLICE array-variable [DEPTH n] IN ARRAY array-expression FOREACH scalar-variable IN ARRAY array-expression FOREACH array-variable SLICE n IN ARRAY array-expression FOREACH scalar IN ARRAY arr_exp DIMS in dim_var It should be not a main subject, but I remember there was a discussion that IN ARRAY array-expression looks redundant for a literal array: IN ARRAY ARRAY[1, 3, 5] Are there any improvement for the issue? yes. It know it. The reason for this is bigger space for possible future features related to FOREACH loop. Regards Pavel -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Fri, Dec 17, 2010 at 12:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: another way: FOREACH scalar IN ARRAY arr_exp DIMS in dim_var dim_var being int[], or possibly text, of length #dimensions, giving per dimesion index. [ scratches head... ] I don't follow what you envision this doing, exactly? I'm not thrilled with that specific syntax because it'd require making DIMS a reserved word, but right at the moment I'm more concerned about what semantics you have in mind. It's like _pg_expandarray but alterted support multiple dimensions: select * from unnest_dims(array[['a','b'],['c','d']]) returns [1,1], 'a' [1,2], 'b' [2,1], 'c' [2,2], 'd' this provides alternate way of pulling slices, slower possibly, but more abstract. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On 12/17/2010 12:15 PM, Pavel Stehule wrote: 2010/12/17 Itagaki Takahiroitagaki.takah...@gmail.com: It should be not a main subject, but I remember there was a discussion that IN ARRAY array-expression looks redundant for a literal array: IN ARRAY ARRAY[1, 3, 5] Are there any improvement for the issue? yes. It know it. The reason for this is bigger space for possible future features related to FOREACH loop. So what you're saying is we need to allow ugliness now so we can have more ugliness in future? I don't find that a convincing argument. I share the dislike for this syntax. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Merlin Moncure mmonc...@gmail.com writes: On Fri, Dec 17, 2010 at 12:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: [ scratches head... ] I don't follow what you envision this doing, exactly? It's like _pg_expandarray but alterted support multiple dimensions: select * from unnest_dims(array[['a','b'],['c','d']]) returns [1,1], 'a' [1,2], 'b' [2,1], 'c' [2,2], 'd' Oh, so that's an *output* not an input. And IIUC what you are returning is the subscripts associated with the current element, not the array's dimensions. Seems like it should go beside the normal target variable then, not at the end. FOREACH variable_for_value [, variable_for_subscripts ] IN ARRAY ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Andrew Dunstan and...@dunslane.net writes: On 12/17/2010 12:15 PM, Pavel Stehule wrote: The reason for this is bigger space for possible future features related to FOREACH loop. So what you're saying is we need to allow ugliness now so we can have more ugliness in future? I don't find that a convincing argument. I share the dislike for this syntax. Well, we did beat up Pavel over trying to shoehorn this facility into the existing FOR syntax, so I can hardly blame him for thinking this way. The question is whether we're willing to assume that FOREACH will be limited to iterating over arrays, meaning we'll be stuck with inventing yet another initial keyword if some other fundamentally different concept comes along. Right at the moment I can't think of any plausible candidates, but ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Andrew Dunstan and...@dunslane.net: On 12/17/2010 12:15 PM, Pavel Stehule wrote: 2010/12/17 Itagaki Takahiroitagaki.takah...@gmail.com: It should be not a main subject, but I remember there was a discussion that IN ARRAY array-expression looks redundant for a literal array: IN ARRAY ARRAY[1, 3, 5] Are there any improvement for the issue? yes. It know it. The reason for this is bigger space for possible future features related to FOREACH loop. So what you're saying is we need to allow ugliness now so we can have more ugliness in future? I don't find that a convincing argument. I share the dislike for this syntax. can be strange from me, but it is. If we close a back door now, then we have not a space after ten years. There can be possible loops over records, maybe over other iterable data. With this design is important one think. A keyword after K_IN must not be a reserved keyword. I am expecting, so typical use case doesn't be a iteration over constant array, but over variable so mostly often you have to write FOREACH var IN ARRAY second_var LOOP ... END LOOP Regards Pavel cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal : cross-column stats
Dne 12.12.2010 15:43, Heikki Linnakangas napsal(a): On 12.12.2010 15:17, Martijn van Oosterhout wrote: On Sun, Dec 12, 2010 at 03:58:49AM +0100, Tomas Vondra wrote: Very cool that you're working on this. +1 Lets talk about one special case - I'll explain how the proposed solution works, and then I'll explain how to make it more general, what improvements are possible, what issues are there. Anyway this is by no means a perfect or complete solution - it's just a starting point. It looks like you handled most of the issues. Just a few points: - This is obviously applicable to more than just integers, probably anything with a b-tree operator class. What you've coded seems rely on calculations on the values. Have you thought about how it could work for, for example, strings? The classic failure case has always been: postcodes and city names. Strongly correlated, but in a way that the computer can't easily see. Yeah, and that's actually analogous to the example I used in my presentation. The way I think of that problem is that once you know the postcode, knowing the city name doesn't add any information. The postcode implies the city name. So the selectivity for postcode = ? AND city = ? should be the selectivity of postcode = ? alone. The measurement we need is implicativeness: How strongly does column A imply a certain value for column B. Perhaps that could be measured by counting the number of distinct values of column B for each value of column A, or something like that. I don't know what the statisticians call that property, or if there's some existing theory on how to measure that from a sample. That's assuming the combination has any matches. It's possible that the user chooses a postcode and city combination that doesn't exist, but that's no different from a user doing city = 'fsdfsdfsd' on a single column, returning no matches. We should assume that the combination makes sense. Well, I've read several papers this week, in an attempt to find out what possibilities are there when implementing cross-column statistics. One of the papers seems like a reasonable solution to this particular problem - discrete data + equality queries. The article is A Bayesian Approach to Estimating The Selectivity of Conjuctive Predicates (written by Heimel, Markl and Murthy). It's freely available as a PDF http:://subs.emis.de/LNI/Proceedings/Proceedings144/52.pdf I do have a PDF with my own notes in it (as annotations), let me know if you're interested ... The basic principle is that instead of 'attribute value independence' (which is the problem with correlated columns) they assume 'uniform correlation' which is a much weaker assumption. In the end, all they need to compute an estimate is number of distinct values for each of the columns (we already have that in pg_stats) and a number of distinct values for the group of columns in a query. They really don't need any multidimensional histogram or something like that. The funny thing is I've implemented most of the PoC before reading the article - the only difference was the way to combine the estimates ( pros and cons -- So let's see the pros: * it's reasonably simple, the overhead should be minimal I guess (we're already estimating distinct values for the columns, so I guess we can do that for multiple columns with reasonable impact) * there are no 'advanced data structures' as multi-dimensional histograms that are expensive to build and maintain * it consistently produces better estimates than the current estimator based on attribute value independence assumption, and it's reasonably accurate (I've been playing with it for some time, so we'll need more tests of course) * it's easily extensible to more columns * I guess we could improve the estimated by our own heuristicts, to catch the special case when one column is perfectly implied by another one (e.g. when state is implied by ZIP code) - we can catch that by 'distinct for combination = distinct for column' and use just the estimate for one of the column but there are some cons too: * this really is not designed to work with real-valued data, it's a very nice solution for discrete data (namely 'labels' as for example city/state names, ZIP codes etc.) * you need to know the list of columns in advance (there's nothing like 'let's build' the stats for columns (a,b,c) and then query just (a,b)) as you need to know the count of distinct values for the queried columns (well, maybe it's possible - I guess we could 'integrate' over all possible values of the omited column) * it's built for 'equality' qeuries, although maybe we could modify it for inequalities too (but it won't be as elegant), but I guess the equality queries are much more common in case of discrete data (OK, you can run something like WHERE (zip_code BETWEEN '49389' AND '54980') AND state = '48'
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Dec 17, 2010, at 9:31 AM, Tom Lane wrote: Well, we did beat up Pavel over trying to shoehorn this facility into the existing FOR syntax, so I can hardly blame him for thinking this way. The question is whether we're willing to assume that FOREACH will be limited to iterating over arrays, meaning we'll be stuck with inventing yet another initial keyword if some other fundamentally different concept comes along. Right at the moment I can't think of any plausible candidates, but ... FOREACH pair IN HSTORE… David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] relaxing sync commit if no WAL written (was Re: unlogged tables)
On Wed, Dec 15, 2010 at 2:20 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Looks ok. I'd suggest rewording this comment though: [ the comment in question ] It's a bit hard to follow, as it first lists exceptions on when we must flush XLOG immediately, and then lists conditions on when we can skip it. See if the attached looks better to you. I mostly adopted your proposal, with a bit of additional wordsmithing, and I also added a parenthetical comment about why we don't skip writing the commit record altogether in this case, since that's come up twice now. I've removed the references to unlogged tables for now, as I'm thinking it makes sense to commit this part first. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company relax-sync-commit-v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
David E. Wheeler da...@kineticode.com writes: On Dec 17, 2010, at 9:31 AM, Tom Lane wrote: Well, we did beat up Pavel over trying to shoehorn this facility into the existing FOR syntax, so I can hardly blame him for thinking this way. The question is whether we're willing to assume that FOREACH will be limited to iterating over arrays, meaning we'll be stuck with inventing yet another initial keyword if some other fundamentally different concept comes along. Right at the moment I can't think of any plausible candidates, but ... FOREACH pair IN HSTORE I don't actually see any problem with allowing that (or any other collection kind of object) with the same syntax as for arrays. The issue that we had with adding this to FOR was that it wasn't clear whether the expression after IN should be thought of as a source of rows, or as a scalar expression yielding a collection object that should get iterated through --- and because SQL allows sub-SELECT as a kind of expression, this was an actual formal ambiguity not just the sort of near-ambiguity that trips up users. If you will, it wouldn't have been clear whether to iterate vertically or horizontally. The direction that this proposal establishes is that FOR is for vertical iteration and FOREACH is for horizontal iteration; that is, the argument of FOREACH is a scalar expression in SQL terms, but it yields some kind of collection object that we are going to iterate through the members of. Given that understanding, I'm not seeing a need for the syntax to distinguish whether the collection object is an array, an hstore, or some other kind of collection. It's sufficient if we can determine this by examining the type of the expression. We would need an extra keyword if there were some third kind of iteration that was fundamentally different from either of these, but like I said, I don't see a plausible candidate. So right at the moment, I'm leaning to the position that we could do without the ARRAY keyword in FOREACH. If we do think of something else that could need its own keyword there, it's arguably going to be different enough that a different leading keyword would be a better idea anyhow. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plperlu problem with utf8
On Fri, Dec 17, 2010 at 08:30, David Fetter da...@fetter.org wrote: On Thu, Dec 16, 2010 at 07:24:46PM -0800, David Wheeler wrote: On Dec 16, 2010, at 6:39 PM, Alex Hunsaker wrote: Grr that should error out with Invalid server encoding, or worst case should return a length of 3 (it utf8 encoded 128 into 2 bytes instead of leaving it as 1). In this case the 333 causes perl store it internally as utf8. Well with SQL_ASCII anything goes, no? Anything except a byte that's all 0s :( Keyword being byte, right? Last time I checked 333 wont fit in a byte :P. In this case perl stores 333 as the utf8 that represents the unicode code point 333. Postgres uses whatever that internal representation is, so in our SQL_ASCII database we actually end up getting back utf8 which _is_ valid SQL_ASCII, but I wouldn't call it right. The behavior im aiming for is similar to what the built-in chr does: # SELECT chr(333); ERROR: requested character too large for encoding: 333 Also note this is just a simple test case, perl *could* elect to store completely ascii strings internally as utf8. In those cases we still do the wrong thing, that is we get back utf8ified bytes :( Although obviously from the lack of bug reports its quite uncommon. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On Tue, Dec 14, 2010 at 5:14 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 14, 2010 at 4:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 4:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, the first idea that comes to mind is to use a counter like the GetXLogRecPtrForTemp() counter I used for temp tables, but global, in shared memory. However, that's a bit problematic because if we store a value from that counter to LSN, it's possible that the counter overtakes the XLOG insert location, and you start to get xlog flush errors. We could avoid that if we added a new field to the GiST page header, and used that to store the value in the parent page instead of the LSN. That doesn't seem ideal, either, because now you're eating up some number of bytes per page in every GIST index just on the off chance that one of them is unlogged. On-disk compatibility seems problematic here as well. Good point. Given the foregoing discussion, I see only two possible paths forward here. 1. Just decide that that unlogged tables can't have GIST indexes, at least until someone figures out a way to make it work. That's sort of an annoying limitation, but I think we could live with it. 2. Write WAL records even though the GIST index is supposedly unlogged. We could either (1) write the usual XLOG_GIST_* records, and arrange to ignore them on replay when the relation in question is unlogged, or (2) write an XLOG_NOOP record to advance the current LSN. The latter sounds safer to me, but it will mean that the XLOG code for GIST needs three separate cases (temp, perm, unlogged). Either way we give up a significant chunk of the benefit of making the relation unlogged in the first place. Thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal : cross-column stats
On Fri, Dec 17, 2010 at 12:58 PM, Tomas Vondra t...@fuzzy.cz wrote: In the end, all they need to compute an estimate is number of distinct values for each of the columns (we already have that in pg_stats) and a number of distinct values for the group of columns in a query. They really don't need any multidimensional histogram or something like that. I haven't read the paper yet (sorry) but just off the top of my head, one possible problem here is that our n_distinct estimates aren't always very accurate, especially for large tables. As we've discussed before, making them accurate requires sampling a significant percentage of the table, whereas all of our other statistics can be computed reasonably accurately by sampling a fixed amount of an arbitrarily large table. So it's possible that relying more heavily on n_distinct could turn out worse overall even if the algorithm is better. Not sure if that's an issue here, just throwing it out there... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Given the foregoing discussion, I see only two possible paths forward here. 1. Just decide that that unlogged tables can't have GIST indexes, at least until someone figures out a way to make it work. That's sort of an annoying limitation, but I think we could live with it. +1 In the small subset of situations that need unlogged tables, I would think the subset of those that need gist is exceedingly small. Unless someone can come up with a use case that needs both unlogged and gist, I'd vote not to spend time on it. (And, if ever someone does come along with a really good use, then time can be put toward it). -Andy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Fri, Dec 17, 2010 at 1:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: David E. Wheeler da...@kineticode.com writes: On Dec 17, 2010, at 9:31 AM, Tom Lane wrote: Well, we did beat up Pavel over trying to shoehorn this facility into the existing FOR syntax, so I can hardly blame him for thinking this way. The question is whether we're willing to assume that FOREACH will be limited to iterating over arrays, meaning we'll be stuck with inventing yet another initial keyword if some other fundamentally different concept comes along. Right at the moment I can't think of any plausible candidates, but ... FOREACH pair IN HSTORE… I don't actually see any problem with allowing that (or any other collection kind of object) with the same syntax as for arrays. The issue that we had with adding this to FOR was that it wasn't clear whether the expression after IN should be thought of as a source of rows, or as a scalar expression yielding a collection object that should get iterated through --- and because SQL allows sub-SELECT as a kind of expression, this was an actual formal ambiguity not just the sort of near-ambiguity that trips up users. If you will, it wouldn't have been clear whether to iterate vertically or horizontally. The direction that this proposal establishes is that FOR is for vertical iteration and FOREACH is for horizontal iteration; that is, the argument of FOREACH is a scalar expression in SQL terms, but it yields some kind of collection object that we are going to iterate through the members of. Given that understanding, I'm not seeing a need for the syntax to distinguish whether the collection object is an array, an hstore, or some other kind of collection. It's sufficient if we can determine this by examining the type of the expression. We would need an extra keyword if there were some third kind of iteration that was fundamentally different from either of these, but like I said, I don't see a plausible candidate. So right at the moment, I'm leaning to the position that we could do without the ARRAY keyword in FOREACH. If we do think of something else that could need its own keyword there, it's arguably going to be different enough that a different leading keyword would be a better idea anyhow. Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. I am still wondering if there's a way to make something like FOR ELEMENT e IN a work. I suspect we'd be less likely to paint ourselves into a corner that way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Why don't we accept exponential format for integers?
Folks,
Is there any good reason that this works:
postgres=# select ('1e+01'::numeric)::integer
postgres-# ;
int4
--
10
But this doesn't?
postgres=# select '1e+01'::Integer
postgres-# ;
ERROR: invalid input syntax for integer: 1e+01
LINE 1: select '1e+01'::Integer
... or did we just never implement it?
--
-- Josh Berkus
PostgreSQL Experts Inc.
http://www.pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
We would need an extra keyword if there were some third kind of iteration that was fundamentally different from either of these, but like I said, I don't see a plausible candidate. So right at the moment, I'm leaning to the position that we could do without the ARRAY keyword in FOREACH. If we do think of something else that could need its own keyword there, it's arguably going to be different enough that a different leading keyword would be a better idea anyhow. Maybe I propage a higher verbosity than is necessary, but it descrease a risk so code will do some unexpected work. With ARRAY keyword we can verify so result of expression is really a array. Next advantage is a clean implementation now and in future. Without a auxilary keyword is necessary to wait on execution time. So now, when we have full control over syntax, we can protect self before FOR statement implementation's complexity. Personally - syntax without ARRAY keyword isn't significant problem for me. Just I think so using it wisely. Second semi argument for using ARRAY keyword is a verbosity of PL/pgSQL. So from this perspective a ARRAY should be minimally optional and ensure, so expr result will be really a array. But with a optional ARRAY keyword we leaving a simple enhancing in future (on parser level). Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Robert Haas robertmh...@gmail.com writes: Given the foregoing discussion, I see only two possible paths forward here. 1. Just decide that that unlogged tables can't have GIST indexes, at least until someone figures out a way to make it work. That's sort of an annoying limitation, but I think we could live with it. 2. Write WAL records even though the GIST index is supposedly unlogged. We could either (1) write the usual XLOG_GIST_* records, and arrange to ignore them on replay when the relation in question is unlogged, or (2) write an XLOG_NOOP record to advance the current LSN. The latter sounds safer to me, but it will mean that the XLOG code for GIST needs three separate cases (temp, perm, unlogged). Either way we give up a significant chunk of the benefit of making the relation unlogged in the first place. Thoughts? IIUC, the problem is that the bufmgr might think that a GIST NSN is an LSN that should affect when to force out a dirty buffer? What if we taught it the difference? We could for example dedicate a pd_flags bit to marking pages whose pd_lsn isn't actually an LSN. This solution would probably imply that all pages in the shared buffer pool have to have a standard PageHeaderData header, not just an LSN at the front as is assumed now. But that doesn't seem like a bad thing to me, unless maybe we were dumb enough to not use a standard page header in some of the secondary forks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plperlu problem with utf8
On Fri, Dec 17, 2010 at 11:51, Alex Hunsaker bada...@gmail.com wrote: Also note this is just a simple test case, perl *could* elect to store completely ascii strings internally as utf8. In those cases we still Erm... not ascii I mean bytes 127 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On 17.12.2010 21:04, Robert Haas wrote: Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. I am still wondering if there's a way to make something like FOR ELEMENT e IN a work. I suspect we'd be less likely to paint ourselves into a corner that way. As a side note, Oracle has FORALL, which is a kind of bulk update operation over a collection type. So whatever we choose, not FORALL... -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Robert Haas robertmh...@gmail.com: On Fri, Dec 17, 2010 at 1:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: David E. Wheeler da...@kineticode.com writes: On Dec 17, 2010, at 9:31 AM, Tom Lane wrote: Well, we did beat up Pavel over trying to shoehorn this facility into the existing FOR syntax, so I can hardly blame him for thinking this way. The question is whether we're willing to assume that FOREACH will be limited to iterating over arrays, meaning we'll be stuck with inventing yet another initial keyword if some other fundamentally different concept comes along. Right at the moment I can't think of any plausible candidates, but ... FOREACH pair IN HSTORE… I don't actually see any problem with allowing that (or any other collection kind of object) with the same syntax as for arrays. The issue that we had with adding this to FOR was that it wasn't clear whether the expression after IN should be thought of as a source of rows, or as a scalar expression yielding a collection object that should get iterated through --- and because SQL allows sub-SELECT as a kind of expression, this was an actual formal ambiguity not just the sort of near-ambiguity that trips up users. If you will, it wouldn't have been clear whether to iterate vertically or horizontally. The direction that this proposal establishes is that FOR is for vertical iteration and FOREACH is for horizontal iteration; that is, the argument of FOREACH is a scalar expression in SQL terms, but it yields some kind of collection object that we are going to iterate through the members of. Given that understanding, I'm not seeing a need for the syntax to distinguish whether the collection object is an array, an hstore, or some other kind of collection. It's sufficient if we can determine this by examining the type of the expression. We would need an extra keyword if there were some third kind of iteration that was fundamentally different from either of these, but like I said, I don't see a plausible candidate. So right at the moment, I'm leaning to the position that we could do without the ARRAY keyword in FOREACH. If we do think of something else that could need its own keyword there, it's arguably going to be different enough that a different leading keyword would be a better idea anyhow. Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. I am still wondering if there's a way to make something like FOR ELEMENT e IN a work. I suspect we'd be less likely to paint ourselves into a corner that way. I understand. But it is true too , so now is FOR statement implementation too rich. You can see to attachment with initial implementation. It's absolutely clean and simple. There is more valid ideas then one. One valid idea is so FOR statement is compatible with PL/SQL (what isn't true now :() and FOREACH can carry a pg's specific features. But I absolutely agree with you, so we can use a only one pg specific keyword. There are FOREACH (pg), FOR (shared with PL/SQL), FORALL (not implemented yet - use a PL/SQL). Pavel -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Robert Haas robertmh...@gmail.com writes: Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. All true, but in the absence of any plausible candidate for third or fourth or fifth types of iteration, this objection seems a bit thin. I am still wondering if there's a way to make something like FOR ELEMENT e IN a work. I suspect we'd be less likely to paint ourselves into a corner that way. I'm afraid that's only really feasible if you are willing for the second word to be a fully reserved word, so it can be distinguished from a plain variable name in that position. Which is probably worse than inventing multiple initial keywords. It doesn't seem to me that this would reduce the intellectual burden of remembering which syntax does what, anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Pavel Stehule pavel.steh...@gmail.com writes: Second semi argument for using ARRAY keyword is a verbosity of PL/pgSQL. So from this perspective a ARRAY should be minimally optional and ensure, so expr result will be really a array. But with a optional ARRAY keyword we leaving a simple enhancing in future (on parser level). No. If we are going to put a keyword there, it can't be optional. Making it optional would require it to be a fully reserved word --- and in the case of ARRAY, even that isn't good enough, because of the conflict with ARRAY[...] syntax. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On 17.12.2010 21:07, Tom Lane wrote: IIUC, the problem is that the bufmgr might think that a GIST NSN is an LSN that should affect when to force out a dirty buffer? What if we taught it the difference? We could for example dedicate a pd_flags bit to marking pages whose pd_lsn isn't actually an LSN. This solution would probably imply that all pages in the shared buffer pool have to have a standard PageHeaderData header, not just an LSN at the front as is assumed now. But that doesn't seem like a bad thing to me, unless maybe we were dumb enough to not use a standard page header in some of the secondary forks. I'm not very fond of expanding buffer manager's knowledge of the page layout. How about a new flag in the buffer desc, BM_UNLOGGED? There was some talk about skipping flushing of unlogged tables at checkpoints, I think we'd need BM_UNLOGGED for that anyway. Or I guess we could hang that behavior on the pd_flags bit too, but it doesn't seem like the right place for that information. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ps_status on fastpath
Excerpts from Tom Lane's message of vie dic 17 12:41:06 -0300 2010: Alvaro Herrera alvhe...@alvh.no-ip.org writes: I noticed that the fastpath code doesn't update ps_status, which would be harmless except that it leads to idle in transaction being logged in log_line_prefix for the command tag. Are there objections to applying this? Hm, what about pgstat_report_activity()? I wasn't sure about that, because of the overhead, but now that I look at it, it's supposed to be cheaper than changing the ps_status in some cases, so I guess there's no harm. The other argument to support the case that this should be done is that the docs suggest that you should use prepared statements instead, which do have the reporting overhead. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 17.12.2010 21:07, Tom Lane wrote: IIUC, the problem is that the bufmgr might think that a GIST NSN is an LSN that should affect when to force out a dirty buffer? What if we taught it the difference? We could for example dedicate a pd_flags bit to marking pages whose pd_lsn isn't actually an LSN. I'm not very fond of expanding buffer manager's knowledge of the page layout. How about a new flag in the buffer desc, BM_UNLOGGED? That could work too, if you can explain how the flag comes to be set without a bunch of ugliness all over the system. I don't want callers of ReadBuffer to have to supply the bit for example. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
2010/12/17 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: Second semi argument for using ARRAY keyword is a verbosity of PL/pgSQL. So from this perspective a ARRAY should be minimally optional and ensure, so expr result will be really a array. But with a optional ARRAY keyword we leaving a simple enhancing in future (on parser level). No. If we are going to put a keyword there, it can't be optional. Making it optional would require it to be a fully reserved word --- and in the case of ARRAY, even that isn't good enough, because of the conflict with ARRAY[...] syntax. yes, it's true Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ps_status on fastpath
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie dic 17 12:41:06 -0300 2010: Hm, what about pgstat_report_activity()? I wasn't sure about that, because of the overhead, but now that I look at it, it's supposed to be cheaper than changing the ps_status in some cases, so I guess there's no harm. Yeah, if we can afford a possible kernel call to set ps status, it doesn't seem like pgstat_report_activity should be a problem. I'm also of the opinion that more people look at pg_stat_activity than ps output these days. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On Fri, Dec 17, 2010 at 2:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: IIUC, the problem is that the bufmgr might think that a GIST NSN is an LSN that should affect when to force out a dirty buffer? What if we taught it the difference? We could for example dedicate a pd_flags bit to marking pages whose pd_lsn isn't actually an LSN. This solution would probably imply that all pages in the shared buffer pool have to have a standard PageHeaderData header, not just an LSN at the front as is assumed now. But that doesn't seem like a bad thing to me, unless maybe we were dumb enough to not use a standard page header in some of the secondary forks. Ah, interesting idea. visibilitymap.c has this: #define MAPSIZE (BLCKSZ - MAXALIGN(SizeOfPageHeaderData)) and fsm_internals.h has this: #define NodesPerPage (BLCKSZ - MAXALIGN(SizeOfPageHeaderData) - \ offsetof(FSMPageData, fp_nodes)) ...which seems to imply, at least on a quick look, that we might be OK, as far as the extra forks go. I have a vague recollection that the documentation says somewhere that a full page header on every page isn't required. And I'm a little worried about my ability to track down every place in the system that might get broken by a change in this area. What do you think about committing the unlogged tables patch initially without support for GIST indexes, and working on fixing this as a separate commit? Another possibly-useful thing about mandating a full page header for every page is that it might give us a way of avoiding unnecessary full page writes. As I wrote previously: However, I think we can avoid this too, by allocating an additional bit in pd_flags, PD_FPI. Instead of emitting an FPI when the old LSN precedes the redo pointer, we'll emit an FPI when the FPI bit is set (in which case we'll also clear the bit) OR when the old LSN precedes the redo pointer. Upon emitting a WAL record that is torn-page safe (such as a freeze or all-visible record), we'll pass a flag to XLogInsert that arranges to suppress FPIs, bump the LSN, and set PD_FPI. That way, if the page is touched again before the next checkpoint by an operation that does NOT suppress FPI, one will be emitted then. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Robert Haas robertmh...@gmail.com writes: Another possibly-useful thing about mandating a full page header for every page is that it might give us a way of avoiding unnecessary full page writes. As I wrote previously: Could we do that via a bufmgr status bit, instead? Heikki's idea has the merit that it actually reduces bufmgr's knowledge of page headers, rather than increasing it (since a buffer marked UNLOGGED would need no assumptions at all about its content). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On Fri, Dec 17, 2010 at 2:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 17.12.2010 21:07, Tom Lane wrote: IIUC, the problem is that the bufmgr might think that a GIST NSN is an LSN that should affect when to force out a dirty buffer? What if we taught it the difference? We could for example dedicate a pd_flags bit to marking pages whose pd_lsn isn't actually an LSN. I'm not very fond of expanding buffer manager's knowledge of the page layout. How about a new flag in the buffer desc, BM_UNLOGGED? That could work too, if you can explain how the flag comes to be set without a bunch of ugliness all over the system. I don't want callers of ReadBuffer to have to supply the bit for example. That's easy enough to solve - ReadBuffer() takes a Relation as an argument, so you can easily check RelationNeedsWAL(reln). I guess the question is whether it's right to conflate table is unlogged with LSN is fake. It's not immediately obvious to me that those concepts are isomorphic, although though the reverse isn't obvious to me either. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On Fri, Dec 17, 2010 at 2:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Another possibly-useful thing about mandating a full page header for every page is that it might give us a way of avoiding unnecessary full page writes. As I wrote previously: Could we do that via a bufmgr status bit, instead? Heikki's idea has the merit that it actually reduces bufmgr's knowledge of page headers, rather than increasing it (since a buffer marked UNLOGGED would need no assumptions at all about its content). That was my first thought, but it doesn't work. The buffer could be evicted from shared_buffers and read back in. If a checkpoint intervenes meanwhile, we're OK, but otherwise you fail to emit an otherwise-needed FPI. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why don't we accept exponential format for integers?
Josh Berkus j...@agliodbs.com writes: postgres=# select '1e+01'::Integer postgres-# ; ERROR: invalid input syntax for integer: 1e+01 I have never heard of any programming system anywhere that accepts such a syntax for integers (assuming it distinguishes integers from other numbers at all). I'm not excited about being the first. Why does this error surprise you? It doesn't seem particularly different from arguing that 1.000 should be considered an integer, which strikes me as a seriously bad idea. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ps_status on fastpath
Excerpts from Tom Lane's message of vie dic 17 16:25:17 -0300 2010: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie dic 17 12:41:06 -0300 2010: Hm, what about pgstat_report_activity()? I wasn't sure about that, because of the overhead, but now that I look at it, it's supposed to be cheaper than changing the ps_status in some cases, so I guess there's no harm. Yeah, if we can afford a possible kernel call to set ps status, it doesn't seem like pgstat_report_activity should be a problem. I'm also of the opinion that more people look at pg_stat_activity than ps output these days. Well, actually, if the functions are cheap, presumably they aren't going to stay for much time in either status report; what I'm after is fixing the log_line_prefix stuff (%i I think it is). -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Fri, Dec 17, 2010 at 2:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. All true, but in the absence of any plausible candidate for third or fourth or fifth types of iteration, this objection seems a bit thin. Well, Heikki just pointed out one that Oracle supports, so that makes at least #3... I am still wondering if there's a way to make something like FOR ELEMENT e IN a work. I suspect we'd be less likely to paint ourselves into a corner that way. I'm afraid that's only really feasible if you are willing for the second word to be a fully reserved word, so it can be distinguished from a plain variable name in that position. What if we cheat and peak ahead an extra token? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On 17.12.2010 21:32, Robert Haas wrote: I guess the question is whether it's right to conflate table is unlogged with LSN is fake. It's not immediately obvious to me that those concepts are isomorphic, although though the reverse isn't obvious to me either. The buffer manager only needs to know if it has to flush the WAL before writing the page to disk. The flag just means that the buffer manager never needs to do that for this buffer. You're still free to store a real LSN there if you want to, it just won't cause any WAL flushes. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 2:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. All true, but in the absence of any plausible candidate for third or fourth or fifth types of iteration, this objection seems a bit thin. Well, Heikki just pointed out one that Oracle supports, so that makes at least #3... If you posit that we might someday wish to support what Oracle is doing there, it seems to me to be a precedent for using a different first keyword, not for what you're suggesting. I'm not arguing that we might want to duplicate Oracle's syntax; only that if it's going to be cited as a precedent that we consider what it's actually a precedent for. I'm afraid that's only really feasible if you are willing for the second word to be a fully reserved word, so it can be distinguished from a plain variable name in that position. What if we cheat and peak ahead an extra token? plpgsql's parser is rickety enough that I don't have a lot of confidence in its ability to do things that way. In particular, there's too much knowledge at the lexer level instead of the grammar --- you'd have to have a way of keeping the lexer from returning T_DATUM in this one particular context, even if element happened to match some variable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
On Fri, Dec 17, 2010 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 2:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Unfortunately, there are likely to be a limited number of such keywords available. While I agree it's helpful to have a clear distinction between what FOR does and what FOREACH does, it's wholly conventional here and won't be obvious without careful reading of the documentation. If we had FOR and FOREACH and FOREVERY and, uh, FORGET, it'd quickly become notational soup. All true, but in the absence of any plausible candidate for third or fourth or fifth types of iteration, this objection seems a bit thin. Well, Heikki just pointed out one that Oracle supports, so that makes at least #3... If you posit that we might someday wish to support what Oracle is doing there, it seems to me to be a precedent for using a different first keyword, not for what you're suggesting. I'm not arguing that we might want to duplicate Oracle's syntax; only that if it's going to be cited as a precedent that we consider what it's actually a precedent for. I don't quite follow what you're getting at here. My goal was to try to think of something more mnemonic than FOREACH, and I thought something involving the word element or array would do the trick. The problem is only to find a place to put it that's before the word IN. But maybe that's hopeless and we should just go with FOREACH. I'm afraid that's only really feasible if you are willing for the second word to be a fully reserved word, so it can be distinguished from a plain variable name in that position. What if we cheat and peak ahead an extra token? plpgsql's parser is rickety enough that I don't have a lot of confidence in its ability to do things that way. Bummer. Rickety is not good. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 17.12.2010 21:32, Robert Haas wrote: I guess the question is whether it's right to conflate table is unlogged with LSN is fake. It's not immediately obvious to me that those concepts are isomorphic, although though the reverse isn't obvious to me either. The buffer manager only needs to know if it has to flush the WAL before writing the page to disk. The flag just means that the buffer manager never needs to do that for this buffer. You're still free to store a real LSN there if you want to, it just won't cause any WAL flushes. Yeah. I think that BM_UNLOGGED might be a poor choice for the flag name, just because it overstates what the bufmgr needs to assume. It might be better to reverse the flag sense, and have a new flag that *is* set if the page contains an LSN that we have to check against WAL. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
On Fri, Dec 17, 2010 at 3:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 17.12.2010 21:32, Robert Haas wrote: I guess the question is whether it's right to conflate table is unlogged with LSN is fake. It's not immediately obvious to me that those concepts are isomorphic, although though the reverse isn't obvious to me either. The buffer manager only needs to know if it has to flush the WAL before writing the page to disk. The flag just means that the buffer manager never needs to do that for this buffer. You're still free to store a real LSN there if you want to, it just won't cause any WAL flushes. Yeah. I think that BM_UNLOGGED might be a poor choice for the flag name, just because it overstates what the bufmgr needs to assume. It might be better to reverse the flag sense, and have a new flag that *is* set if the page contains an LSN that we have to check against WAL. I was actually thinking of adding BM_UNLOGGED even before this discussion, because that would allow unlogged buffers to be excluded from non-shutdown checkpoints. We could add two flags with different semantics that take on, under present rules, the same value, but I'd be disinclined to burn the extra bit without a concrete need. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: FOREACH-IN-ARRAY (probably for 9.2?)
Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: plpgsql's parser is rickety enough that I don't have a lot of confidence in its ability to do things that way. Bummer. Rickety is not good. Agreed, but it's not entirely the parser's fault: the language definition is pretty d*mn bogus to start with. Read the comments for the for_variable production, and ask yourself whether you really want to inject even more difficult-to-disambiguate cases right there. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables vs. GIST
Robert Haas robertmh...@gmail.com writes: On Fri, Dec 17, 2010 at 3:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah. I think that BM_UNLOGGED might be a poor choice for the flag name, just because it overstates what the bufmgr needs to assume. I was actually thinking of adding BM_UNLOGGED even before this discussion, because that would allow unlogged buffers to be excluded from non-shutdown checkpoints. We could add two flags with different semantics that take on, under present rules, the same value, but I'd be disinclined to burn the extra bit without a concrete need. bufmgr is currently using eight bits out of a 16-bit flag field, and IIRC at least five of those have been there since the beginning. So our accretion rate is something like one bit every four years. I think not being willing to use two bits to describe two unrelated behaviors is penny-wise and pound-foolish --- bufmgr is already complicated enough, let's not add useless barriers to readability. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
