Re: [HACKERS] WIP patch for Todo Item : Provide fallback_application_name in contrib/pgbench, oid2name, and dblink
On Wed, Apr 16, 2014 at 11:01:56PM -0400, Bruce Momjian wrote: On Tue, Apr 1, 2014 at 10:32:01AM -0400, Tom Lane wrote: Adrian Vondendriesch adrian.vondendrie...@credativ.de writes: I patched the function conninfo_array_parse() which is used by PQconnectStartParams to behave like PQsetdbLogin. The patch also contains a document patch which clarify unspecified parameters. I see no documentation update here. I'm also fairly concerned about the implication that no connection parameter, now or in future, can ever have an empty string as the correct value. I thought about this. We have never needed PQsetdbLogin() to handle zero-length strings specially in all the years we used it, so I am not sure why we would ever need PQconnectdbParams() to handle it. I am thinking we should make PQconnectdbParams() handle zero-length strings the same as NULL, and document it. Attached is a slightly-modified version of Adrian Vondendriesch's patch. Patch applied. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [DOCS] Docs incorrectly claiming equivalence between show and pg_settings
On Wed, Feb 12, 2014 at 03:54:31PM +0100, Stefan Seifert wrote: Hi! http://www.postgresql.org/docs/devel/static/sql-show.html claims: Also, the pg_settings system view produces the same information. This is not entirely correct. On IRC I was told, that the view only contains settings set from C, not user defined settings defined by the SET command: timemngt=# set my.test = 'test'; SET timemngt=# show my.test; ┌─┐ │ my.test │ ├─┤ │ test│ └─┘ (1 row) timemngt=# select * from pg_settings where name = 'my.test'; timemngt=# Including this rather obscure bit of information might help another user down the road. [ Thread reposted to hackers as this is perhaps a coding issue.] This is an interesting report. You are correct that there are several places in the docs that say that SHOW and pg_settings display the same information, and even state that pg_settings shows _more_ information than SHOW. However, in the case of custom variables, you are right that pg_settings doesn't show custom variables. I have found the place in the code where we do that using GUC_NO_SHOW_ALL. I believe the original reason for this is that custom variables were designed to modify plugin languages and therefore not useful for people doing SHOW ALL, but if you specifically asked for it, it would show it to you. Because pg_settings is built as a view, the API doesn't really have an ALL mode. We can do a few things: 1 show custom variables in SHOW ALL and pg_settings 2 show custom and other non-SHOW-ALL variables in pg_settings 3 document this restriction I am not sure which approach is best. I am attaching a patch that does #1. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c new file mode 100644 index 15020c4..c0f9880 *** a/src/backend/utils/misc/guc.c --- b/src/backend/utils/misc/guc.c *** add_placeholder_variable(const char *nam *** 3929,3936 gen-context = PGC_USERSET; gen-group = CUSTOM_OPTIONS; ! gen-short_desc = GUC placeholder variable; ! gen-flags = GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE | GUC_CUSTOM_PLACEHOLDER; gen-vartype = PGC_STRING; /* --- 3929,3936 gen-context = PGC_USERSET; gen-group = CUSTOM_OPTIONS; ! gen-short_desc = Custom variable.; ! gen-flags = GUC_NOT_IN_SAMPLE | GUC_CUSTOM_PLACEHOLDER; gen-vartype = PGC_STRING; /* -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [DOCS] Docs incorrectly claiming equivalence between show and pg_settings
Bruce Momjian br...@momjian.us writes: However, in the case of custom variables, you are right that pg_settings doesn't show custom variables. That is entirely intentional: the reason we do not show placeholder variables in pg_settings is that we have no accurate information about them, and so everything pg_settings might show would be fabricated, and probably wrong, information. Once the placeholder has been replaced by a proper declaration of the GUC variable, it will be shown (with correct info), unless of course the proper declaration includes GUC_NO_SHOW_ALL. What this gets back to is that manually created custom variables are an abuse of a loophole that was only meant to allow postgresql.conf to set a parameter belonging to an extension module that hasn't been loaded yet. If we want to actually support such variables, there should be a way to properly declare one, including giving its type and other properties ... and ideally we'd not let you set one without having declared it, though it's not quite clear how to enforce that without breaking the parameter-placeholder case. We can do a few things: 1 show custom variables in SHOW ALL and pg_settings 2 show custom and other non-SHOW-ALL variables in pg_settings 3 document this restriction or (4) fix the lack of a declaration capability. But both (1) and (2) are horrid ideas. There are good reasons for having invented GUC_NO_SHOW_ALL, and just trashing it is not the answer. As for (3), I might be wrong, but I don't think the documentation mentions the possibility of abusing SET this way at all. Restrictions in undocumented quasi-features are likewise undocumented. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?
On Sat, Apr 19, 2014 at 3:37 AM, Bruce Momjian br...@momjian.us wrote: One thing that I discussed with Merlin offline and am now concerned about is how will the actual eviction work. We cannot traverse the entire list and then find all the buffers with refcount 0 and then do another traversal to find the oldest one. I thought if there was memory pressure the clock sweep would run and we wouldn't have everything at the max counter access value. Hmm, I see your point. With that applicable as well, I feel that the clocksweep counting/logical clock system shall be useful when deciding between multiple candidates for eviction. At worst, it can serve to replace the gettimeofday() calls. One thing I have thought of with ideas and inputs from Joshua Yanowski offline is that we can probably have a maxheap which is on the logical clock age of buffers. Each time clocksweep sees a buffer whose refcount has become zero, it will push the buffer into minheap. This can be a new representation of freelist or a new additional data structure. This still does not solve the problem of seeing the entire list by the clocksweep, even if that makes the eviction process O(1) with the addition of the maxheap. I am working on a PoC patch but am stuck on this point. My current approach sees the entire shared buffers list to search for any candidate buffers. Another thing that is a pain point here is the concurrency and locking overheads of introducing a new data structure. Can the existing buffer header spinlock handle this problem or is it hitting the granularity of the spinlock too much? I see some blockers for this idea still. Nevertheless, the point of clocksweep counts as logical clocks seems to be promising,atleast intuitively. Thoughts and comments? Regards, Atri -- Regards, Atri *l'apprenant*
Re: [HACKERS] Re: [DOCS] Docs incorrectly claiming equivalence between show and pg_settings
On Sat, Apr 19, 2014 at 01:38:16PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: However, in the case of custom variables, you are right that pg_settings doesn't show custom variables. That is entirely intentional: the reason we do not show placeholder variables in pg_settings is that we have no accurate information about them, and so everything pg_settings might show would be fabricated, and probably wrong, information. Once the placeholder has been replaced by a proper declaration of the GUC variable, it will be shown (with correct info), unless of course the proper declaration includes GUC_NO_SHOW_ALL. Oh, I had forgotten these were placeholders that get replaced later. What this gets back to is that manually created custom variables are an abuse of a loophole that was only meant to allow postgresql.conf to set a parameter belonging to an extension module that hasn't been loaded yet. If we want to actually support such variables, there should be a way to properly declare one, including giving its type and other properties ... and ideally we'd not let you set one without having declared it, though it's not quite clear how to enforce that without breaking the parameter-placeholder case. We can do a few things: 1 show custom variables in SHOW ALL and pg_settings 2 show custom and other non-SHOW-ALL variables in pg_settings 3 document this restriction or (4) fix the lack of a declaration capability. But both (1) and (2) are horrid ideas. There are good reasons for having invented GUC_NO_SHOW_ALL, and just trashing it is not the answer. As for (3), I might be wrong, but I don't think the documentation mentions the possibility of abusing SET this way at all. Restrictions in undocumented quasi-features are likewise undocumented. OK, let's wait to see if anyone else complains --- if so, we can document the SHOW ALL and pg_settings behavior in the placeholders section. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Avoiding deeply nested AND/OR trees in the parser
On Wed, Feb 26, 2014 at 10:21:03PM -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Tue, Feb 25, 2014 at 01:15:09PM -0500, Tom Lane wrote: Really if we wanted to fix this issue we'd need to hack up transformAExprAnd/transformAExprOr so that they recognized nested ANDs/ORs and flattened them on the spot. This might not be a bad idea, but it's starting to look like more than a quick hack patch. Reminds me of this work: http://www.postgresql.org/message-id/flat/CABwTF4XJKN1smMjHv_O-QzTpokqSjHBouMWVw-E8kyb2bC=_...@mail.gmail.com http://www.postgresql.org/message-id/flat/cafj8prdd9qtyoy0cbpoodr-hfj1xambuxwoxazg0kyvtvau...@mail.gmail.com Oh, I'd forgotten about that thread. I never particularly liked the patch as presented: like Robert, I thought it far too complicated. My inclination would just be to tweak the parser enough so that a simple list of ANDs or ORs (ie, a left-deep raw parse tree) gets flattened. The most likely bet for making that happen in an uncomplicated way would be to alter gram.y's processing: if we had the productions for AND/OR notice whether their left inputs were already AND/OR clauses, they could extend the argument lists instead of building nested clauses. The reason the proposed patch is so complicated is it's trying to avoid recursing while handling a fundamentally recursive data structure, and that's just the hard way to do it. We do need to look at whether there are any implications for ruleutils and other places, though. Where are we on this? Is it being kept for 9.5? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Avoiding deeply nested AND/OR trees in the parser
Bruce Momjian br...@momjian.us writes: On Wed, Feb 26, 2014 at 10:21:03PM -0500, Tom Lane wrote: Oh, I'd forgotten about that thread. I never particularly liked the patch as presented: like Robert, I thought it far too complicated. My inclination would just be to tweak the parser enough so that a simple list of ANDs or ORs (ie, a left-deep raw parse tree) gets flattened. The most likely bet for making that happen in an uncomplicated way would be to alter gram.y's processing: if we had the productions for AND/OR notice whether their left inputs were already AND/OR clauses, they could extend the argument lists instead of building nested clauses. The reason the proposed patch is so complicated is it's trying to avoid recursing while handling a fundamentally recursive data structure, and that's just the hard way to do it. We do need to look at whether there are any implications for ruleutils and other places, though. Where are we on this? Is it being kept for 9.5? I think we rejected the patch-as-presented, and no one's bothered to create a new one, which suggests that the problem isn't all that important ... I suspect the gram.y change I suggest above would be about a ten-line patch. What makes it less than completely trivial is the need to chase down all the downstream implications, such as whether ruleutils would need any work, and whether anything else is expecting parser output to contain only binary clauses. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] PATCH: Allow empty targets in unaccent dictionary
Hi,
Currently, unaccent extension only allows replacing one source character
with one or more target characters. In Arabic, Hebrew and possibly other
languages, diacritics are standalone characters that are being added to
normal letters. To use unaccent dictionary for these languages, we need
to allow empty targets to remove diacritics instead of replacing them.
The attached patch modfies unaacent.c so that dictionary parser uses
zero-length target when the line has no target.
Best Regards,
Mohammad Alhashash
diff --git a/contrib/unaccent/unaccent.c b/contrib/unaccent/unaccent.c
old mode 100644
new mode 100755
index a337df6..4e72829
--- a/contrib/unaccent/unaccent.c
+++ b/contrib/unaccent/unaccent.c
@@ -58,7 +58,9 @@ placeChar(TrieChar *node, unsigned char *str, int lenstr,
char *replaceTo, int r
{
curnode-replacelen = replacelen;
curnode-replaceTo = palloc(replacelen);
- memcpy(curnode-replaceTo, replaceTo, replacelen);
+ /* palloc(0) returns a valid address, not NULL */
+ if (replaceTo) /* memcpy() is undefined for NULL
pointers*/
+ memcpy(curnode-replaceTo, replaceTo,
replacelen);
}
}
else
@@ -105,10 +107,10 @@ initTrie(char *filename)
while ((line = tsearch_readline(trst)) != NULL)
{
/*
-* The format of each line must be src trg
where src and trg
+* The format of each line must be src [trg]
where src and trg
* are sequences of one or more non-whitespace
characters,
* separated by whitespace. Whitespace at
start or end of
-* line is ignored.
+* line is ignored. If no trg added, a
zero-length string is used.
*/
int state;
char *ptr;
@@ -160,6 +162,13 @@ initTrie(char *filename)
}
}
+ /* if no trg (loop stops at state 1 or 2), use
zero-length target */
+ if (state == 1 || state == 2)
+ {
+ trglen = 0;
+ state = 5;
+ }
+
if (state = 3)
rootTrie = placeChar(rootTrie,
(unsigned char *) src, srclen,
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Avoiding deeply nested AND/OR trees in the parser
On Sat, Apr 19, 2014 at 05:50:17PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Wed, Feb 26, 2014 at 10:21:03PM -0500, Tom Lane wrote: Oh, I'd forgotten about that thread. I never particularly liked the patch as presented: like Robert, I thought it far too complicated. My inclination would just be to tweak the parser enough so that a simple list of ANDs or ORs (ie, a left-deep raw parse tree) gets flattened. The most likely bet for making that happen in an uncomplicated way would be to alter gram.y's processing: if we had the productions for AND/OR notice whether their left inputs were already AND/OR clauses, they could extend the argument lists instead of building nested clauses. The reason the proposed patch is so complicated is it's trying to avoid recursing while handling a fundamentally recursive data structure, and that's just the hard way to do it. We do need to look at whether there are any implications for ruleutils and other places, though. Where are we on this? Is it being kept for 9.5? I think we rejected the patch-as-presented, and no one's bothered to create a new one, which suggests that the problem isn't all that important ... I suspect the gram.y change I suggest above would be about a ten-line patch. What makes it less than completely trivial is the need to chase down all the downstream implications, such as whether ruleutils would need any work, and whether anything else is expecting parser output to contain only binary clauses. OK, thanks for the feedback. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] DISCARD ALL (Again)
On Fri, Apr 18, 2014 at 12:00 PM, Peter Eisentraut pete...@gmx.net wrote: On 4/17/14, 8:24 PM, Tom Lane wrote: We could in fact implement #2, I imagine, by destroying and recreating the entire language interpreter. So I could imagine implementing a DISCARD INTERPRETERS kind of command that would zap the current interpreter(s) for whichever PL languages happened to feel like cooperating with the command. More generally, any extension could maintain any kind of cross-call state. plproxy, dblink, pgmemcache come to mind. A general hook into DISCARD might be doable, but then it's not clear how to categorize this into DISCARD subcommands. I proposed some like that in a previous message [1]. Regards, [1] http://www.postgresql.org/message-id/cafcns+p+pmfhkiptf1xbz3neg8d0mtenb021h2fug049333...@mail.gmail.com -- Fabrízio de Royes Mello Consultoria/Coaching PostgreSQL Timbira: http://www.timbira.com.br Blog sobre TI: http://fabriziomello.blogspot.com Perfil Linkedin: http://br.linkedin.com/in/fabriziomello Twitter: http://twitter.com/fabriziomello
