Re: [HACKERS] Specification for Trusted PLs?
On Fri, May 21, 2010 at 7:25 PM, Magnus Hagander mag...@hagander.net wrote: On Fri, May 21, 2010 at 12:22 PM, David Fetter da...@fetter.org wrote: On Fri, May 21, 2010 at 11:57:33AM -0400, Magnus Hagander wrote: On Fri, May 21, 2010 at 11:55 AM, Josh Berkus j...@agliodbs.com wrote: So, here's a working definition: 1) cannot directly read or write files on the server. 2) cannot bind network ports To make that more covering, don't yu really need something like cannot communicate with outside processes? These need to be testable conditions, and new tests need to get added any time we find that we've missed something. Making this concept fuzzier is exactly the wrong direction to go. Well, the best way to define what a trusted language can do is to define a *whitelist* of what it can do, not a blacklist of what it can't do. That's the only way to get a complete definition. It's then up to the implementation step to figure out how to represent that in the form of tests. Yes, PL/Perl is following this approach. For a whitelist see plperl_opmask.h (generated by plperl_opmask.pl at build phase). -- Alexey Klyukin www.CommandPrompt.com The PostgreSQL Company - Command Prompt, Inc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] unexpected query behaviour after i change parser code
All, Lately i have play with parser-part. I was try to make valid query command without using FROM clause, so far it's work. I know this modification will make all query that using FROM clause failed, for example /df command. But normal or simple select statement so far is work. Now before my question, this what i do to make query without FROM clause work : 1) change src/backend/parser/gram.y at simple_select: delete from_clause 2) change src/backend/parser/parse_relation.c at function warnAutoRange, comment or delete if (!add_missing_from) part and change the else above to if (add_missing_from). Ok this my test result to customer and item table : - select id_item,name from item; -- failed, because there is from clause (failed like i expected) - select item.id_item, item.name; -- work, like i expected - select id_item,name; -- failed, with error : column id_item does not exist (failed like i expected) - select item.id_item,customer.fname; -- work, the data not acurate though because there is no joined atribut - select item.id_item,customer.fname where item.id_item=customer.id; -- work, normaly - select item.id,item; -- work, the result was concanted in item column. (i expected this query was failed). Try many combination including using more than one table with previous test, the result always work ONLY IF i put table_name.colId first. My question : 1) Can someone explain why my last test it's work? 2) Why PostgreSQL won't query my 3rd test? Considering my last test it's work. Thank You. *) Btw if you try my modification make sure you already have the data. I havent try any command yet, but i assume SET, or CREATE will failed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages
Josh Berkus j...@agliodbs.com writes: From a discussion at dinner at pgcon, I wanted to send this to the list for people to poke holes in it: Somebody (I think Joe or Heikki) poked a big hole in this last night at the Royal Oak. Although the scheme would get rid of the need to replace old XIDs with FrozenXid, it does not get rid of the need to set hint bits before you can truncate CLOG. So in your example of an insert-only table that's probably never read again, there's still a minimum of one update visit required on every old page. Now that's still better than two update visits ... but we could manage that already, just by tweaking vacuum's heuristics about when to freeze vs when to set hint bits. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages
Somebody (I think Joe or Heikki) poked a big hole in this last night at the Royal Oak. Although the scheme would get rid of the need to replace old XIDs with FrozenXid, it does not get rid of the need to set hint bits before you can truncate CLOG. So in your example of an insert-only table that's probably never read again, there's still a minimum of one update visit required on every old page. Now that's still better than two update visits ... but we could manage that already, just by tweaking vacuum's heuristics about when to freeze vs when to set hint bits. Yeah, someone pointed that out to me too and suggested that a freeze map was the better solution. I still think there's something we can do with pages on the visibility map but I'll have to think about it some more. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] release notes
On Mon, May 17, 2010 at 11:34:37AM -0400, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
Why do the release notes say this, under plperl:
* PL/Perl subroutines are now given names (Tim Bunce)
This is for the use of profiling and code coverage tools. DIDN'T
THEY HAVE NAMES BEFORE?
If whoever put this in the release notes had bothered to ask I am sure
we would have been happy to explain.
You don't need to complain, just fix it. The point of the comment is
that more explanation is needed.
I think you can just delete it. It's too esoteric to be worth noting.
Tim.
p.s. It also turned to be insufficiently useful for NYTProf since it
doesn't also update some internals to record the 'filename' and line
number range of the sub. So PostgreSQL::PLPerl::NYTProf works around
that by wrapping mkfuncsrc() to edit the generated perl code to include
a subname in the usual sub $name { ... } style. I may offer a patch
for 9.1 to to make it work that way.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unexpected query behaviour after i change parser code
On Sat, May 22, 2010 at 9:21 AM, Mohammad Heykal Abdillah heykal.abdil...@gmail.com wrote: All, Lately i have play with parser-part. I was try to make valid query command without using FROM clause, so far it's work. I know this modification will make all query that using FROM clause failed, for example /df command. But normal or simple select statement so far is work. Now before my question, this what i do to make query without FROM clause work : 1) change src/backend/parser/gram.y at simple_select: delete from_clause 2) change src/backend/parser/parse_relation.c at function warnAutoRange, comment or delete if (!add_missing_from) part and change the else above to if (add_missing_from). Ok this my test result to customer and item table : - select id_item,name from item; -- failed, because there is from clause (failed like i expected) - select item.id_item, item.name; -- work, like i expected - select id_item,name; -- failed, with error : column id_item does not exist (failed like i expected) - select item.id_item,customer.fname; -- work, the data not acurate though because there is no joined atribut - select item.id_item,customer.fname where item.id_item=customer.id; -- work, normaly - select item.id,item; -- work, the result was concanted in item column. (i expected this query was failed). Try many combination including using more than one table with previous test, the result always work ONLY IF i put table_name.colId first. My question : 1) Can someone explain why my last test it's work? In standard PostgreSQL, select item from item is valid SQL. It returns a single column whose value is a record containing all the columns from the item table. I suspect something similar is happening in your case. 2) Why PostgreSQL won't query my 3rd test? Considering my last test it's work. I'm not sure which test you're referring to here, but all of your results look like about what would happen with adding_missing_from set. Which brings me to another point: I'm not really sure what you're trying to accomplish with this modification, considering that adding_missing_from sounds like it does about what you want, but without breaking nearly as much stuff. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Specification for Trusted PLs?
2010/5/21 Jan Wieck janwi...@yahoo.com: The original idea was that a trusted language does not allow an unprivileged user to gain access to any object or data, he does not have access to without that language. This does not include data transformation functionality, like string processing or the like. As long as the user had legitimate access to the input datum, then every derived form thereof is OK. I find the current doc enough, add this prose from Jan as a comment might help people perhaps. Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Cédric Villemain -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages
Josh Berkus j...@agliodbs.com writes: Somebody (I think Joe or Heikki) poked a big hole in this last night at the Royal Oak. Although the scheme would get rid of the need to replace old XIDs with FrozenXid, it does not get rid of the need to set hint bits before you can truncate CLOG. So in your example of an insert-only table that's probably never read again, there's still a minimum of one update visit required on every old page. Now that's still better than two update visits ... but we could manage that already, just by tweaking vacuum's heuristics about when to freeze vs when to set hint bits. Yeah, someone pointed that out to me too and suggested that a freeze map was the better solution. I still think there's something we can do with pages on the visibility map but I'll have to think about it some more. It occurred to me on the flight home that maybe we could salvage something from this if there were some mechanism that caused hint bits to get set before the page got written out from shared buffers the first time. This assumes that you have enough slack in shared-buffer space that the transactions that touched a particular page all commit or abort before the page first gets flushed to disk. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages
On 5/22/2010 9:16 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Somebody (I think Joe or Heikki) poked a big hole in this last night at the Royal Oak. Although the scheme would get rid of the need to replace old XIDs with FrozenXid, it does not get rid of the need to set hint bits before you can truncate CLOG. So in your example of an insert-only table that's probably never read again, there's still a minimum of one update visit required on every old page. Now that's still better than two update visits ... but we could manage that already, just by tweaking vacuum's heuristics about when to freeze vs when to set hint bits. Yeah, someone pointed that out to me too and suggested that a freeze map was the better solution. I still think there's something we can do with pages on the visibility map but I'll have to think about it some more. It occurred to me on the flight home that maybe we could salvage something from this if there were some mechanism that caused hint bits to get set before the page got written out from shared buffers the first time. This assumes that you have enough slack in shared-buffer space that the transactions that touched a particular page all commit or abort before the page first gets flushed to disk. At least the background writer should have a few spare cycles to look over a to be flushed page before writing it. Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unexpected query behaviour after i change parser code
On Sab, 2010-05-22 at 14:42 -0400, Robert Haas wrote: Ok this my test result to customer and item table : - select id_item,name from item; -- failed, because there is from clause (failed like i expected) - select item.id_item, item.name; -- work, like i expected - select id_item,name; -- failed, with error : column id_item does not exist (failed like i expected) - select item.id_item,customer.fname; -- work, the data not acurate though because there is no joined atribut - select item.id_item,customer.fname where item.id_item=customer.id; -- work, normaly - select item.id,item; -- work, the result was concanted in item column. (i expected this query was failed). Try many combination including using more than one table with previous test, the result always work ONLY IF i put table_name.colId first. My question : 1) Can someone explain why my last test it's work? In standard PostgreSQL, select item from item is valid SQL. It returns a single column whose value is a record containing all the columns from the item table. I suspect something similar is happening in your case. Hmm.., i know that select item from item is valid SQL. But since in my case from cause was deleted. Shouldnt select item.id_item,item; failed? Since select id_item,name; was also failed. What i am not understand why it's always work if i put table_name.ColId first. In the case select item from item PostgreSQL rely on from_clause to find the relation/table right? So after i delete the from_clause in the case select item.id_item,item; i thought PostgreSQL will also lost it ability to find where those ColId came form, thus it will fail all query. Instead, it work in PostgreSQL. So what's it the meaning from_clause in PostgreSQL after all? 2) Why PostgreSQL won't query my 3rd test? Considering my last test it's work. I'm not sure which test you're referring to here, but all of your results look like about what would happen with adding_missing_from set. I am refering to select item.id_item,item; (sorry it was writen select item.id,item;) test. Which brings me to another point: I'm not really sure what you're trying to accomplish with this modification, considering that adding_missing_from sounds like it does about what you want, but without breaking nearly as much stuff. I am trying to make some kind automate join relation without have to explicitly declare the join relation key. Example : select item_id,fname; in my modified query will be eqivalen with SQL query. select item.id_item,customer.fname from item,fname where item.id_item=customer.id Ah, yes my conversion will be do after raw_parsertree was forming by lex and yacc. Thank You. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] beta testing - pg_upgrade bug fix - double free
On Fri, May 21, 2010 at 10:11 PM, Pavel Stehule pavel.steh...@gmail.com wrote: it fixes bug pg_upgrade(13359) malloc: *** error for object 0x801600: non-page-aligned, non-allocated pointer being freed *** set a breakpoint in malloc_error_break to debug arget 03:31 /usr/local/src/postgresql/contrib/pg_upgrade git diff . diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c index 31f12fb..f989229 100644 --- a/contrib/pg_upgrade/check.c +++ b/contrib/pg_upgrade/check.c @@ -154,7 +154,6 @@ issue_warnings(migratorContext *ctx, char *sequence_script_file_name) ctx-new.bindir, ctx-new.port, sequence_script_file_name, ctx-logfile); unlink(sequence_script_file_name); - pg_free(sequence_script_file_name); check_ok(ctx); } by Jan Matousek This patch looks correct to me, although I haven't had a chance to test it yet. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unexpected query behaviour after i change parser code
On Sat, May 22, 2010 at 9:33 PM, Mohammad Heykal Abdillah heykal.abdil...@gmail.com wrote: My question : 1) Can someone explain why my last test it's work? In standard PostgreSQL, select item from item is valid SQL. It returns a single column whose value is a record containing all the columns from the item table. I suspect something similar is happening in your case. Hmm.., i know that select item from item is valid SQL. But since in my case from cause was deleted. Shouldnt select item.id_item,item; failed? Since select id_item,name; was also failed. Well, it's hard for me to speculate about what your code might do. I think your best bet is to fire up gdb and maybe stick in some debugging printfs and see if you can figure out what's happening. Which brings me to another point: I'm not really sure what you're trying to accomplish with this modification, considering that adding_missing_from sounds like it does about what you want, but without breaking nearly as much stuff. I am trying to make some kind automate join relation without have to explicitly declare the join relation key. Example : select item_id,fname; in my modified query will be eqivalen with SQL query. select item.id_item,customer.fname from item,fname where item.id_item=customer.id Ah, yes my conversion will be do after raw_parsertree was forming by lex and yacc. It's probably not possible to do this well in general. You might want to think about writing some kind of preprocessor or query generator that would rewrite your queries like this for you before sending them to the database, rather than modifying PostgreSQL itself. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Specification for Trusted PLs?
On Sat, May 22, 2010 at 4:53 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 2010/5/21 Jan Wieck janwi...@yahoo.com: The original idea was that a trusted language does not allow an unprivileged user to gain access to any object or data, he does not have access to without that language. This does not include data transformation functionality, like string processing or the like. As long as the user had legitimate access to the input datum, then every derived form thereof is OK. I find the current doc enough, add this prose from Jan as a comment might help people perhaps. Yeah, Jan's description is very clear and to the point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] Move 'long long' check to c.h
Greetings,
While reviewing bfba40e2c7b3909d3de13bd1b83b7e85fa8dfec2 (mmm, we like
git diff -p), I noted that c.h is already included by both extern.h
and ecpg.header through postgres_fe.h. Given this and that we're
already doing alot of similar #define's there (unlike in those other
files), I felt c.h was a more appropriate place. Putting it in c.h
also means we don't have to duplicate that code.
Patch attached.
Thanks,
Stephen
diff --git a/src/include/c.h b/src/include/c.h
index ea31635..d729e04 100644
*** a/src/include/c.h
--- b/src/include/c.h
*** typedef unsigned long long int uint64;
*** 284,289
--- 284,294
#error must have a working 64-bit integer datatype
#endif
+ /* Do we know the C99 data type long long? */
+ #if defined(LLONG_MIN) || defined(LONGLONG_MIN) || defined(HAVE_LONG_LONG_INT_64)
+ #define HAVE_LONG_LONG 1
+ #endif
+
/* Decide if we need to decorate 64-bit constants */
#ifdef HAVE_LL_CONSTANTS
#define INT64CONST(x) ((int64) x##LL)
diff --git a/src/interfaces/ecpg/ecpglib/extern.h b/src/interfaces/ecpg/ecpglib/extern.h
index e3a7c82..7a5259f 100644
*** a/src/interfaces/ecpg/ecpglib/extern.h
--- b/src/interfaces/ecpg/ecpglib/extern.h
***
*** 13,23
#include limits.h
#endif
- /* Do we know the C99 data type long long? */
- #if defined(LLONG_MIN) || defined(LONGLONG_MIN) || defined(HAVE_LONG_LONG_INT_64)
- #define HAVE_LONG_LONG 1
- #endif
-
enum COMPAT_MODE
{
ECPG_COMPAT_PGSQL = 0, ECPG_COMPAT_INFORMIX, ECPG_COMPAT_INFORMIX_SE
--- 13,18
diff --git a/src/interfaces/ecpg/preproc/ecpg.header b/src/interfaces/ecpg/preproc/ecpg.header
index dcc87a1..6b05776 100644
*** a/src/interfaces/ecpg/preproc/ecpg.header
--- b/src/interfaces/ecpg/preproc/ecpg.header
***
*** 7,17
#include extern.h
#include unistd.h
- /* Do we know the C99 datatype long long? */
- #if defined(LLONG_MIN) || defined(LONGLONG_MIN) || defined(HAVE_LONG_LONG_INT_64)
- #define HAVE_LONG_LONG 1
- #endif
-
/* Location tracking support --- simpler than bison's default */
#define YYLLOC_DEFAULT(Current, Rhs, N) \
do { \
--- 7,12
signature.asc
Description: Digital signature
[HACKERS] mapping object names to role IDs
Suppose you have an object name as a CString and you want to convert it to an OID. The method of doing this varies widely depending on the object type: oid = get_database_oid(name); oid = get_tablespace_oid(name); oid = GetForeignDataWrapperOidByName(name, true); oid = GetForeignServerOidByName(name, true); oid = LookupFuncNameTypeNames(name, args, true); oid = LookupAggNameTypeNames(name, args, true); oid = LookupFuncNameTypeNames(name, args, true); oid = LookupOperNameTypeNames(name, args, true); oid = GetSysCacheOid1(LANGNAME, CStringGetDatum(name)); oid = GetSysCacheOid1(NAMESPACENAME, CStringGetDatum(name)); oid = GetSysCacheOid1(AMNAME, CStringGetDatum(name)); oid = get_roleid(name); oid = GetConstraintByName(reloid, name); oid = FindConversionByName(list_make1(name)); oid = TSDictionaryGetDictid(list_make1(name), true); oid = TSTemplateGetTmplid(list_make1(name), true); oid = TSConfigGetCfgid(list_make1(name), true); If you'd like to throw an error if the object doesn't exist, then you can change the true parameter in the above examples to false, where it exists, or you can use get_roleid_checked for roles. For the types where a direct syscache lookup is used, you get to write the check yourself. For constraints, GetConstraintByName throws an error anyway; there's no equivalent that doesn't. Some other object types - like rules and casts - have even more complex logic that is sometimes duplicated in multiple places; and others (like tablespaces) that have convenience functions still manage to duplicate the logic anyway. Long story short, this is kind of a mess. Looking at the different cases, there seem to be three main cases: 1. Objects that just have one name, like databases and procedural languages. 2. Objects that have possibly-qualified names, like conversions and text search dictionaries. 3. Objects that have both names and argument, like functions and operators. There's enough complexity about the stuff in category #3 that I'm inclined to leave it alone, but try to make the other stuff more standardized. What I would propose is that we create a new source file somewhere (maybe utils/cache), move all of the other functions of this type there, give them standardized names, and provide them all with an argument that specifies whether an error is to be thrown if the object doesn't exist. Begin bikeshedding here. I suggest that the names be get_object-type_oid and that they take two parameters, either a List * for possibly-qualified names (a little wonky, but it's what we do now) or a char * for unqualified names, and a boolean indicating whether to throw an error if the object isn't found. Thus: Oid get_object-type_oid(List *qualname, bool missingok); -or- Oid get_object-type_oid(char *name, bool missingok); Thus get_database_oid and get_tablespace_oid would remain unchanged except for taking a second argument, get_roleid and get_roleid_checked would merge, and all of the others would change to match that style. Thoughts? P.S. Purple. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mapping object names to role IDs
And of course I meant for the subject line to be mapping object names to OIDs, not role IDs. Sigh. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
