Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays
Given: CREATE DOMAIN int_array AS int[]; The operator [] works fine in 4.1beta1: SELECT (ARRAY[1,2,3]::int_array)[1]; proving that int_array is an array type with element type int. It is inconsistent that other array functions and operators don't work. On Mon, 2011-05-09 at 23:32 -0400, Tom Lane wrote: So we basically had three alternatives to make it better: #1 downcast to the array type, which would possibly silently break applications that were relying on the function result being considered of the domain type I do not think of this as Downcasting int_array to int[] but as allowing an ANYARRAY to match int_array which is an array type. Since no cast is logically required, the return type is the same as the first argument type, as expected and as PostgreSQL has done for some time. #2 re-apply domain checks on the function result, which would be a performance hit and possibly again result in unobvious breakage If the function result is a new value then nothing is being re-applied. If it is an existing value of the domain type which was passed in or extracted from a data structure, then the domain checks have already been applied. This is a red herring. #3 explicitly break it by throwing a parse error until you downcast (and then upcast the function result if you want) I realize that #3 is a bit unpleasant, but are either of the other two better? At least #3 shows you where you need to check for problems. Wrapping most (but not all) of your array operations in downcasts and upcasts is horrible. There is another issue that wasn't really mentioned in the previous thread, which is that if we are matching a domain-over-array to a function's ANYARRAY argument, what exactly should be allowed to match to ANYELEMENT --- or if the function returns ANYELEMENT, what should the imputed result type be? Since PostgreSQL allows indexing of the domain type, we already know the answer. I don't even get why there is confusion abou the element type of an array. AFAICS it's impossible to give an answer to that without effectively deciding that function argument matching smashes the domain to its base type (the array type). It's not very clear what's the point of a domain type if every operation on it is going to neglect its domain-ness. Yes, what is the point of neglecting the domain-ness of a domain type by being forced to downcast it to an unchecked type before (some) array operations? If a value is being constructed of a domain-type which has constraints, check them. When I don't want the security of a domain type I can cast it to its representation type before I passed it, but it seems bizarre to be required to do such a thing! I did read the previous threads some time ago. They seemed mostly to be concerned with discussing the internal implementation of these matters and the typmod feature (which I still don't understand). The internal algorithms and deta structures which PostgreSQL uses to internally represent SQL types and operations are a weak justification for PostgreSQL's behavior - they can be changed if they are wrong. I am still hoping to get rid of my domains which are arrays when PostgreSQL supports arrays of elements which are of domain types. Could we at least defer this change until that is done? _Greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays
E.1.2.2. Casting * Tighten casting checks for domains based on arrays (Tom Lane) When a domain is based on an array type,..., such a domain type is no longer allowed to match an anyarray parameter of a polymorphic function, except by explicitly downcasting it to the base array type. This will require me to add hundreds of casts to my code. I do not get how this will Tighten casting checks. It will certainly not tighten my code! Could you explain how it is good to not be able to do array operations with a type which is an array? BTW: All of my DOMAINs which are array types exist because of PostgreSQL's inability to infer array types for DOMAINs, so I have lots of code like this: CREATE DOMAIN foo_ids AS integer; CREATE DOMAIN foo_id_arrays AS integer[]; I would love to be able to simply use foo_ids[] instead of having to have the second DOMAIN foo_id_arrays. If there is some value which I'm missing in the above Tightening, perhaps it could be put in *after* PostgreSQL were given the ability to understand foo_ids[] as an array of foo_ids. Thanks, _Greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Server Programming Interface underspecified in 4.1beta1
The documentation for the Server Programming Interface is underspecified such that it is impossible to create real extensions using only the documented interface. For example, in the example 43.5 http://www.postgresql.org/docs/9.1/static/spi-examples.html a variable of type SPITupleTable is being dereferenced as tupdesc-natts but this is not documented. The SPI documentation suggests studying the contributed extension code for further examples but that code is full of hundreds of features which are not in the SPI at all. My own SPI code uses dozens of macros and field names which I found in existing source and in conversations on the mailing lists. I'm always nervous when a new release comes out because the API I am using is not official and might therefore change and break my code. This is especially bad since much of the code is not type-safe and problems will tend to be hidden by all of the casts in the pre-ISO-C-style macro code. It would be good if the Server Programming Interface were sufficiently documented that most of the contributed extensions which ship with PostgreSQL and most of the SPI-based modules in the backend were using ONLY the documented features of the SPI. Macros hiding casts and typedefs hiding void * types should replaced with inline functions and pointers to specific strong types. To be very clear: Yes, I can always rummage around in the include files and source to find out how to do things IN THIS RELEASE. But I can't expect PostgreSQL development to avoid breaking the idioms I happen upon, nor can the developers write unit tests to ensure that proper extensions using the SPI will not break. I would like to be able to program to a C or C++ SPI which is clean, complete and type-safe. I am good at reading API documentation in C or C++ and would be happy to review any proposed improvements. _Greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xml2 still essential for us
On Wed, 2010-01-13 at 00:34 -0500, Tom Lane wrote: Are you sufficiently excited about it to fix its memory management issues? On Wed, 2010-01-13 at 09:12 -0800, Josh Berkus wrote: Given your interest in XML2, would you like to be come a maintainer of the module? I'm wonderfully flattered by these requests. At the moment I'm more than a year behind my schedule for the PostgreSQL-based system I'm currently developing! When that's been released (as Free Software, etc.) I will certainly be looking for how I can contribute back to the PostgreSQL project. One of my challenges has been the poverty of the documented Server Programming API - which existing extensions seem to casually disregard in favor of using all kinds of undocumented features like direct function calls and caching - all of which I avoided because I want my extensions to work in future PostgreSQL versions. I'm imagining that some of these mechanisms could be added to the documented API - that's a task which I could help with. When I release my code I want to contribute back first by helping with things which made my project unnecessarily difficult. I hope that PostgreSQL's XML is in great shape by then, but we'll see! PostgreSQL is a great tool and y'all are doing great work in maintaining and improving it. The community has also been very helpful since I got over my shyness about asking for help. _Greg J. Greg Davidson Ancient Mariner of the C -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] xml2 still essential for us
Reporting as requested, there are two things our projects continue to need xml2 for, and a third which we conceptually need xml2 for: (1) We need some way to do xslt processing. (2) We need the equivalent of xml_is_well_formed(text), e.g. a documented way to call XMLPARSE speculatively and get a NULL or be able to efficiently catch an error, etc. (3) My third point is a bit subtle. The xml2 package is clearly just a set of operations on text. The PostgreSQL xml datatype might RIGHT NOW be internally represented as text, but I don't think we're supposed to think of it that way. The official XML datatype is free to evolve into one of the more sophisticated XML storage and processing formats, at the expense of a bit more overhead at XMLPARSE time. The advantage of the xml2 package is that it says upfront: this is just a way to treat text as xml. It is useful precisely when one is doing a mixture of text and xml processing steps on data, and perhaps also when generating short-lived xml values. (4) In conclusion, I hope that PostgreSQL will keep xml2 or something similar even when the XSLT and speculative parsing issues have been addressed. Thanks for your consideration, _Greg J. Greg Davidson -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] array casts that worked in 8.3 no longer work
Dear PostgreSQL Hackers, I have a couple of user defined types, which have reference semantics. One is more specialized, so by Liskov I can upcast both single values and arrays of values. -- no problem: CREATE CAST (derived_refs AS base_refs) WITHOUT FUNCTION AS IMPLICIT; -- ok with pg-8.3, error with pg-8.4-b2: CREATE CAST (_derived_refs AS _base_refs) WITHOUT FUNCTION AS IMPLICIT; -- ERROR: 42P17: array data types are not binary-compatible -- LOCATION: CreateCast, functioncmds.c:1648 Reading the comment in functioncmds: /* * We know that composite, enum and array types are never binary- * compatible with each other. They all have OIDs embedded in them. */ I'm guessing that what I was doing was not safe although I have been getting away with it quite nicely for several years. Unfortunately I do this all over the place (I have a master ref type and lots of specializations of it). Is there an efficient (and maybe even easy) way to legally convert such arrays? Thanks for any suggestions you may have, _Greg J. Greg Davidson P.S. If you want a more complete example to refer to, here it is: CREATE TYPE base_refs; CREATE OR REPLACE FUNCTION base_ref_in(cstring) RETURNS base_refs AS 'pg-array-problem.so' LANGUAGE 'c' STRICT; CREATE OR REPLACE FUNCTION base_ref_out(base_refs) RETURNS cstring AS 'pg-array-problem.so' LANGUAGE 'c' STRICT; CREATE TYPE base_refs ( INTERNALLENGTH = 8, ALIGNMENT = double, input = base_ref_in, output = base_ref_out, PASSEDBYVALUE ); CREATE TYPE derived_refs; CREATE OR REPLACE FUNCTION derived_ref_in(cstring) RETURNS derived_refs AS 'pg-array-problem.so' LANGUAGE 'c' STRICT; CREATE OR REPLACE FUNCTION derived_ref_out(derived_refs) RETURNS cstring AS 'pg-array-problem.so' LANGUAGE 'c' STRICT; CREATE TYPE derived_refs ( INTERNALLENGTH = 8, ALIGNMENT = double, input = derived_ref_in, output = derived_ref_out, PASSEDBYVALUE ); -- ** Safe Upcasts -- no problem: CREATE CAST (derived_refs AS base_refs) WITHOUT FUNCTION AS IMPLICIT; -- error with pg-8.4-b2: CREATE CAST (_derived_refs AS _base_refs) WITHOUT FUNCTION AS IMPLICIT; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] 8.4beta[12] set returning function fails -- was O.K. with 8.3
Dear PostgreSQL Hackers, Through PostgreSQL 8.3, both of the following functions worked, using generate_series(array_lower($1, 1), array_upper($1, 1)) i instead of generate_subscripts($1, 1). With PostgreSQL 8.4, both are accepted, but only the second one works, regardless of whether I use generate_subscripts or the old way. The error is shown. What's going on? Thanks, _Greg CREATE OR REPLACE FUNCTION array_to_set(ANYARRAY) RETURNS SETOF RECORD AS $$ SELECT i AS index, $1[i] AS value FROM generate_subscripts($1, 1) i $$ LANGUAGE SQL STRICT IMMUTABLE; COMMENT ON FUNCTION array_to_set(ANYARRAY) IS 'returns the array as a set of RECORD(index, value) pairs'; SELECT array_to_set(ARRAY['one', 'two']); -- BREAKS IN PG 8.4 beta1 beta2, vis: -- -- ERROR: 0A000: set-valued function called in context that cannot accept a set -- CONTEXT: SQL function array_to_set during startup -- LOCATION: fmgr_sql, functions.c:644 CREATE OR REPLACE FUNCTION array_to_list(ANYARRAY) RETURNS SETOF ANYELEMENT AS $$ SELECT $1[i] FROM generate_subscripts($1, 1) i $$ LANGUAGE SQL STRICT IMMUTABLE; COMMENT ON FUNCTION array_to_list(ANYARRAY) IS 'returns the array as a set of its elements from lowest to highest; - can we guarantee the values will be seen in order???'; SELECT array_to_list(ARRAY['one', 'two']); -- Works great, vis: -- --array_to_list -- --- -- one -- two -- (2 rows) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] blocking referencing system catalogs in 8.4 breaks my code
In testing my software on postgresql-8.4beta[12] I got ERROR: 42501: permission denied: pg_namespace is a system catalog LOCATION: ATAddForeignKeyConstraint, tablecmds.c:4599 from my code which has a column schema_name name NOT NULL REFERENCES pg_namespace ON DELETE CASCADE this despite PostgreSQL happily accepting: GRANT REFERENCES ON pg_namespace TO greg; I found the change in tablecmds.c and a mention of it in PostgreSQL Weekly News - May 17 2009, yet I don't understand why referencing a system table in this manner should be a problem - is it? I'm trying to have certain rows of my tables go away if certain schemas are dropped. Is there a preferred way to do this? If this regression is retained, it needs to be documented in the reference manual AND I think that an attempt to GRANT REFERENCES privileges to a system table should not be allowed. BTW, I'm loving most of what I see in 8.4, _Greg J. Greg Davidson -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] blocking referencing system catalogs in 8.4 breaks my code
Hi Josh, I think that any special features common to all on system tables would go nicely near the top of 44.1 System Catalogs Overview. That plus a specific error message, not just a PERMISSION DENIED, so that someone like me can put the error message in the PostgreSQL.org search box to find relevant documentation. If referencing system tables was always unreliable, I'm glad that it is now forbidden! I've now worked around the problem by creating my own schemas table along with functions create_schema and drop_schema which insert and delete a referenceable row in my own table as they create and drop a system schema. I've been delighted to see a number of irregularities and exceptions disappear as PostgreSQL evolves, leaving the system simpler as well as better. Perhaps at some point someone more knowledgeable than me will see a way to eliminate this limitation. Information wants to be referenceable. Thanks for the help, _Greg On Fri, 2009-06-05 at 15:34 -0700, Josh Berkus wrote: Greg, this despite PostgreSQL happily accepting: GRANT REFERENCES ON pg_namespace TO greg; We should probably error out on this. I found the change in tablecmds.c and a mention of it in PostgreSQL Weekly News - May 17 2009, yet I don't understand why referencing a system table in this manner should be a problem - is it? System tables have special properties which can make modifications to them behave non-transactionally. So it's never safe to have a trigger on a system table. I'm frankly surprised that this worked for you before. I'm trying to have certain rows of my tables go away if certain schemas are dropped. Is there a preferred way to do this? I can't off the top of my head think of a preferred way. If this regression is retained, it needs to be documented in the reference manual AND I think that an attempt to GRANT REFERENCES privileges to a system table should not be allowed. I agree that GRANT REFERENCES should error. Do you have a suggestion where in the manual would be a good place to warn people that FKs against the system tables are a bad idea? -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
