On 10/11/2016 08:56 AM, Pavel Stehule wrote:
2016-10-11 7:49 GMT+02:00 Heikki Linnakangas <hlinn...@iki.fi>:
Unfortunately there are cases that are fundamentally ambiguous.
create type comptype as (intarray int[]);
create function array_return() returns comptype[] as $$
return [[[[1]]]];
$$ language plpython;
What does the function return? It could be two-dimension array of
comptype, with a single-dimension intarray, or a single-dimension comptype,
with a two-dimension intarray.
We could resolve it for simpler cases, but not the general case. The
simple cases would probably cover most things people do in practice. But if
the distinction between a tuple and a list feels natural to Python
programmers, I think it would be more clear in the long run to have people
adjust their applications.
I agree. The distinction is natural - and it is our issue, so we don't
distinguish strongly.
Ok, let's do that then. Here is a patch set that does that. The first is
the main patch. The second patch adds some code to give a hint, if you
do that thing that whose behavior changed. That code isn't very pretty,
but I think a good error message is absolutely required, if we are to
make this change. Does anyone have better suggestions on how to catch
the common cases of that?
Please review. Are the docs and the error messages now clear enough on
this? We'll need a mention in the release notes too, when it's time for
that.
- Heikki
>From 945b33bd77fae0605ae9ff9d9282ab543cb16ec8 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakan...@iki.fi>
Date: Fri, 14 Oct 2016 11:51:45 +0300
Subject: [PATCH 1/2] Support multi-dimensional arrays in PL/python.
Multi-dimensional arrays can now be used as arguments to a PL/python function
(used to throw an error), and they can be returned as nested Python lists.
This makes a backwards-incompatible change to the handling of composite
types in arrays. Previously, you could return an array of composite types
as "[[col1, col2], [col1, col2]]", but now that is interpreted as a two-
dimensional array. Composite types in arrays must now be returned as
Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1, col2),
(col1, col2)]".
To avoid breaking backwards-compatibility, when not necessary, () is still
accepted for arrays at the top-level, but it is always treated as a
single-dimensional array. Likewise, [] is still accepted for composite types,
when they are not in an array. Update the documentation to recommend using []
for arrays, and () for composite types, with a mention that those other things
are also accepted.
This needs to be mentioned in the release notes.
Alexey Grishchenko, Dave Cramer and me.
Discussion: <CAH38_tmbqwaUyKs9yagyRra=smat45fpbxk1pmtycm0tyxg...@mail.gmail.com>
---
doc/src/sgml/plpython.sgml | 41 +++-
src/pl/plpython/expected/plpython_composite.out | 6 +-
src/pl/plpython/expected/plpython_types.out | 151 ++++++++++++-
src/pl/plpython/expected/plpython_types_3.out | 151 ++++++++++++-
src/pl/plpython/plpy_typeio.c | 280 ++++++++++++++++++++----
src/pl/plpython/sql/plpython_composite.sql | 6 +-
src/pl/plpython/sql/plpython_types.sql | 86 ++++++++
7 files changed, 658 insertions(+), 63 deletions(-)
diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index bb69c75..4639778 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -451,13 +451,13 @@ $$ LANGUAGE plpythonu;
<para>
SQL array values are passed into PL/Python as a Python list. To
return an SQL array value out of a PL/Python function, return a
- Python sequence, for example a list or tuple:
+ Python list:
<programlisting>
CREATE FUNCTION return_arr()
RETURNS int[]
AS $$
-return (1, 2, 3, 4, 5)
+return [1, 2, 3, 4, 5]
$$ LANGUAGE plpythonu;
SELECT return_arr();
@@ -467,6 +467,34 @@ SELECT return_arr();
(1 row)
</programlisting>
+ Multidimensional arrays are passed into PL/Python as nested Python lists.
+ A 2-dimensional array is a list of lists, for example. When returning
+ a multi-dimensional SQL array out of a PL/Python function, the inner
+ lists at each level must all be of the same size. For example:
+
+<programlisting>
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+</programlisting>
+
+ Other Python sequences, like tuples, are also accepted for
+ backwards-compatibility with PostgreSQL versions 9.6 and below, when
+ multi-dimensional arrays were not supported. However, they are always
+ treated as one-dimensional arrays, because they are ambiguous with
+ composite types. For the same reason, when a composite type is used in a
+ multi-dimensional array, it must be represented by a tuple, rather than a
+ list.
+ </para>
+ <para>
Note that in Python, strings are sequences, which can have
undesirable effects that might be familiar to Python programmers:
@@ -541,14 +569,19 @@ CREATE TYPE named_value AS (
CREATE FUNCTION make_pair (name text, value integer)
RETURNS named_value
AS $$
- return [ name, value ]
- # or alternatively, as tuple: return ( name, value )
+ return ( name, value )
+ # or alternatively, as tuple: return [ name, value ]
$$ LANGUAGE plpythonu;
</programlisting>
To return a SQL null for any column, insert <symbol>None</symbol> at
the corresponding position.
</para>
+ <para>
+ When an array of composite types is returned, it cannot be returned as a list,
+ because it is ambiguous whether the Python list represents a composite type,
+ or another array dimension.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/pl/plpython/expected/plpython_composite.out b/src/pl/plpython/expected/plpython_composite.out
index 0ef0c21..f23dccc 100644
--- a/src/pl/plpython/expected/plpython_composite.out
+++ b/src/pl/plpython/expected/plpython_composite.out
@@ -465,13 +465,13 @@ SELECT * FROM changing_test();
-- tables of composite types
CREATE FUNCTION composite_types_table(OUT tab table_record[], OUT typ type_record[] ) RETURNS SETOF record AS $$
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..5243aab 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,133 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..1592219 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,133 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..56cfdd1 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -45,6 +45,8 @@ static PyObject *PLyBytes_FromBytea(PLyDatumToOb *arg, Datum d);
static PyObject *PLyString_FromDatum(PLyDatumToOb *arg, Datum d);
static PyObject *PLyObject_FromTransform(PLyDatumToOb *arg, Datum d);
static PyObject *PLyList_FromArray(PLyDatumToOb *arg, Datum d);
+static PyObject *PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p);
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
@@ -53,6 +55,9 @@ static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *pl
static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
+static void PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem);
/* conversion from Python objects to composite Datums (used by triggers and SRFs) */
static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string);
@@ -631,43 +636,105 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
- int i;
+ int ndim;
+ int *dims;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ Assert(ndim < MAXDIM);
+
+ /*
+ * We will iterate the Postgres array in the physical order its stored in
+ * the datum. For 3-dimensional array the order of iteration would be the
+ * following: first you start with [0,0,0] elements through [0,0,k], then
+ * [0,1,0] till [0,1,k] till [0,m,k], then [1,0,0] till [1,0,k] till
+ * [1,m,k], and so on.
+ *
+ * In Python, there is no multi-dimensional lists as such, but they are
+ * represented as a list of lists. So a 3-d array of [n,m,k] element is a
+ * list of n m-element arrays, each element of which is k-element array.
+ * PLyList_FromArray_recurse() builds the Python list for a single
+ * dimension, and recurses for the next inner dimension.
+ */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
+
+ return PLyList_FromArray_recurse(elm, dims, ndim, 0,
+ &dataptr, &bitmap, &bitmask);
+}
+
+static PyObject *
+PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p)
+{
+ int i;
+ PyObject *list;
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ list = PyList_New(dims[dim]);
- for (i = 0; i < length; i++)
+ if (dim < ndim - 1)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* Outer dimension. Recurse for each inner slice. */
+ for (i = 0; i < dims[dim]; i++)
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ PyObject *sublist;
+
+ sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
+ dataptr_p, bitmap_p, bitmask_p);
+ PyList_SET_ITEM(list, i, sublist);
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
+ }
+ else
+ {
+ /*
+ * Innermost dimension. Fill the list with the values from the array
+ * for this slice.
+ */
+ char *dataptr = *dataptr_p;
+ bits8 *bitmap = *bitmap_p;
+ int bitmask = *bitmask_p;
+
+ for (i = 0; i < dims[dim]; i++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(list, i, Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(list, i, elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */ )
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
+
+ *dataptr_p = dataptr;
+ *bitmap_p = bitmap;
+ *bitmask_p = bitmask;
}
return list;
@@ -864,39 +931,108 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
- int len;
- int lbs;
+ int64 len;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int currelem;
+ Datum rv;
+ PyObject *pyptr = plrv;
+ PyObject *next;
Assert(plrv != Py_None);
- if (!PySequence_Check(plrv))
- PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
+ /*
+ * Determine the number of dimensions, and their sizes
+ */
+ ndim = 0;
+ len = 1;
- len = PySequence_Length(plrv);
- elems = palloc(sizeof(*elems) * len);
- nulls = palloc(sizeof(*nulls) * len);
+ Py_INCREF(plrv);
- for (i = 0; i < len; i++)
+ for (;;)
{
- PyObject *obj = PySequence_GetItem(plrv, i);
+ if (!PySequence_Check(pyptr))
+ break;
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* composite type */
+ if (PyTuple_Check(pyptr))
+ break;
+
+ /* string */
+ if (PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr))
+ break;
+
+ if (ndim == MAXDIM)
+ PLy_elog(ERROR, "number of array dimensions exceeds the maximum allowed (%d)", MAXDIM);
+
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "cannot determine sequence length for function return value");
+
+ if (dims[ndim] > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed");
+
+ len *= dims[ndim];
+ if (len > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed");
+
+ if (dims[ndim] == 0)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ /* empty sequence */
+ break;
}
- Py_XDECREF(obj);
+
+ ndim++;
+
+ next = PySequence_GetItem(pyptr, 0);
+ Py_XDECREF(pyptr);
+ pyptr = next;
+ }
+ Py_XDECREF(pyptr);
+
+ /*
+ * Check for zero dimensions. This happens if the object is a tuple or a
+ * string, rather than a list, or is not a sequence at all. We don't map
+ * tuples or strings to arrays in general, but in the first level, be
+ * lenient, for historical reasons. So if the object is a sequence of any
+ * kind, treat it as a one-dimensional array.
+ */
+ if (ndim == 0)
+ {
+ if (!PySequence_Check(plrv))
+ PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
+
+ ndim = 1;
+ len = dims[0] = PySequence_Length(plrv);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ /*
+ * Traverse the Python lists, in depth-first order, and collect all the
+ * elements at the bottom level into 'elems'/'nulls' arrays.
+ */
+ elems = palloc(sizeof(Datum) * len);
+ nulls = palloc(sizeof(bool) * len);
+ currelem = 0;
+ PLySequence_ToArray_recurse(arg->elm, plrv,
+ dims, ndim, 0,
+ elems, nulls, &currelem);
+
+ for (i = 0; i < ndim; i++)
+ lbs[i] = 1;
+
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
@@ -908,6 +1044,56 @@ PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
return rv;
}
+/*
+ * Helper function for PLySequence_ToArray. Traverse a Python list of lists in
+ * depth-first order, storing the elements in 'elems'.
+ */
+static void
+PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem)
+{
+ int i;
+
+ if (PySequence_Length(list) != dims[dim])
+ PLy_elog(ERROR,
+ "multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int) PySequence_Length(list), dims[dim]);
+
+ if (dim < ndim - 1)
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *sublist = PySequence_GetItem(list, i);
+
+ PLySequence_ToArray_recurse(elm, sublist, dims, ndim, dim + 1,
+ elems, nulls, currelem);
+ Py_XDECREF(sublist);
+ }
+ }
+ else
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *obj = PySequence_GetItem(list, i);
+
+ if (obj == Py_None)
+ {
+ nulls[*currelem] = true;
+ elems[*currelem] = (Datum) 0;
+ }
+ else
+ {
+ nulls[*currelem] = false;
+ elems[*currelem] = elm->func(elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ (*currelem)++;
+ }
+ }
+}
+
static Datum
PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
diff --git a/src/pl/plpython/sql/plpython_composite.sql b/src/pl/plpython/sql/plpython_composite.sql
index 473342c..72a35c3 100644
--- a/src/pl/plpython/sql/plpython_composite.sql
+++ b/src/pl/plpython/sql/plpython_composite.sql
@@ -169,13 +169,13 @@ SELECT * FROM changing_test();
-- tables of composite types
CREATE FUNCTION composite_types_table(OUT tab table_record[], OUT typ type_record[] ) RETURNS SETOF record AS $$
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..802dd41 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,80 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +318,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +342,18 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
+
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
--
2.9.3
>From d952cfc08e2f161fa99f6ab1ce568cce411de017 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakan...@iki.fi>
Date: Fri, 14 Oct 2016 11:52:05 +0300
Subject: [PATCH 2/2] Give a hint, when [] is incorrectly used for a composite
type in array.
That used to be accepted, so let's try to give a hint to users on why
their PL/python functions no longer work.
---
src/pl/plpython/plpy_cursorobject.c | 3 +-
src/pl/plpython/plpy_exec.c | 7 +-
src/pl/plpython/plpy_spi.c | 3 +-
src/pl/plpython/plpy_typeio.c | 117 +++++++++++++++++++++--------
src/pl/plpython/plpy_typeio.h | 17 +++--
src/pl/plpython/sql/plpython_composite.sql | 9 +++
6 files changed, 116 insertions(+), 40 deletions(-)
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 0e17a03..e682bfe 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -240,7 +240,8 @@ PLy_cursor_plan(PyObject *ob, PyObject *args)
plan->values[j] =
plan->args[j].out.d.func(&(plan->args[j].out.d),
-1,
- elem);
+ elem,
+ false);
}
PG_CATCH();
{
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 25e4744..fa583fa 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -245,7 +245,7 @@ PLy_exec_function(FunctionCallInfo fcinfo, PLyProcedure *proc)
desc = lookup_rowtype_tupdesc(proc->result.out.d.typoid,
proc->result.out.d.typmod);
- rv = PLyObject_ToCompositeDatum(&proc->result, desc, plrv);
+ rv = PLyObject_ToCompositeDatum(&proc->result, desc, plrv, false);
fcinfo->isnull = (rv == (Datum) NULL);
ReleaseTupleDesc(desc);
@@ -253,7 +253,7 @@ PLy_exec_function(FunctionCallInfo fcinfo, PLyProcedure *proc)
else
{
fcinfo->isnull = false;
- rv = (proc->result.out.d.func) (&proc->result.out.d, -1, plrv);
+ rv = (proc->result.out.d.func) (&proc->result.out.d, -1, plrv, false);
}
}
PG_CATCH();
@@ -984,7 +984,8 @@ PLy_modify_tuple(PLyProcedure *proc, PyObject *pltd, TriggerData *tdata,
modvalues[i] = (att->func) (att,
tupdesc->attrs[atti]->atttypmod,
- plval);
+ plval,
+ false);
modnulls[i] = ' ';
}
else
diff --git a/src/pl/plpython/plpy_spi.c b/src/pl/plpython/plpy_spi.c
index 09ee06d..0d556a2 100644
--- a/src/pl/plpython/plpy_spi.c
+++ b/src/pl/plpython/plpy_spi.c
@@ -264,7 +264,8 @@ PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit)
plan->values[j] =
plan->args[j].out.d.func(&(plan->args[j].out.d),
-1,
- elem);
+ elem,
+ false);
}
PG_CATCH();
{
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 56cfdd1..9c62be7 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -14,6 +14,7 @@
#include "parser/parse_type.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/numeric.h"
@@ -49,21 +50,21 @@ static PyObject *PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndi
char **dataptr_p, bits8 **bitmap_p, int *bitmask_p);
/* conversion from Python objects to Datums */
-static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
+static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
static void PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
int *dims, int ndim, int dim,
Datum *elems, bool *nulls, int *currelem);
/* conversion from Python objects to composite Datums (used by triggers and SRFs) */
-static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string);
+static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string, bool inarray);
static Datum PLyMapping_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *mapping);
static Datum PLySequence_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *sequence);
-static Datum PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object);
+static Datum PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object, bool inarray);
void
PLy_typeinfo_init(PLyTypeInfo *arg, MemoryContext mcxt)
@@ -341,12 +342,12 @@ PLyDict_FromTuple(PLyTypeInfo *info, HeapTuple tuple, TupleDesc desc)
* as an object that has __getattr__ support.
*/
Datum
-PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv)
+PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv, bool inarray)
{
Datum datum;
if (PyString_Check(plrv) || PyUnicode_Check(plrv))
- datum = PLyString_ToComposite(info, desc, plrv);
+ datum = PLyString_ToComposite(info, desc, plrv, inarray);
else if (PySequence_Check(plrv))
/* composite type as sequence (tuple, list etc) */
datum = PLySequence_ToComposite(info, desc, plrv);
@@ -355,7 +356,7 @@ PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv)
datum = PLyMapping_ToComposite(info, desc, plrv);
else
/* returned as smth, must provide method __getattr__(name) */
- datum = PLyGenericObject_ToComposite(info, desc, plrv);
+ datum = PLyGenericObject_ToComposite(info, desc, plrv, inarray);
return datum;
}
@@ -747,7 +748,7 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
* type can parse.
*/
static Datum
-PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
Datum rv;
@@ -766,7 +767,7 @@ PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
* with embedded nulls. And it's faster this way.
*/
static Datum
-PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
PyObject *volatile plrv_so = NULL;
Datum rv;
@@ -810,7 +811,7 @@ PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
* for obtaining PostgreSQL tuples.
*/
static Datum
-PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
Datum rv;
PLyTypeInfo info;
@@ -837,7 +838,7 @@ PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
* that info instead of looking it up every time a tuple is returned from
* the function.
*/
- rv = PLyObject_ToCompositeDatum(&info, desc, plrv);
+ rv = PLyObject_ToCompositeDatum(&info, desc, plrv, inarray);
ReleaseTupleDesc(desc);
@@ -909,26 +910,70 @@ PLyObject_AsString(PyObject *plrv)
* cstring into PostgreSQL type.
*/
static Datum
-PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
+ char *str;
+
Assert(plrv != Py_None);
+ str = PLyObject_AsString(plrv);
+
+ /*
+ * If we are parsing a composite type within an array, and the string
+ * isn't a valid record literal, there's a high chance that the function
+ * did something like:
+ *
+ * CREATE FUNCTION .. RETURNS comptype[] AS $$ return [['foo', 'bar']] $$
+ * LANGUAGE plpython;
+ *
+ * Before PostgreSQL 10, that was interpreted as a single-dimensional
+ * array, containing record ('foo', 'bar'). PostgreSQL 10 added support
+ * for multi-dimensional arrays, and it is now interpreted as a
+ * two-dimensional array, containing two records, 'foo', and 'bar'.
+ * record_in() will throw an error, because "foo" is not a valid record
+ * literal.
+ *
+ * To make that less confusing to users who are upgrading from older
+ * versions, try to give a hint in the typical cases of that. If we are
+ * parsing an array of composite types, and we see a string literal that
+ * is not a valid record literal, give a hint. We only want to give the
+ * hint in the narrow case of a malformed string literal, not any error
+ * from record_in(), so check for that case here specifically.
+ *
+ * This check better match the one in record_in(), so that we don't forbid
+ * literals that are actually valid!
+ */
+ if (inarray && arg->typfunc.fn_oid == F_RECORD_IN)
+ {
+ char *ptr = str;
+
+ /* Allow leading whitespace */
+ while (*ptr && isspace((unsigned char) *ptr))
+ ptr++;
+ if (*ptr++ != '(')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("malformed record literal: \"%s\"", str),
+ errdetail("Missing left parenthesis."),
+ errhint("To return a composite type in an array, return the composite type as a Python tuple, e.g. \"[('foo')]\"")));
+ }
+
return InputFunctionCall(&arg->typfunc,
- PLyObject_AsString(plrv),
+ str,
arg->typioparam,
typmod);
}
static Datum
-PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
return FunctionCall1(&arg->typtransform, PointerGetDatum(plrv));
}
static Datum
-PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
ArrayType *array;
int i;
@@ -1086,7 +1131,7 @@ PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
else
{
nulls[*currelem] = false;
- elems[*currelem] = elm->func(elm, -1, obj);
+ elems[*currelem] = elm->func(elm, -1, obj, true);
}
Py_XDECREF(obj);
(*currelem)++;
@@ -1094,9 +1139,8 @@ PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
}
}
-
static Datum
-PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
+PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string, bool inarray)
{
Datum result;
HeapTuple typeTup;
@@ -1121,7 +1165,7 @@ PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
ReleaseSysCache(typeTup);
- result = PLyObject_ToDatum(&locinfo.out.d, desc->tdtypmod, string);
+ result = PLyObject_ToDatum(&locinfo.out.d, desc->tdtypmod, string, inarray);
MemoryContextDelete(cxt);
@@ -1173,7 +1217,7 @@ PLyMapping_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *mapping)
}
else if (value)
{
- values[i] = (att->func) (att, -1, value);
+ values[i] = (att->func) (att, -1, value, false);
nulls[i] = false;
}
else
@@ -1266,7 +1310,7 @@ PLySequence_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *sequence)
}
else if (value)
{
- values[i] = (att->func) (att, -1, value);
+ values[i] = (att->func) (att, -1, value, false);
nulls[i] = false;
}
@@ -1295,7 +1339,7 @@ PLySequence_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *sequence)
static Datum
-PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object)
+PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object, bool inarray)
{
Datum result;
HeapTuple tuple;
@@ -1336,16 +1380,29 @@ PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object
}
else if (value)
{
- values[i] = (att->func) (att, -1, value);
+ values[i] = (att->func) (att, -1, value, false);
nulls[i] = false;
}
else
+ {
+ /*
+ * No attribute for this column in the object.
+ *
+ * If we are parsing a composite type in an array, a likely
+ * cause is that the function contained something like "[[123,
+ * 'foo']]". Before PostgreSQL 10, that was interpreted as an
+ * array, with a composite type (123, 'foo') in it.. But now
+ * it's interpreted as a two-dimensional array, and we try to
+ * interpret "123" as the composite type. See also similar
+ * heuristic in PLyObject_ToDatum().
+ */
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_COLUMN),
errmsg("attribute \"%s\" does not exist in Python object", key),
- errhint("To return null in a column, "
- "let the returned object have an attribute named "
- "after column with value None.")));
+ inarray ?
+ errhint("To return a composite type in an array, return the composite type as a Python tuple, e.g. \"[('foo')]\"") :
+ errhint("To return null in a column, let the returned object have an attribute named after column with value None.")));
+ }
Py_XDECREF(value);
value = NULL;
diff --git a/src/pl/plpython/plpy_typeio.h b/src/pl/plpython/plpy_typeio.h
index 29fff61..5f5c1ad 100644
--- a/src/pl/plpython/plpy_typeio.h
+++ b/src/pl/plpython/plpy_typeio.h
@@ -10,8 +10,11 @@
#include "fmgr.h"
#include "storage/itemptr.h"
+/*
+ * Conversion from PostgreSQL Datum to a Python object.
+ */
struct PLyDatumToOb;
-typedef PyObject *(*PLyDatumToObFunc) (struct PLyDatumToOb *, Datum);
+typedef PyObject *(*PLyDatumToObFunc) (struct PLyDatumToOb *arg, Datum val);
typedef struct PLyDatumToOb
{
@@ -39,11 +42,15 @@ typedef union PLyTypeInput
PLyTupleToOb r;
} PLyTypeInput;
-/* convert PyObject to a Postgresql Datum or tuple.
- * output from Python
+/*
+ * Conversion from Python object to a Postgresql Datum.
+ *
+ * The 'inarray' argument to the conversion function is true, if the
+ * converted value was in an array (Python list). It is used to give a
+ * better error message in some cases.
*/
struct PLyObToDatum;
-typedef Datum (*PLyObToDatumFunc) (struct PLyObToDatum *, int32, PyObject *);
+typedef Datum (*PLyObToDatumFunc) (struct PLyObToDatum *arg, int32 typmod, PyObject *val, bool inarray);
typedef struct PLyObToDatum
{
@@ -104,7 +111,7 @@ extern void PLy_output_tuple_funcs(PLyTypeInfo *arg, TupleDesc desc);
extern void PLy_output_record_funcs(PLyTypeInfo *arg, TupleDesc desc);
/* conversion from Python objects to composite Datums */
-extern Datum PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv);
+extern Datum PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv, bool isarray);
/* conversion from heap tuples to Python dictionaries */
extern PyObject *PLyDict_FromTuple(PLyTypeInfo *info, HeapTuple tuple, TupleDesc desc);
diff --git a/src/pl/plpython/sql/plpython_composite.sql b/src/pl/plpython/sql/plpython_composite.sql
index 72a35c3..1a343af 100644
--- a/src/pl/plpython/sql/plpython_composite.sql
+++ b/src/pl/plpython/sql/plpython_composite.sql
@@ -207,3 +207,12 @@ SELECT * FROM return_record_2('v4') AS (v1 int, v3 int, v2 int);
-- works
SELECT * FROM return_record_2('v3') AS (v1 int, v3 int, v2 int);
SELECT * FROM return_record_2('v3') AS (v1 int, v2 int, v3 int);
+
+-- Starting with PostgreSQL 10, a composite type in an array cannot be represented as
+-- a Python list, because it's ambiguous with multi-dimensional arrays. So this
+-- throws an error now. The error should contain a useful hint on the issue.
+CREATE FUNCTION composite_type_as_list() RETURNS type_record[] AS $$
+ return [['first', 1]];
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM composite_type_as_list();
--
2.9.3
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
