Hi
2016-09-21 19:53 GMT+02:00 Dave Cramer <p...@fastcrypt.com>:
>
> On 18 September 2016 at 09:27, Dave Cramer <p...@fastcrypt.com> wrote:
>
>>
>> On 10 August 2016 at 01:53, Pavel Stehule <pavel.steh...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> 2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishche...@pivotal.io>:
>>>
>>>> On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
>>>> agrishche...@pivotal.io> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Current implementation of PL/Python does not allow the use of
>>>>> multi-dimensional arrays, for both input and output parameters. This
>>>>> forces
>>>>> end users to introduce workarounds like casting arrays to text before
>>>>> passing them to the functions and parsing them after, which is an
>>>>> error-prone approach
>>>>>
>>>>> This patch adds support for multi-dimensional arrays as both input and
>>>>> output parameters for PL/Python functions. The number of dimensions
>>>>> supported is limited by Postgres MAXDIM macrovariable, by default equal to
>>>>> 6. Both input and output multi-dimensional arrays should have fixed
>>>>> dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
>>>>> represent MxNxK cube, etc.
>>>>>
>>>>> This patch does not support multi-dimensional arrays of composite
>>>>> types, as composite types in Python might be represented as iterators and
>>>>> there is no obvious way to find out when the nested array stops and
>>>>> composite type structure starts. For example, if we have a composite type
>>>>> of (int, text), we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'],
>>>>> [4,'d'] ] ]", and it is hard to find out that the first two lists are
>>>>> lists, and the third one represents structure. Things are getting even
>>>>> more
>>>>> complex when you have arrays as members of composite type. This is why I
>>>>> think this limitation is reasonable.
>>>>>
>>>>> Given the function:
>>>>>
>>>>> CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS
>>>>> int4[] AS $$
>>>>> plpy.info(x, type(x))
>>>>> return x
>>>>> $$ LANGUAGE plpythonu;
>>>>>
>>>>> Before patch:
>>>>>
>>>>> # SELECT * FROM test_type_conversion_array_int
>>>>> 4(ARRAY[[1,2,3],[4,5,6]]);
>>>>> ERROR: cannot convert multidimensional array to Python list
>>>>> DETAIL: PL/Python only supports one-dimensional arrays.
>>>>> CONTEXT: PL/Python function "test_type_conversion_array_int4"
>>>>>
>>>>>
>>>>> After patch:
>>>>>
>>>>> # SELECT * FROM test_type_conversion_array_int
>>>>> 4(ARRAY[[1,2,3],[4,5,6]]);
>>>>> INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
>>>>> test_type_conversion_array_int4
>>>>> ---------------------------------
>>>>> {{1,2,3},{4,5,6}}
>>>>> (1 row)
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Alexey Grishchenko
>>>>>
>>>>
>>>> Also this patch incorporates the fix for https://www.postgresql.org
>>>> /message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-2NR9jUfv3q
>>>> wHA%40mail.gmail.com, as they touch the same piece of code - array
>>>> manipulation in PL/Python
>>>>
>>>>
>>> I am sending review of this patch:
>>>
>>> 1. The implemented functionality is clearly benefit - passing MD arrays,
>>> pretty faster passing bigger arrays
>>> 2. I was able to use this patch cleanly without any errors or warnings
>>> 3. There is no any error or warning
>>> 4. All tests passed - I tested Python 2.7 and Python 3.5
>>> 5. The code is well commented and clean
>>> 6. For this new functionality the documentation is not necessary
>>>
>>> 7. I invite more regress tests for both directions (Python <-> Postgres)
>>> for more than two dimensions
>>>
>>> My only one objection is not enough regress tests - after fixing this
>>> patch will be ready for commiters.
>>>
>>
Now, the tests are enough - so I'll mark this patch as ready for commiters.
I had to fix tests - there was lot of white spaces, and the result for
python3 was missing
Regards
Pavel
>
>>> Good work, Alexey
>>>
>>> Thank you
>>>
>>> Regards
>>>
>>> Pavel
>>>
>>>
>>>> --
>>>> Best regards,
>>>> Alexey Grishchenko
>>>>
>>>
>>>
>>
>> Pavel,
>>
>> I will pick this up.
>>
>>
>> Pavel,
>
> Please see attached patch which provides more test cases
>
> I just realized this patch contains the original patch as well. What is
> the protocol for sending in subsequent patches ?
>
>>
>> Dave Cramer
>>
>> da...@postgresintl.com
>> www.postgresintl.com
>>
>>
>
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..296ef8b 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,126 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +668,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +702,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..b244d2a 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,126 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +668,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +702,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..9436412 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -631,46 +631,111 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
+ int ndim;
+ int *dims;
+ int *lb;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
int i;
+ int dim;
+ int indx[MAXDIM];
+ PyObject *lists[MAXDIM];
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ lb = ARR_LBOUND(array);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /* Internal array representation pointers */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ /* Iterators initialization */
+ for (i = 0; i < ndim; i++) {
+ indx[i] = lb[i];
+ lists[i] = NULL;
+ }
+ lists[0] = PyList_New(dims[0]);
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array */
+ Py_INCREF(lists[0]);
+
+ /* In this cycle we are going over array dimensions. Postgres offers you an
+ * option to iterate over all the multi-dimensional array elemens in order.
+ * For 3-dimesnional array the order of iteration would be following - first
+ * you start with [0,0,0] elements through [0,0,k], then [0,1,0] till [0,1,k]
+ * till [0,m,k], then [1,0,0] till [1,0,k] till [1,m,k], and so on.
+ * In Python, each 1-d array is a separate list object, so 3-d array of
+ * [n,m,k] element is a list of n m-element arrays, each element of which is
+ * k-element array. In this cycle we traverse from outter dimensions to
+ * inner ones, creating nested Python lists during traversal */
+ dim = 0;
+ while (dim >= 0)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* If we finished up iterating over current dimension - go one level up */
+ if (indx[dim] > dims[dim])
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ Py_DECREF(lists[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner dimension - create one more list */
+ else if (dim < ndim - 1)
+ {
+ lists[dim+1] = PyList_New(dims[dim+1]);
+
+ /* We need this INCREF as we keep array pointer on our side,
+ * while PyList_SET_ITEM steals the reference */
+ Py_INCREF(lists[dim+1]);
+
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], lists[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ indx[dim] = lb[dim];
+ }
+ /* If we are iterating over the outter dimension, fill the list with
+ * values from the original Postgres array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = lb[dim]; indx[dim] <= dims[dim]; indx[dim]++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
}
- return list;
+ return lists[0];
}
/*
@@ -864,39 +929,131 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
int len;
- int lbs;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int indx[MAXDIM];
+ PyObject *stack[MAXDIM];
+ int dim;
+ int idxelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
+ ndim = 1;
+ dims[0] = PySequence_Length(plrv);
+ len = dims[0];
+ stack[0] = plrv;
+
+ /* We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, and when we have strings */
+ if (len > 0 && !type_is_rowtype(get_base_element_type(arg->typoid)) &&
+ !PyString_Check(plrv) && !PyBytes_Check(plrv) && !PyUnicode_Check(plrv))
+ {
+ PyObject *pyptr = PySequence_GetItem(plrv, 0);
+
+ /* We want to iterate through all iterable objects except by strings on nested levels */
+ while (pyptr != NULL && PySequence_Check(pyptr) &&
+ !(PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr)))
+ {
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "Cannot determine sequence length for function return value");
+ len *= dims[ndim];
+ stack[ndim] = pyptr;
+ ndim += 1;
+ if (dims[ndim - 1] == 0) {
+ pyptr = NULL;
+ break;
+ }
+ pyptr = PySequence_GetItem(pyptr, 0);
+ }
+
+ /* Pyptr points to element of n-dimensional array, we don't need its reference */
+ Py_XDECREF(pyptr);
+ }
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array, and
+ * stack[0] corresponds to function return value */
+ Py_INCREF(stack[0]);
+
elems = palloc(sizeof(*elems) * len);
nulls = palloc(sizeof(*nulls) * len);
- for (i = 0; i < len; i++)
- {
- PyObject *obj = PySequence_GetItem(plrv, i);
+ for (i = 0; i < ndim; i++) {
+ indx[i] = 0;
+ lbs[i] = 1;
+ }
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* In this cycle we are going over nested Python lists, fetching elements
+ * from the deepest level and putting them into a linear array for Postgres
+ * to interpret them as n-dimensional array. This is a cycle implementation
+ * of DFS (recursive traversal of nested arrays here), keeping the stack in
+ * "stack" variable */
+ dim = 0;
+ idxelem = 0;
+ while (dim >= 0)
+ {
+ /* If we finished up iterating over current list - go one level up */
+ if (indx[dim] == dims[dim])
+ {
+ Py_DECREF(stack[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner list - create one more list */
+ else if (dim < ndim - 1)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ stack[dim+1] = PySequence_GetItem(stack[dim], indx[dim]);
+ if (PySequence_Length(stack[dim+1]) != dims[dim+1])
+ PLy_elog(ERROR, "Multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int)PySequence_Length(stack[dim+1]), dims[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ }
+ /* If we are iterating over the outter list, fill the output array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = 0; indx[dim] < dims[dim]; indx[dim]++)
+ {
+ PyObject *obj = PySequence_GetItem(stack[dim], indx[dim]);
+
+ if (obj == Py_None)
+ nulls[idxelem] = true;
+ else
+ {
+ nulls[idxelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[idxelem] = arg->elm->func(arg->elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ idxelem += 1;
+ }
}
- Py_XDECREF(obj);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..b3599bc 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,79 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +317,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +341,11 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
