Hi hackers,

Commit 23a27b039d94ba359286694831eafe03cd970eef changed the type of
numbers-of-tuples-processed counters to uint64 and adjusted various PLs
to cope with this.  I noticed the PL/Perl changes did not take full
advantage of what Perl is capable of handling, so here's a patch that
improves that.

1) Perl's integers are at least pointer-sized and either signed or
   unsigned, so can potentially hold up to 2⁶⁴-1. Floating point numbers
   can also be larger than double (up to 128bit), allowing for exact
   representation of integers beyond 2⁵³.  Hence, adjust the setting of
   the "processed" hash item to use UV_MAX for the limit and (NV) or
   (UV) for the casts.

2) Perl 5.20 and later use SSize_t for array indices, so can cope with
   up to SSize_t_max items in an array (if you have the memory).

3) To be able to actually return such result sets from sp_exec_query(),
   I had to change the repalloc() in spi_printtup() to repalloc_huge().

"A disappointingly low fraction of the human race is,
 at any given time, on fire." - Stig Sandbeck Mathisen

>From f6d38f1a5c7d6135dd5f580aa8ed38bb74162a0f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilm...@ilmari.org>
Date: Sun, 13 Mar 2016 01:21:43 +0000
Subject: [PATCH] Use correct types and limits for PL/Perl SPI query results

Perl's integers are pointer-sized, so can hold more than INT_MAX on LP64
platforms, and come in both signed (IV) and unsigned (UV).  Floating
point values (NV) may also be larger than double.

Since Perl 5.19.4 array indices are SSize_t instead of I32, so allow up
to SSize_t_max on those versions.  The limit is not imposed just by
av_extend's argument type, but all the array handling code, so remove
the speculative comment.

Finally, change spi_printtup() to use repalloc_huge() to be able to
return a tuple table of more than 1GiB.
 src/backend/executor/spi.c |  2 +-
 src/pl/plperl/plperl.c     | 13 ++++---------
 src/pl/plperl/plperl.h     |  7 +++++++
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 3d04c23..fd94179 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1800,7 +1800,7 @@ spi_printtup(TupleTableSlot *slot, DestReceiver *self)
 		/* Double the size of the pointer array */
 		tuptable->free = tuptable->alloced;
 		tuptable->alloced += tuptable->free;
-		tuptable->vals = (HeapTuple *) repalloc(tuptable->vals,
+		tuptable->vals = (HeapTuple *) repalloc_huge(tuptable->vals,
 									  tuptable->alloced * sizeof(HeapTuple));
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 269f7f3..d405179 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -3104,9 +3104,9 @@ plperl_spi_execute_fetch_result(SPITupleTable *tuptable, uint64 processed,
 	hv_store_string(result, "status",
 	hv_store_string(result, "processed",
-					(processed > (uint64) INT_MAX) ?
-					newSVnv((double) processed) :
-					newSViv((int) processed));
+					(processed > (uint64) UV_MAX) ?
+					newSVnv((NV) processed) :
+					newSVuv((UV) processed));
 	if (status > 0 && tuptable)
@@ -3114,12 +3114,7 @@ plperl_spi_execute_fetch_result(SPITupleTable *tuptable, uint64 processed,
 		SV		   *row;
 		uint64		i;
-		/*
-		 * av_extend's 2nd argument is declared I32.  It's possible we could
-		 * nonetheless push more than INT_MAX elements into a Perl array, but
-		 * let's just fail instead of trying.
-		 */
-		if (processed > (uint64) INT_MAX)
+		if (processed > AV_SIZE_MAX)
 			errmsg("query result has too many rows to fit in a Perl array")));
diff --git a/src/pl/plperl/plperl.h b/src/pl/plperl/plperl.h
index 9179bbd..2f376ee 100644
--- a/src/pl/plperl/plperl.h
+++ b/src/pl/plperl/plperl.h
@@ -100,4 +100,11 @@ void		plperl_spi_freeplan(char *);
 void		plperl_spi_cursor_close(char *);
 char	   *plperl_sv_to_literal(SV *, char *);
+/* Perl 5.19.4 changed array indices from I32 to SSize_t */
+#if PERL_BCDVERSION >= 0x5019004
+#define AV_SIZE_MAX ((uint64) SSize_t_MAX)
+#define AV_SIZE_MAX ((uint64) I32_MAX)
 #endif   /* PL_PERL_H */

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to