Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns
On Sun, Oct 15, 2006 at 06:15:27PM -0400, Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
I am also wondering, now that it's been raised, if we need to issue a use
utf8; in the startup code, so that literals in the code get the right
encoding.
Good question. I took care to ensure that the code strings passed to
Perl are marked as UTF8; perhaps that makes it happen implicitly?
If not, are there any downsides to issuing use utf8?
What use utf8 does is allow the *source* to be in utf8, thus affecting
what's a valid identifier and such. It doesn't affect the data, for
that you need use encoding 'utf8'.
It's clear whether you actually want to allow people to put utf8
characters directly into their source (especially if the database is
not in utf8 encoding anyway). There is always the \u{} escape.
The perlunicode man page describe it better, though I only have
perl5.8. In know the perl5.6 model was different and somewhat more
awkward to use.
Have a nice day,
--
Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/
From each according to his ability. To each according to his ability to
litigate.
signature.asc
Description: Digital signature
Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns
Martijn van Oosterhout kleptog@svana.org writes:
It's clear whether you actually want to allow people to put utf8
characters directly into their source (especially if the database is
not in utf8 encoding anyway). There is always the \u{} escape.
Well, if the database encoding isn't utf8 then we'd not issue any such
command anyway. But if it is, then AFAICS the text of pg_proc entries
could be expected to be utf8 too.
regards, tom lane
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns column names which are not marked as UTF8
I wrote:
It looks to me like basically everywhere in plperl.c that does newSVpv()
should follow it with
#if PERL_BCDVERSION = 0x5006000L
if (GetDatabaseEncoding() == PG_UTF8)
SvUTF8_on(sv);
#endif
Experimentation proved that this was insufficient to fix Vitali's
problem --- the string he's unhappy about is actually a hash key entry,
and there's no documented way to mark the second argument of hv_store()
as being a UTF-8 string. Some digging in the Perl source code found
that since at least Perl 5.8.0, hv_fetch and hv_store recognize a
negative key length as meaning a UTF-8 key (ick!!), so I used that hack.
I am not sure there is any reasonable fix available in Perl 5.6.x.
Attached patch applied to HEAD, but I'm not going to risk back-patching
it without some field testing.
regards, tom lane
*** src/pl/plperl/plperl.c.orig Tue Oct 3 23:17:16 2006
--- src/pl/plperl/plperl.c Sun Oct 15 14:47:27 2006
***
*** 114,119
--- 114,122
static SV *plperl_hash_from_tuple(HeapTuple tuple, TupleDesc tupdesc);
static void plperl_init_shared_libs(pTHX);
static HV *plperl_spi_execute_fetch_result(SPITupleTable *, int, int);
+ static SV *newSVstring(const char *str);
+ static SV **hv_store_string(HV *hv, const char *key, SV *val);
+ static SV **hv_fetch_string(HV *hv, const char *key);
/*
* This routine is a crock, and so is everyplace that calls it. The problem
***
*** 471,531
)
);
! hv_store(hv, name, 4, newSVpv(tdata-tg_trigger-tgname, 0), 0);
! hv_store(hv, relid, 5, newSVpv(relid, 0), 0);
if (TRIGGER_FIRED_BY_INSERT(tdata-tg_event))
{
event = INSERT;
if (TRIGGER_FIRED_FOR_ROW(tdata-tg_event))
! hv_store(hv, new, 3,
!
plperl_hash_from_tuple(tdata-tg_trigtuple, tupdesc),
!0);
}
else if (TRIGGER_FIRED_BY_DELETE(tdata-tg_event))
{
event = DELETE;
if (TRIGGER_FIRED_FOR_ROW(tdata-tg_event))
! hv_store(hv, old, 3,
!
plperl_hash_from_tuple(tdata-tg_trigtuple, tupdesc),
!0);
}
else if (TRIGGER_FIRED_BY_UPDATE(tdata-tg_event))
{
event = UPDATE;
if (TRIGGER_FIRED_FOR_ROW(tdata-tg_event))
{
! hv_store(hv, old, 3,
!
plperl_hash_from_tuple(tdata-tg_trigtuple, tupdesc),
!0);
! hv_store(hv, new, 3,
!
plperl_hash_from_tuple(tdata-tg_newtuple, tupdesc),
!0);
}
}
else
event = UNKNOWN;
! hv_store(hv, event, 5, newSVpv(event, 0), 0);
! hv_store(hv, argc, 4, newSViv(tdata-tg_trigger-tgnargs), 0);
if (tdata-tg_trigger-tgnargs 0)
{
AV *av = newAV();
for (i = 0; i tdata-tg_trigger-tgnargs; i++)
! av_push(av, newSVpv(tdata-tg_trigger-tgargs[i], 0));
! hv_store(hv, args, 4, newRV_noinc((SV *) av), 0);
}
! hv_store(hv, relname, 7,
!newSVpv(SPI_getrelname(tdata-tg_relation), 0), 0);
! hv_store(hv, table_name, 10,
!newSVpv(SPI_getrelname(tdata-tg_relation), 0), 0);
! hv_store(hv, table_schema, 12,
!newSVpv(SPI_getnspname(tdata-tg_relation), 0), 0);
if (TRIGGER_FIRED_BEFORE(tdata-tg_event))
when = BEFORE;
--- 474,534
)
);
! hv_store_string(hv, name, newSVstring(tdata-tg_trigger-tgname));
! hv_store_string(hv, relid, newSVstring(relid));
if (TRIGGER_FIRED_BY_INSERT(tdata-tg_event))
{
event = INSERT;
if (TRIGGER_FIRED_FOR_ROW(tdata-tg_event))
! hv_store_string(hv, new,
!
plperl_hash_from_tuple(tdata-tg_trigtuple,
!
tupdesc));
}
else if (TRIGGER_FIRED_BY_DELETE(tdata-tg_event))
{
event = DELETE;
if (TRIGGER_FIRED_FOR_ROW(tdata-tg_event))
! hv_store_string(hv, old,
!
plperl_hash_from_tuple(tdata-tg_trigtuple,
!
Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns
Tom Lane wrote: I wrote: It looks to me like basically everywhere in plperl.c that does newSVpv() should follow it with #if PERL_BCDVERSION = 0x5006000L if (GetDatabaseEncoding() == PG_UTF8) SvUTF8_on(sv); #endif Experimentation proved that this was insufficient to fix Vitali's problem --- the string he's unhappy about is actually a hash key entry, and there's no documented way to mark the second argument of hv_store() as being a UTF-8 string. Some digging in the Perl source code found that since at least Perl 5.8.0, hv_fetch and hv_store recognize a negative key length as meaning a UTF-8 key (ick!!), so I used that hack. I am not sure there is any reasonable fix available in Perl 5.6.x. Attached patch applied to HEAD, but I'm not going to risk back-patching it without some field testing. Hmm. That negative pointer hack is mighty ugly. I am also wondering, now that it's been raised, if we need to issue a use utf8; in the startup code, so that literals in the code get the right encoding. cheers andrew ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns
Andrew Dunstan [EMAIL PROTECTED] writes: I am also wondering, now that it's been raised, if we need to issue a use utf8; in the startup code, so that literals in the code get the right encoding. Good question. I took care to ensure that the code strings passed to Perl are marked as UTF8; perhaps that makes it happen implicitly? If not, are there any downsides to issuing use utf8? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns
On Sun, Oct 15, 2006 at 04:50:15PM -0500, Andrew Dunstan wrote: Tom Lane wrote: I wrote: It looks to me like basically everywhere in plperl.c that does newSVpv() should follow it with #if PERL_BCDVERSION = 0x5006000L if (GetDatabaseEncoding() == PG_UTF8) SvUTF8_on(sv); #endif Experimentation proved that this was insufficient to fix Vitali's problem --- the string he's unhappy about is actually a hash key entry, and there's no documented way to mark the second argument of hv_store() as being a UTF-8 string. Some digging in the Perl source code found that since at least Perl 5.8.0, hv_fetch and hv_store recognize a negative key length as meaning a UTF-8 key (ick!!), so I used that hack. I am not sure there is any reasonable fix available in Perl 5.6.x. Attached patch applied to HEAD, but I'm not going to risk back-patching it without some field testing. Hmm. That negative pointer hack is mighty ugly. I am also wondering, now that it's been raised, if we need to issue a use utf8; in the startup code, so that literals in the code get the right encoding. That would be a reason to go to 5.8, as 'use utf8;' is tricky at best in 5.6. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns column names which are not marked as UTF8
Vitali Stupin [EMAIL PROTECTED] writes: If database uses UTF8 encoding, then spi_exec_query in plperl should return query results in UTF8 encoding. But unfortunately only data is marked as UTF8, while column names are not. It looks to me like basically everywhere in plperl.c that does newSVpv() should follow it with #if PERL_BCDVERSION = 0x5006000L if (GetDatabaseEncoding() == PG_UTF8) SvUTF8_on(sv); #endif whereas currently there are only a couple of places that do that. I'm tempted to consolidate this into a function on the order of newSVstring(const char *) or some such. Comments? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
