Re: [HACKERS] Bad Data back Door

2012-10-09 Thread Albe Laurenz
David E. Wheeler wrote: As the author I agree that this is a bug in oracle_fdw. Thanks. Should I file a report somewhere? That's not necessary. Thanks for reporting the problem. It may be a few days until I get around to fix that. Oracle does not care much about correct encoding. Yeah,

Re: [HACKERS] Bad Data back Door

2012-10-09 Thread Albe Laurenz
I wrote: Tom Lane wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against oracle_fdw, not the core system. As the author I agree that this is a bug in oracle_fdw. Ok, fixed. David, could you try

Re: [HACKERS] Bad Data back Door

2012-10-08 Thread Albe Laurenz
Tom Lane wrote: David E. Wheeler da...@justatheory.com writes: On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against oracle_fdw, not the core

Re: [HACKERS] Bad Data back Door

2012-10-08 Thread David E. Wheeler
On Oct 8, 2012, at 12:25 AM, Albe Laurenz laurenz.a...@wien.gv.at wrote: As the author I agree that this is a bug in oracle_fdw. Thanks. Should I file a report somewhere? This was caused by ignorance on my part: I had assumed that the type input functions would perform the necessary checks,

Re: [HACKERS] Bad Data back Door

2012-10-08 Thread David E. Wheeler
On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against oracle_fdw, not the core system. (FWIW, contrib/file_fdw depends on the COPY code,

Re: [HACKERS] Bad Data back Door

2012-10-08 Thread Tom Lane
David E. Wheeler da...@justatheory.com writes: On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against oracle_fdw, not the core system. (FWIW,

Re: [HACKERS] Bad Data back Door

2012-10-08 Thread David E. Wheeler
On Oct 8, 2012, at 11:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: FWIW, I believe that dblink does not check encoding. In dblink's case, that boils down to trusting a remote instance of Postgres to get this right, which doesn't seem totally unreasonable. But I wouldn't object to adding checks

Re: [HACKERS] Bad Data back Door

2012-10-08 Thread Andrew Dunstan
On 10/08/2012 02:13 PM, Tom Lane wrote: David E. Wheeler da...@justatheory.com writes: On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against

Re: [HACKERS] Bad Data back Door

2012-10-06 Thread Heikki Linnakangas
On 06.10.2012 05:14, John R Pierce wrote: I'd like to see some encoding validation and substitution functions in postgres. for instance, one that can take any supported encoding and convert it to the database encoding and generate an error on any invalid character. this translation could be

Re: [HACKERS] Bad Data back Door

2012-10-06 Thread David E. Wheeler
On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Probably not so much assumed as nobody thought about it. In e.g. plperl we expend the cycles to do encoding validity checking on *every* string entering the system from Perl. I'm not sure why foreign tables ought to get a pass

Re: [HACKERS] Bad Data back Door

2012-10-06 Thread Atri Sharma
On Sat, Oct 6, 2012 at 1:34 PM, David E. Wheeler da...@justatheory.com wrote: On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Probably not so much assumed as nobody thought about it. In e.g. plperl we expend the cycles to do encoding validity checking on *every* string

Re: [HACKERS] Bad Data back Door

2012-10-06 Thread John R Pierce
On 10/06/12 3:45 AM, Heikki Linnakangas wrote: At the SQL level, there's the convert(bytea, name, name) function. ahhh, right. (forehead slap) a 2nd function would do the same, but replace errors with the substitution character in the target charset and not error. Hmm, I don't think we

Re: [HACKERS] Bad Data back Door

2012-10-06 Thread Tom Lane
David E. Wheeler da...@justatheory.com writes: On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against oracle_fdw, not the core system. (FWIW,

Re: [HACKERS] Bad Data back Door

2012-10-06 Thread Andrew Dunstan
On 10/06/2012 03:35 PM, Tom Lane wrote: David E. Wheeler da...@justatheory.com writes: On Oct 5, 2012, at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against

Re: [HACKERS] Bad Data back Door

2012-10-05 Thread Tom Lane
David E. Wheeler da...@justatheory.com writes: I’ve discovered something a bit disturbing at $work. We’re migrating (slowly) from Oracle to PostgreSQL, and in some cases are using oracle_fdw to copy data over. Alas, there are a fair number of text values in the Oracle database that,

Re: [HACKERS] Bad Data back Door

2012-10-05 Thread John R Pierce
On 10/05/12 6:12 PM, Tom Lane wrote: Now, having said that, I think it has to be the reponsibility of the FDW to apply any required check ... which makes this a bug report against oracle_fdw, not the core system. (FWIW, contrib/file_fdw depends on the COPY code, which will check encoding.)