Re: [PATCHES] [HACKERS] patches for items from TODO list
Tom Lane wrote:
Third, I found out that psql has some unusual handling of escaped
numbers. Instead of using \ddd as octal, it has \ddd is decimal, \0ddd
is octal, and \0xddd is decimal. It is basically following the strtol()
rules for an escaped value. This seems confusing and contradicts how
the rest of our system works.
I agree, that's just going to confuse people.
Fixed and applied, with no hex addition.
--
Bruce Momjian| http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: src/bin/psql/psqlscan.l
===
RCS file: /cvsroot/pgsql/src/bin/psql/psqlscan.l,v
retrieving revision 1.10
diff -c -c -r1.10 psqlscan.l
*** src/bin/psql/psqlscan.l 26 May 2005 01:24:29 - 1.10
--- src/bin/psql/psqlscan.l 30 May 2005 14:35:43 -
***
*** 849,877
\\r { appendPQExpBufferChar(output_buf, '\r'); }
\\f { appendPQExpBufferChar(output_buf, '\f'); }
! \\[1-9][0-9]* {
! /* decimal case */
! appendPQExpBufferChar(output_buf,
!
(char) strtol(yytext + 1, NULL, 0));
! }
!
! \\0[0-7]* {
/* octal case */
appendPQExpBufferChar(output_buf,
!
(char) strtol(yytext + 1, NULL, 0));
! }
!
! \\0[xX][0-9A-Fa-f]+ {
! /* hex case */
! appendPQExpBufferChar(output_buf,
!
(char) strtol(yytext + 1, NULL, 0));
! }
!
! \\0[xX] {
! /* failed hex case */
! yyless(2);
! appendPQExpBufferChar(output_buf,
!
(char) strtol(yytext + 1, NULL, 0));
}
\\. { emit(yytext + 1, 1); }
--- 849,858
\\r { appendPQExpBufferChar(output_buf, '\r'); }
\\f { appendPQExpBufferChar(output_buf, '\f'); }
! \\[0-7]{1,3}{
/* octal case */
appendPQExpBufferChar(output_buf,
!
(char) strtol(yytext + 1, NULL, 8));
}
\\. { emit(yytext + 1, 1); }
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] [HACKERS] patches for items from TODO list
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Here is an updated version of the COPY \x patch. It is the first patch attached. Also, I realized that if we support \x in COPY, we should also support \x in strings to the backend. This is the second patch. Do we really want to do any of these things? We've been getting beaten up recently about the fact that we have non-SQL-spec string escapes (ie, all the backslash stuff) so I'm a bit dubious about adding more, especially when there's little if any demand for it. I don't object too much to the COPY addition, since that's outside any spec anyway, but I do think we ought to think twice about adding this to SQL literal handling. +1 from me on this for Tom -- please don't break spec compliance! Best Regards, Michael Paesold ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] patches for items from TODO list
Here is an updated version of the COPY \x patch. It is the first patch
attached.
Also, I realized that if we support \x in COPY, we should also support
\x in strings to the backend. This is the second patch.
Third, I found out that psql has some unusual handling of escaped
numbers. Instead of using \ddd as octal, it has \ddd is decimal, \0ddd
is octal, and \0xddd is decimal. It is basically following the strtol()
rules for an escaped value. This seems confusing and contradicts how
the rest of our system works. I looked at 'bash' and found that it
supports the \000 and \x00 just like C, so I am confused why we have
the current behavior. This patch makes psql consistent with the rest of
our system for back slashes. It does break backward compatibility. It
wouldn't be a big deal to fix, except we document this in the psql
manual page, and that adds confusion.
FYI, here is the current psql behavior:
test= \set x '\42'
test= \echo :x
*
test= \set x '\042'
test= \echo :x
test= \set x '\0x42'
test= \echo :x
B
The new behavior is:
test= \set x '\42'
test= \echo :x
test= \set x '\042'
test= \echo :x
test= \set x '\x42'
test= \echo :x
B
--
Bruce Momjian| http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: doc/src/sgml/ref/copy.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/copy.sgml,v
retrieving revision 1.65
diff -c -c -r1.65 copy.sgml
*** doc/src/sgml/ref/copy.sgml 7 May 2005 02:22:45 - 1.65
--- doc/src/sgml/ref/copy.sgml 28 May 2005 14:02:59 -
***
*** 424,436
entryBackslash followed by one to three octal digits specifies
the character with that numeric code/entry
/row
/tbody
/tgroup
/informaltable
! Presently, commandCOPY TO/command will never emit an octal-digits
! backslash sequence, but it does use the other sequences listed above
! for those control characters.
/para
para
--- 424,441
entryBackslash followed by one to three octal digits specifies
the character with that numeric code/entry
/row
+ row
+entryliteral\x/replaceabledigits//entry
+entryBackslash literalx/ followed by one or two hex digits
specifies
+the character with that numeric code/entry
+ /row
/tbody
/tgroup
/informaltable
! Presently, commandCOPY TO/command will never emit an octal or
! hex-digits backslash sequence, but it does use the other sequences
! listed above for those control characters.
/para
para
Index: src/backend/commands/copy.c
===
RCS file: /cvsroot/pgsql/src/backend/commands/copy.c,v
retrieving revision 1.244
diff -c -c -r1.244 copy.c
*** src/backend/commands/copy.c 7 May 2005 02:22:46 - 1.244
--- src/backend/commands/copy.c 28 May 2005 14:03:07 -
***
*** 2274,2279
--- 2274,2291
return result;
}
+ /*
+ *Return decimal value for a hexadecimal digit
+ */
+ static
+ int GetDecimalFromHex(char hex)
+ {
+ if (isdigit(hex))
+ return hex - '0';
+ else
+ return tolower(hex) - 'a' + 10;
+ }
+
/*--
* Read the value of a single attribute, performing de-escaping as needed.
*
***
*** 2335,2340
--- 2347,2353
case '5':
case '6':
case '7':
+ /* handle \013 */
{
int val;
***
*** 2360,2365
--- 2373,2402
c = val 0377;
}
break;
+ case 'x':
+ /* Handle \x3F */
+ if (line_buf.cursor line_buf.len)
+ {
+ char hexchar =
line_buf.data[line_buf.cursor];
+
+ if (isxdigit(hexchar))
+ {
+ int val =
GetDecimalFromHex(hexchar);
+
+ line_buf.cursor++;
+ if (line_buf.cursor
line_buf.len)
+
Re: [PATCHES] [HACKERS] patches for items from TODO list
Bruce Momjian pgman@candle.pha.pa.us writes: Here is an updated version of the COPY \x patch. It is the first patch attached. Also, I realized that if we support \x in COPY, we should also support \x in strings to the backend. This is the second patch. Do we really want to do any of these things? We've been getting beaten up recently about the fact that we have non-SQL-spec string escapes (ie, all the backslash stuff) so I'm a bit dubious about adding more, especially when there's little if any demand for it. I don't object too much to the COPY addition, since that's outside any spec anyway, but I do think we ought to think twice about adding this to SQL literal handling. Third, I found out that psql has some unusual handling of escaped numbers. Instead of using \ddd as octal, it has \ddd is decimal, \0ddd is octal, and \0xddd is decimal. It is basically following the strtol() rules for an escaped value. This seems confusing and contradicts how the rest of our system works. I agree, that's just going to confuse people. ! xqescape[\\][^0-7x] If you are going to insist on this, at least make it case-insensitive. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] patches for items from TODO list
I have extracted the COPY \x part of your patch, and have attached it for later application to CVS for 8.1. --- Sergey Ten wrote: Hello all, Thank you to all who replied for suggestions and help. Enclosed please find code changes for the following items: - Allow COPY to understand \x as a hex byte, and - Add XML output to COPY The changes include implementation of the features as well as modification of the copy regression test. After a careful consideration we decided to - put XML implementation in the backend and - use XML format described below, with justification of our decision. The XML schema used by the COPY TO command was designed for ease of use and to avoid the problem of column names appearing in XML element names. XML doesn't allow spaces and punctuation in element names but Postgres does allow these characters in column names; therefore, a direct mapping would be problematic. The solution selected places the column names into attribute fields where any special characters they contain can be properly escaped using XML entities. An additional attribute is used to distinguish null fields from empty ones. The example below is taken from the test suite. It demonstrates some basic XML escaping in row 2. Row 3 demonstrates the difference between an empty string (in col2) and a null string (in col3). If a field is null it will always be empty but a field which is empty may or may not be null. Always check the value of the 'null' attribute to be sure when a field is truly null. ?xml version='1.0'? table row col name='col1' null='n'Jackson, Sam/col col name='col2' null='n'\h/col /row row col name='col1' null='n'It is quot;perfectquot;./col col name='col2' null='n'#09;/col /row row col name='col1' null='n'/col col name='col2' null='y'/col /row /table Please let us know if about any concerns, objections the proposed change may cause. Best regards, Jason Lucas, Sergey Ten SourceLabs -Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 11, 2005 7:11 PM To: Sergey Ten Cc: pgsql-hackers@postgresql.org; [EMAIL PROTECTED] Subject: Re: [HACKERS] patches for items from TODO list Sergey Ten wrote: Hello all, We would like to contribute to the Postgresql community by implementing the following items from the TODO list (http://developer.postgresql.org/todo.php): . Allow COPY to understand \x as a hex byte . Allow COPY to optionally include column headings in the first line . Add XML output to COPY The changes are straightforward and include implementation of the features as well as modification of the regression tests and documentation. Before sending a diff file with the changes, we would like to know if these features have been already implemented. Please check the web site version. Someone has already implemented Allow COPY to optionally include column headings in the first line. As far as XML, there has been discussion on where that should be done? In the backend, libpq, or psql. It will need discussion on hackers. I assume you have read the developer's FAQ too. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 [ Attachment, skipping... ] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 Index: doc/src/sgml/ref/copy.sgml === RCS file: /cvsroot/pgsql/doc/src/sgml/ref/copy.sgml,v retrieving revision 1.65 diff -c -c -r1.65 copy.sgml *** doc/src/sgml/ref/copy.sgml 7 May 2005 02:22:45 - 1.65 --- doc/src/sgml/ref/copy.sgml 28 May 2005 03:49:10 - *** *** 424,436 entryBackslash followed by one to three octal digits specifies the character with that numeric code/entry /row /tbody /tgroup /informaltable ! Presently, commandCOPY TO/command will never emit an octal-digits ! backslash sequence, but it does use the other sequences listed above ! for those control characters. /para para --- 424,441 entryBackslash followed by one to three octal digits specifies
