Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-21 Thread Bruce Momjian
Added to TODO: * Research storing disk pages with no alignment/padding --- Lincoln Yeoh wrote: I believe that one should leave such on-the-fly disk compression to the O/S. Postgresql already does compression for

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-21 Thread Dawid Kuroczko
On 7/19/05, Jim C. Nasby [EMAIL PROTECTED] wrote: CREATE TABLE sample1 ( a boolean, b int, c boolean ); ...it will take more storage than: CREATE TABLE sample2 ( b int, a boolean, c boolean ); Actually, I believe that's the case with just about

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-21 Thread Scott Marlowe
On Thu, 2005-07-21 at 09:23, Dawid Kuroczko wrote: On 7/19/05, Jim C. Nasby [EMAIL PROTECTED] wrote: CREATE TABLE sample1 ( a boolean, b int, c boolean ); ...it will take more storage than: CREATE TABLE sample2 ( b int, a boolean, c

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-19 Thread Jim C. Nasby
On Tue, Jul 19, 2005 at 02:02:28AM +0200, Dawid Kuroczko wrote: On 7/18/05, Lincoln Yeoh lyeoh@pop.jaring.my wrote: However, maybe padding for alignment is a waste on the disk - disks being so much slower than CPUs (not sure about that once the data is in memory ). Maybe there should be an

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-19 Thread Martijn van Oosterhout
On Tue, Jul 19, 2005 at 02:02:28AM +0200, Dawid Kuroczko wrote: Out of curiosity, do I understand right that if I create table CREATE TABLE sample1 ( a boolean, b int, c boolean ); ...it will take more storage than: CREATE TABLE sample2 ( b int, a boolean,

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-19 Thread Jim C. Nasby
On Tue, Jul 19, 2005 at 11:48:16AM +0200, Martijn van Oosterhout wrote: int, bool, bool 6 bytes, no padding bool, int, bool 9 bytes, including 3 bytes padding bool, bool, int 8 bytes, including 2 bytes padding We store bool's in a byte and don't compact? That would be another very

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-18 Thread Lincoln Yeoh
I believe that one should leave such on-the-fly disk compression to the O/S. Postgresql already does compression for TOAST. However, maybe padding for alignment is a waste on the disk - disks being so much slower than CPUs (not sure about that once the data is in memory ). Maybe there should

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-18 Thread Dawid Kuroczko
On 7/18/05, Lincoln Yeoh lyeoh@pop.jaring.my wrote: However, maybe padding for alignment is a waste on the disk - disks being so much slower than CPUs (not sure about that once the data is in memory ). Maybe there should be an option to reorder columns so that less space is wasted. Out of

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-17 Thread Jim C. Nasby
On Sat, Jul 16, 2005 at 03:18:24PM -0700, Ron Mayer wrote: Jim C. Nasby wrote: On Thu, Jul 14, 2005 at 11:29:23PM +0200, Martijn van Oosterhout wrote: On Thu, Jul 14, 2005 at 11:30:36AM -0500, Jim C. Nasby wrote: On Wed, Jul 13, 2005 at 07:52:04PM -0400, Bruce Momjian wrote: This is a good

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-16 Thread Ron Mayer
Jim C. Nasby wrote: On Thu, Jul 14, 2005 at 11:29:23PM +0200, Martijn van Oosterhout wrote: On Thu, Jul 14, 2005 at 11:30:36AM -0500, Jim C. Nasby wrote: On Wed, Jul 13, 2005 at 07:52:04PM -0400, Bruce Momjian wrote: This is a good point. We have always stored data on disk that exactly

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-14 Thread Jim C. Nasby
On Wed, Jul 13, 2005 at 07:52:04PM -0400, Bruce Momjian wrote: Ron Mayer wrote: Martijn van Oosterhout wrote: Well, you get another issue, alignment. If you squeeze your string down, the next field, if it is an int or string, will get padded to a multiple of 4 negating most of the

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-14 Thread Martijn van Oosterhout
On Thu, Jul 14, 2005 at 11:30:36AM -0500, Jim C. Nasby wrote: On Wed, Jul 13, 2005 at 07:52:04PM -0400, Bruce Momjian wrote: This is a good point. We have always stored data on disk that exactly matches its layout in memory. We could change that, but no one has shown it would be a win.

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-14 Thread Jim C. Nasby
On Thu, Jul 14, 2005 at 11:29:23PM +0200, Martijn van Oosterhout wrote: On Thu, Jul 14, 2005 at 11:30:36AM -0500, Jim C. Nasby wrote: On Wed, Jul 13, 2005 at 07:52:04PM -0400, Bruce Momjian wrote: This is a good point. We have always stored data on disk that exactly matches its layout in

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-13 Thread Dawid Kuroczko
On 7/13/05, Tom Lane [EMAIL PROTECTED] wrote: Greg Stark [EMAIL PROTECTED] writes: Personally I would settle for a fuller set of small fixed size datatypes. The char datatype is pretty much exactly what's needed except that it provides such a quirky interface. I'm not actually against

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-13 Thread Dawid Kuroczko
On 7/12/05, Joe [EMAIL PROTECTED] wrote: Dawid Kuroczko wrote: smallint takes two bytes. Numeric(1) will take around 10 bytes and char(1) will take 5 bytes (4 bytes for length of data). I never would've imagined *that* amount of overhead for CHAR(1)! I would've imagined that it would

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-13 Thread Martijn van Oosterhout
On Wed, Jul 13, 2005 at 10:48:56AM +0200, Dawid Kuroczko wrote: As for the char/varchar type -- I was wondering. Worst case scenario for UTF-8 (correct me on this) is when 1 character takes 4 bytes. And biggest problem with char/varchar is that length indicator takes 4 bytes... How much

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-13 Thread Ron Mayer
Martijn van Oosterhout wrote: Well, you get another issue, alignment. If you squeeze your string down, the next field, if it is an int or string, will get padded to a multiple of 4 negating most of the gains. Like in C structures, there is padding to optimise access. Anecdotally I hear at

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-13 Thread Bruce Momjian
Ron Mayer wrote: Martijn van Oosterhout wrote: Well, you get another issue, alignment. If you squeeze your string down, the next field, if it is an int or string, will get padded to a multiple of 4 negating most of the gains. Like in C structures, there is padding to optimise access.

[GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Joe
I have a MySQL database that I'm converting to PostgreSQL which has 10 columns with TINYINT type, i.e., a one-byte integer. Only one of them qualifies as a true BOOLEAN. Two are entity identifiers (for limited range classes or categories) and three others are type/code values. The last four

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread John DeSoi
On Jul 12, 2005, at 1:16 AM, Joe wrote: I have a MySQL database that I'm converting to PostgreSQL which has 10 columns with TINYINT type, i.e., a one-byte integer. Only one of them qualifies as a true BOOLEAN. Two are entity identifiers (for limited range classes or categories) and

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Tom Lane
Joe [EMAIL PROTECTED] writes: I have a MySQL database that I'm converting to PostgreSQL which has 10 columns with TINYINT type, i.e., a one-byte integer. I'm wondering what would be the best conversion choice for these columns: smallint, numeric(1), char(1), something else? smallint, for

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Dawid Kuroczko
On 7/12/05, Joe [EMAIL PROTECTED] wrote: I have a MySQL database that I'm converting to PostgreSQL which has 10 columns with TINYINT type, i.e., a one-byte integer. Only one of them qualifies as a true BOOLEAN. Two are entity identifiers (for limited range classes or categories) and three

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Alvaro Herrera
On Tue, Jul 12, 2005 at 01:16:07AM -0400, Joe wrote: I have a MySQL database that I'm converting to PostgreSQL which has 10 columns with TINYINT type, i.e., a one-byte integer. Only one of them qualifies as a true BOOLEAN. Two are entity identifiers (for limited range classes or

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Joe
Dawid Kuroczko wrote: smallint takes two bytes. Numeric(1) will take around 10 bytes and char(1) will take 5 bytes (4 bytes for length of data). I never would've imagined *that* amount of overhead for CHAR(1)! I would've imagined that it would take up one byte (or two with a NULL

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Tom Lane
Joe [EMAIL PROTECTED] writes: I never would've imagined *that* amount of overhead for CHAR(1)! I would've imagined that it would take up one byte (or two with a NULL indicator). After all, we're not talking about VARCHAR(1) [which is sort of useless]. Don't the catalogs know the declared

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Joe
Tom Lane wrote: Because the length specification is in *characters*, which is not by any means the same as *bytes*. We could possibly put enough intelligence into the low-level tuple manipulation routines to count characters in whatever encoding we happen to be using, but it's a lot faster and

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Alvaro Herrera
On Tue, Jul 12, 2005 at 05:37:32PM -0400, Joe wrote: Tom Lane wrote: Because the length specification is in *characters*, which is not by any means the same as *bytes*. We could possibly put enough intelligence into the low-level tuple manipulation routines to count characters in whatever

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Joe
Alvaro Herrera wrote: On Tue, Jul 12, 2005 at 05:37:32PM -0400, Joe wrote: If it stored character data in Unicode (UCS-16) it would always take up two-bytes per character. Really? We don't support UCS-16, for good reasons (we'd have to rewrite several parts of the code in order to support

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: I don't see how UCS-16 could always use only 2 bytes. Simple: it fails to handle Unicode code points above 0x1. (We only recently fixed a similar limitation in our UTF8 support, by the by, but it *is* fixed and I doubt we want to backpedal.) The

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Certainly the idea of not having to store a length word for CHAR(1) fields is not going to inspire anyone to invest the effort involved ;-) That's a pretty big motivation though. Storage space efficiency is a huge factor in raw sequential scan speed.

Re: [GENERAL] Converting MySQL tinyint to PostgreSQL

2005-07-12 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Personally I would settle for a fuller set of small fixed size datatypes. The char datatype is pretty much exactly what's needed except that it provides such a quirky interface. I'm not actually against inventing an int1/tinyint type. I used to be worried