Re: [PATCHES] cast bytea to/from bit strings
Dear Tom, I think that the inability to convert nearly binary compatible standard types one to the other is a postgresql issue. Even if it is not often useful, the point is completeness and soundness of the type provided by the core. OK, can I get some feedback from others about this patch? I think Fabien is way overstating his case here. Maybe. It's not immediately obvious that there should be a cast between bit(n) and bytea, and it's even less obvious that it should be done in a way that exposes the internal representation of bit(n) as this does. Hmmm... I think people guessed it anyway;-) Well, if there is a big/little endian issue, I agree that it is not a good idea. As I cast at the byte level, it seems to me that it should be okay. If so, I see no real harm in having an *explicit* cast allowed, which by nature may be a little destructive, as long as it is reasonnable. There's no principled reason for one bit ordering over the other, for example, nor any very clean way to handle coercions where N isn't a multiple of 8. It could be rejected instead. I think this request has more to do with a lack of adequate operators for one type or the other. If we're missing, say, bitwise logical operators for bytea, then let's add those rather than create a bogus equivalence between the types. Indeed, what triggers my development for this cast was that I needed a xor on md5 results, which can only be converted to bytea with convert. I could develop a bunch of bitwise operators for bytea, but casting to varbit where they are already available was the quickest path. -- Fabien. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] cast bytea to/from bit strings
Bruce Momjian writes: > Fabien COELHO wrote: >> I think that the inability to convert nearly binary compatible standard >> types one to the other is a postgresql issue. Even if it is not often >> useful, the point is completeness and soundness of the type provided by >> the core. > OK, can I get some feedback from others about this patch? I think Fabien is way overstating his case here. It's not immediately obvious that there should be a cast between bit(n) and bytea, and it's even less obvious that it should be done in a way that exposes the internal representation of bit(n) as this does. There's no principled reason for one bit ordering over the other, for example, nor any very clean way to handle coercions where N isn't a multiple of 8. I think this request has more to do with a lack of adequate operators for one type or the other. If we're missing, say, bitwise logical operators for bytea, then let's add those rather than create a bogus equivalence between the types. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] cast bytea to/from bit strings
Fabien COELHO wrote: > > Dear Bruce, > > > I am not sure this is of general enough usefulness to be in the backend. > > Hmm... > > I think that the inability to convert nearly binary compatible standard > types one to the other is a postgresql issue. Even if it is not often > useful, the point is completeness and soundness of the type provided by > the core. More over I haven't found any work around with decode/encode and > other casts functions. Bytea is somehow an isolated type, which makes it > not so useful from within the database. > > > Can you add it as a pgfoundry project? > > I could do it, but ISTM it is really overkill for two stupid 10 lines > functions that deal with internal core types. OK, can I get some feedback from others about this patch? -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] cast bytea to/from bit strings
Dear Bruce, I am not sure this is of general enough usefulness to be in the backend. Hmm... I think that the inability to convert nearly binary compatible standard types one to the other is a postgresql issue. Even if it is not often useful, the point is completeness and soundness of the type provided by the core. More over I haven't found any work around with decode/encode and other casts functions. Bytea is somehow an isolated type, which makes it not so useful from within the database. Can you add it as a pgfoundry project? I could do it, but ISTM it is really overkill for two stupid 10 lines functions that deal with internal core types. -- Fabien. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] cast bytea to/from bit strings
I am not sure this is of general enough usefulness to be in the backend. Can you add it as a pgfoundry project? --- Fabien COELHO wrote: > > Dear PostgreSQL developers, > > Please find attached a small patch to convert bytea to bit strings and > vice versa. > > I used it in order to be able xor md5 results so as to checksum bundles of > tuples together. The MD5 result is an hexa text convertible to bytea with > decode, but then I was stuck... > > ISTM that having these types explicitely castable may be useful to others, > hence this small contribution. The cast allows to work on a bytea at the > bit level and to perform bitwise operations. > > ./src/backend/utils/adt/varbit.c > - add two conversion functions > > ./src/include/catalog/pg_proc.h > - declare the above functions in the catalog > > ./src/include/catalog/pg_cast.h > - declare the 4 explicit casts > > ./src/test/regress/sql/bit.sql > - test all those new casts > > ./src/test/regress/expected/bit.out > - new regression results > > ./src/test/regress/expected/opr_sanity.out > - pg figures out that bit and varbit are binary compatible, > which is the case (well, at least I assumed it). > > -- > Fabien. Content-Description: [ Attachment, skipping... ] > > ---(end of broadcast)--- > TIP 4: Have you searched our list archives? > >http://archives.postgresql.org -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[PATCHES] cast bytea to/from bit strings
Dear PostgreSQL developers,
Please find attached a small patch to convert bytea to bit strings and
vice versa.
I used it in order to be able xor md5 results so as to checksum bundles of
tuples together. The MD5 result is an hexa text convertible to bytea with
decode, but then I was stuck...
ISTM that having these types explicitely castable may be useful to others,
hence this small contribution. The cast allows to work on a bytea at the
bit level and to perform bitwise operations.
./src/backend/utils/adt/varbit.c
- add two conversion functions
./src/include/catalog/pg_proc.h
- declare the above functions in the catalog
./src/include/catalog/pg_cast.h
- declare the 4 explicit casts
./src/test/regress/sql/bit.sql
- test all those new casts
./src/test/regress/expected/bit.out
- new regression results
./src/test/regress/expected/opr_sanity.out
- pg figures out that bit and varbit are binary compatible,
which is the case (well, at least I assumed it).
--
Fabien.*** ./src/backend/utils/adt/varbit.c.orig Sun Mar 5 16:58:44 2006
--- ./src/backend/utils/adt/varbit.cThu May 4 15:57:34 2006
***
*** 1484,1486
--- 1484,1554
}
PG_RETURN_INT32(0);
}
+
+ /* create a bit string from a byte array.
+ */
+ Datum varbitfrombytea(PG_FUNCTION_ARGS)
+ {
+ bytea *arg = PG_GETARG_BYTEA_P(0);
+ int32 typmod = PG_GETARG_INT32(1); /* for ::BIT(10) syntax */
+ int datalen = VARSIZE(arg) - VARHDRSZ;
+ int bitlen, len, resdatalen, needlen;
+ VarBit *result;
+
+ /* truncate or expand if required */
+ if (typmod>=0)
+ {
+ bitlen = typmod;
+ resdatalen = (bitlen + BITS_PER_BYTE - 1) / BITS_PER_BYTE;
+ needlen = datalen>resdatalen? resdatalen: datalen;
+ }
+ else
+ {
+ resdatalen = datalen;
+ bitlen = BITS_PER_BYTE * datalen;
+ needlen = datalen;
+ }
+
+ len = VARBITTOTALLEN(bitlen);
+ result = (VarBit *) palloc(len);
+ VARATT_SIZEP(result) = len;
+ VARBITLEN(result) = bitlen;
+ memcpy(VARBITS(result), VARDATA(arg), needlen);
+
+ if (resdatalen > needlen)
+ {
+ char *ptr = VARBITS(result) + needlen;
+ while (needlen BIT/VARBIT */
+ DATA(insert OID = 2790 ( bytea PGNSP PGUID 12 f f t f i 3 17
"1560 23 16" _null_ _null_ _null_ varbittobytea - _null_));
+ DESCR("convert bit() to bytea");
+ DATA(insert OID = 2791 ( bytea PGNSP PGUID 12 f f t f i 3 17
"1562 23 16" _null_ _null_ _null_ varbittobytea - _null_));
+ DESCR("convert varbit() to bytea");
+ DATA(insert OID = 2792 ( tobit PGNSP PGUID 12 f f t f i 3 1560
"17 23 16" _null_ _null_ _null_ varbitfrombytea - _null_));
+ DESCR("convert bytea to bit()");
+ DATA(insert OID = 2793 ( varbit PGNSP PGUID 12 f f t f i 3 1562
"17 23 16" _null_ _null_ _null_ varbitfrombytea - _null_));
+ DESCR("convert bytea to varbit()");
+
/*
* Symbolic values for provolatile column: these indicate whether the result
* of a function is dependent *only* on the values of its explicit arguments,
*** ./src/test/regress/expected/bit.out.origSun Jul 27 06:53:11 2003
--- ./src/test/regress/expected/bit.out Thu May 4 16:37:04 2006
***
*** 509,511
--- 509,598
DROP TABLE BIT_SHIFT_TABLE;
DROP TABLE VARBIT_SHIFT_TABLE;
+ -- bytea casts
+ -- x80
+ SELECT
+ (VARBIT '1')::BYTEA,
+ (BIT '1')::BYTEA,
+ (VARBIT '1000')::BYTEA,
+ (BIT '1000')::BYTEA,
+ (VARBIT '1000')::BYTEA,
+ (BIT '1000')::BYTEA;
+ bytea | bytea | bytea | bytea | bytea | bytea
+ ---+---+---+---+---+---
+ \200 | \200 | \200 | \200 | \200 | \200
+ (1 row)
+
+ SELECT
+ (VARBIT '01110101010011')::BYTEA AS "CAL",
+ (VARBIT '0101011')::BYTEA AS "V",
+ (VARBIT '010010010100111')::BYTEA AS "IN";
+ CAL | V | IN
+ -+---+
+ CAL | V | IN
+ (1 row)
+
+ -- bit and varbit casts
+ SELECT (BYTEA 'hello')::VARBIT AS "hello";
+ hello
+ --
+ 01101110010101101100011011000110
+ (1 row)
+
+ SELECT (BYTEA 'he')::VARBIT;
+ varbit
+ --
+ 011011100101
+ (1 row)
+
+ SELECT (BYTEA 'he')::BIT(4) AS "restricted bit(4)";
+ restricted bit(4)
+ ---
+ 0110
+ (1 row)
+
+ SELECT (BYTEA 'he')::BIT(12) AS "restricted bit(12)";
+ restricted bit(12)
+
+ 01101110
+ (1 row)
+
+ SELECT (BYTEA 'he')::BIT(16) AS "bit(16)";
+ bit(16)
+ --
+ 011011100101
+ (1 row)
+
+ SELECT (BYTEA 'he')::BIT(20) AS "expanded bit(20)";
+expanded bit(20)
+ --
+ 011011100101
+ (1 row)
+
+ SELECT (BYTEA 'he')::BIT(33) AS "expanded bit(33)";
+ expanded bit(33)
