Re: bytea bitwise logical operations implementation (xor / and / or / not)

[Craig Ringer]

On 13 January 2018 at 01:57, Christian Rossow 
wrote:

> Hi Fabien,
>
> > I think that the probability of getting these useful things into pg is
> > alas small. In the mean time, you may package and register it as an
> > extension?
> I aimed to close the asymmetry between bit vector operations (they also
> offer xor/and/etc.) and bytea operations. My code is more or less a 1:1
> copy of the corresponding bit vector operations (in fact, just a tiny
> bit easier, since bytea is byte-aligned, wheres bit vectors aren't).
>
> I just wanted to dump the code here, given that Tom suggested (in 2006)
> such an implementation might be a useful addition to bytea. But I won't
> feel offended in any way if the code won't be merged.
>

Seems like a good idea to me.

Conversion between bit varying and bytea would be a big help.

So would casts from bit varying or bytea <-> integer and other types.

-- 
 Craig Ringer   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: bytea bitwise logical operations implementation (xor / and / or / not)

[Fabien COELHO]

Hello Christian,


Currently, `bytea` does not have any bitwise logical operations yet.
This issue came up in an old thread from 2006 [1], but nobody seemed to
have picked this issue so far.


I remember this one because I needed them for checksuming set of rows. 
There is a whole set of missing (from my point of view) operators, casts 
and aggregates.


See https://github.com/zx80/pg_comparator where I packaged the subset I 
needed as an extension. In particular, there is a bitxor which can be used 
for bytea if casting is allowed.



Tested on PG 9.6. I hope you find this useful.


I think that the probability of getting these useful things into pg is 
alas small. In the mean time, you may package and register it as an 
extension?


--
Fabien.

bytea bitwise logical operations implementation (xor / and / or / not)

[Christian Rossow]

Hackers,

Currently, `bytea` does not have any bitwise logical operations yet.
This issue came up in an old thread from 2006 [1], but nobody seemed to
have picked this issue so far.

Being in the need for this myself, I copied the bit vector's bitwise
logical operations and converted them to bytea. I'm using this as a
PostgreSQL extension right now, but would be very happy to see this
integrated into mainstream as default bytea operations in the future.

Find attached the implementation, plus a SQL file, for:

 * bytea_xor
 * bytea_and
 * bytea_or
 * bytea_not
 * bytea_bitsset (returns number of set bits in a bytea; feel free to
  drop this one if you don't see utility)

Tested on PG 9.6. I hope you find this useful.

Cheers,
Christian

[1]: https://www.postgresql.org/message-id/5171.1146927915%40sss.pgh.pa.us
DROP FUNCTION IF EXISTS bytea_xor(bytea, bytea);
CREATE FUNCTION bytea_xor(bytea, bytea) RETURNS bytea
AS 'bytea_bitops.so', 'bytea_xor'
LANGUAGE C STRICT;

DROP FUNCTION IF EXISTS bytea_and(bytea, bytea);
CREATE FUNCTION bytea_and(bytea, bytea) RETURNS bytea
AS 'bytea_bitops.so', 'bytea_and'
LANGUAGE C STRICT;

DROP FUNCTION IF EXISTS bytea_or(bytea, bytea);
CREATE FUNCTION bytea_or(bytea, bytea) RETURNS bytea
AS 'bytea_bitops.so', 'bytea_or'
LANGUAGE C STRICT;

DROP FUNCTION IF EXISTS bytea_not(bytea);
CREATE FUNCTION bytea_not(bytea) RETURNS bytea
AS 'bytea_bitops.so', 'bytea_not'
LANGUAGE C STRICT;

DROP FUNCTION IF EXISTS bytea_bitsset(bytea);
CREATE FUNCTION bytea_bitsset(bytea) RETURNS integer
AS 'bytea_bitops.so', 'bytea_bitsset'
LANGUAGE C STRICT;
/* bytea_bitops.c
 *
 * Adding bytea bit operation functions similar to varbit's bit operations.
 * Written by Christian Rossow
 * License: Do whatever you want with this code.
 
 * To build:
 * gcc -I`pg_config --includedir` -fpic -c bytea_bitops.c
 * gcc -shared -o bytea_bitops.so bytea_bitops.o
 * sudo cp bytea_bitops.so /usr/lib/postgresql/9.1/lib/
 * cd /usr/lib/postgresql/9.1/lib/
 * sudo chmod +r bytea_bitops.so
 */

#include 
#include 

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif

PG_FUNCTION_INFO_V1(bytea_xor);
Datum bytea_xor(PG_FUNCTION_ARGS)
{
bytea*  ba1 = PG_GETARG_BYTEA_P(0);
bytea*  ba2 = PG_GETARG_BYTEA_P(1);
bytea*  result;
char*   str1 = VARDATA(ba1);
char*   str2 = VARDATA(ba2);
char*   resstr;
int32   len;
int32   i;

if (VARSIZE(ba1) != VARSIZE(ba2))
{
ereport(ERROR,
(errcode(ERRCODE_STRING_DATA_LENGTH_MISMATCH),
 errmsg("cannot XOR bytea of different 
sizes")));
}

len = VARSIZE(ba1);
result = (bytea *) palloc(len);
SET_VARSIZE(result, len);

resstr = VARDATA(result);
for (i=0; i < len - VARHDRSZ; ++i)
{ 
resstr[i] = str1[i] ^ str2[i];
}

PG_RETURN_BYTEA_P(result);
}

PG_FUNCTION_INFO_V1(bytea_and);
Datum bytea_and(PG_FUNCTION_ARGS)
{
bytea*  ba1 = PG_GETARG_BYTEA_P(0);
bytea*  ba2 = PG_GETARG_BYTEA_P(1);
bytea*  result;
char*   str1 = VARDATA(ba1);
char*   str2 = VARDATA(ba2);
char*   resstr;
int32   len;
int32   i;

if (VARSIZE(ba1) != VARSIZE(ba2))
{
ereport(ERROR,
(errcode(ERRCODE_STRING_DATA_LENGTH_MISMATCH),
 errmsg("cannot AND bytea of different 
sizes")));
}

len = VARSIZE(ba1);
result = (bytea *) palloc(len);
SET_VARSIZE(result, len);

resstr = VARDATA(result);
for (i=0; i < len - VARHDRSZ; ++i)
{ 
resstr[i] = str1[i] & str2[i];
}

PG_RETURN_BYTEA_P(result);
}

PG_FUNCTION_INFO_V1(bytea_or);
Datum bytea_or(PG_FUNCTION_ARGS)
{
bytea*  ba1 = PG_GETARG_BYTEA_P(0);
bytea*  ba2 = PG_GETARG_BYTEA_P(1);
bytea*  result;
char*   str1 = VARDATA(ba1);
char*   str2 = VARDATA(ba2);
char*   resstr;
int32   len;
int32   i;

if (VARSIZE(ba1) != VARSIZE(ba2))
{
ereport(ERROR,
(errcode(ERRCODE_STRING_DATA_LENGTH_MISMATCH),
 errmsg("cannot AND bytea of different 
sizes")));
}

len = VARSIZE(ba1);
result = (bytea *) palloc(len);
SET_VARSIZE(result, len);

resstr = VARDATA(result);
for (i=0; i < len - VARHDRSZ; ++i)
{ 
resstr[i] = str1[i] | str2[i];
}

PG_RETURN_BYTEA_P(result);
}

PG_FUNCTION_INFO_V1(bytea_not);
Datum bytea_not(PG_FUNCTION_ARGS)
{
bytea*  ba1 = PG_GETARG_BYTEA_P(0);
bytea*  result;
char*   str1 = VARDATA(ba1);
char*   resstr;
int32   len = VARSIZE(ba1);
int32   i;

result = (bytea *) palloc(len);
SET_VARSIZE(result, len);

resstr = VARDATA(result);
for (i=0; i < len - VARHDRSZ; ++i)
{ 
resstr[i] = ~str1[i];
}

PG_RETURN_BYTEA_P(result);
}

PG_FUNCTION_INFO_V1(bytea_bitsset);
Datum bytea_bitsset(PG_FUNCTION_ARGS)
{
bytea*  ba1 = PG_GETARG_BYTEA_P(0);