Re: ubytes to ulong problem
On 12/21/2013 07:57 PM, Ali Çehreli wrote:
On 12/21/2013 05:44 PM, Charles Hixson wrote:
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:
Ideally the compiler will optimise your version to be fast, but
you may
find you get better performance by doing the bit manipulations
eplicitly:
Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:
import std.bitmanip;
import std.system;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);
if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);
} else {
return value;
}
}
Ali
Will that work even when the alignment is to odd bytes? Because that's
the case I was really worried about. The ubyte array is a packed
mixture of types, some of which are isolated bytes.
No, it is not guaranteed to work unless the alignment is right.
How about this, then: :)
import std.array;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = block.front;
block.popFront();
foreach (ub; block) {
value = 8;
value |= ub;
}
return value;
}
Ali
Nice, but the block is longer than 8 bytes, so I should use a for (i =
n; i n + 8; i++) rather than a foreach, and index off of i. I
clearly need to redo the documentation a bit (even though it's form me
of a few months from now). It needs to say something like Convert a 8
byte slice from a ubyte array starting at index n into a ulong. n
should always be required to be specified, so I don't want a default
value. (0 was used as a test case, because I'd made a really stupid
mistake and used ^ for exponentiation, and then couldn't see what was
going on, so I was simplifying everything...and I still couldn't see
it. Actually the array starts with a ushort, which specifies the number
of ulongs to follow before a bunch of bytes that are unintelligible data
to the class that's using this function. (OTOH, it seems like something
generally useful, so I'll probably put it in a utils.d file, with some
other generally useful routines.)
OTOH, if I'm going to consider this to be a general utility function,
then I really don't want to make assumptions about where things start,
etc. Perhaps I should throw an exception (other than assertion error)
if the index is bad or the array is to short for the given index. I
need to think about that a bit more. The alternative is to use enforce
rather than assertions...though as long as I'm the only user assertions
suffice. (It's not going to be separately compiled.)
--
Charles Hixson
Re: ubytes to ulong problem
On 12/22/2013 01:04 AM, Charles Hixson wrote: Nice, but the block is longer than 8 bytes, so I should use a for (i = n; i n + 8; i++) rather than a foreach, and index off of i. Makes sense. That reminded me of the Phobos function that does exactly what you want. Have you considered std.bitmanip.read? Ali
Re: ubytes to ulong problem
On Sunday, 22 December 2013 at 03:57:38 UTC, Ali Çehreli wrote:
On 12/21/2013 05:44 PM, Charles Hixson wrote:
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:
Ideally the compiler will optimise your version to be fast,
but you may
find you get better performance by doing the bit
manipulations
eplicitly:
Assuming that the program needs to support only big endian
and little
endian systems (i.e. excluding systems where no D compiler
exists :)),
the following is less wordy and should be equally fast:
import std.bitmanip;
import std.system;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);
if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);
} else {
return value;
}
}
Ali
Will that work even when the alignment is to odd bytes?
Because that's
the case I was really worried about. The ubyte array is a
packed
mixture of types, some of which are isolated bytes.
No, it is not guaranteed to work unless the alignment is right.
It's just an unaligned load. If your target cpu architecture
can't do unaligned loads then you're either using something very
small or very old.
Re: ubytes to ulong problem
On 12/22/2013 02:22 AM, Ali Çehreli wrote: On 12/22/2013 01:04 AM, Charles Hixson wrote: Nice, but the block is longer than 8 bytes, so I should use a for (i = n; i n + 8; i++) rather than a foreach, and index off of i. Makes sense. That reminded me of the Phobos function that does exactly what you want. Have you considered std.bitmanip.read? Ali No, thanks. That's precisely what I was looking for. -- Charles Hixson
ubytes to ulong problem
I was planning to ask if there were a better way to do this, but instead
I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array, and an
index of 0, it returns a value of 8, even though all the values in the
array are 0.
The error has to be somewhere in the ret = statement, but I sure
don't see it.
/**Convert 8 consecutive bytes sliced from a ubyte[] into a ulong
* @paramblockThe array from which to slice.
* @paramnThe starting index within the block*/
ulongubytesToUlong(ubyte[] block, int n)
{ulongret;
assert (n = 0);
assert (n + 8 = block.length);
writefln (n = %s, n);
writefln (block[0] = %s, cast(ulong)block[0]);
writefln (block[1] = %s, cast(ulong)block[1]);
writefln (block[2] = %s, cast(ulong)block[2]);
writefln (block[3] = %s, cast(ulong)block[3]);
writefln (block[4] = %s, cast(ulong)block[4]);
writefln (block[5] = %s, cast(ulong)block[5]);
writefln (block[6] = %s, cast(ulong)block[6]);
writefln (block[7] = %s, cast(ulong)block[7]);
ret=cast(ulong)block[n] * 2^21
+cast(ulong)block[n+1] * 2^18
+cast(ulong)block[n+2] * 2^15
+cast(ulong)block[n+3] * 2^12
+cast(ulong)block[n+4] * 2^9
+cast(ulong)block[n+5] * 2^6
+cast(ulong)block[n+6] * 2^3
+cast(ulong)block[n+7] * 2^0;
writefln (ret = %s, ret);
returnret;
}
--
Charles Hixson
Re: ubytes to ulong problem
On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote:
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
but instead I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all
the values in the array are 0.
The error has to be somewhere in the ret = statement, but I
sure don't see it.
/**Convert 8 consecutive bytes sliced from a ubyte[] into
a ulong
* @paramblockThe array from which to slice.
* @paramnThe starting index within the block*/
ulongubytesToUlong(ubyte[] block, int n)
{ulongret;
assert (n = 0);
assert (n + 8 = block.length);
writefln (n = %s, n);
writefln (block[0] = %s, cast(ulong)block[0]);
writefln (block[1] = %s, cast(ulong)block[1]);
writefln (block[2] = %s, cast(ulong)block[2]);
writefln (block[3] = %s, cast(ulong)block[3]);
writefln (block[4] = %s, cast(ulong)block[4]);
writefln (block[5] = %s, cast(ulong)block[5]);
writefln (block[6] = %s, cast(ulong)block[6]);
writefln (block[7] = %s, cast(ulong)block[7]);
ret=cast(ulong)block[n] * 2^21
+cast(ulong)block[n+1] * 2^18
+cast(ulong)block[n+2] * 2^15
+cast(ulong)block[n+3] * 2^12
+cast(ulong)block[n+4] * 2^9
+cast(ulong)block[n+5] * 2^6
+cast(ulong)block[n+6] * 2^3
+cast(ulong)block[n+7] * 2^0;
writefln (ret = %s, ret);
returnret;
}
Use the exponentiation operator which is spelled: ^^
Re: ubytes to ulong problem
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
but instead I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all the
values in the array are 0.
The error has to be somewhere in the ret = statement, but I
sure don't see it.
/**Convert 8 consecutive bytes sliced from a ubyte[] into a
ulong
* @paramblockThe array from which to slice.
* @paramnThe starting index within the block*/
ulongubytesToUlong(ubyte[] block, int n)
{ulongret;
assert (n = 0);
assert (n + 8 = block.length);
writefln (n = %s, n);
writefln (block[0] = %s, cast(ulong)block[0]);
writefln (block[1] = %s, cast(ulong)block[1]);
writefln (block[2] = %s, cast(ulong)block[2]);
writefln (block[3] = %s, cast(ulong)block[3]);
writefln (block[4] = %s, cast(ulong)block[4]);
writefln (block[5] = %s, cast(ulong)block[5]);
writefln (block[6] = %s, cast(ulong)block[6]);
writefln (block[7] = %s, cast(ulong)block[7]);
ret=cast(ulong)block[n] * 2^21
+cast(ulong)block[n+1] * 2^18
+cast(ulong)block[n+2] * 2^15
+cast(ulong)block[n+3] * 2^12
+cast(ulong)block[n+4] * 2^9
+cast(ulong)block[n+5] * 2^6
+cast(ulong)block[n+6] * 2^3
+cast(ulong)block[n+7] * 2^0;
writefln (ret = %s, ret);
returnret;
}
As pointed out before, you're using '^' which is xor, instead of
^^. Ideally the compiler will optimise your version to be fast,
but you may find you get better performance by doing the bit
manipulations eplicitly:
/**Convert 8 consecutive bytes sliced from a ubyte[]
* to a ulong using bigendian byte ordering.
* @paramblockThe array from which to slice.
* @paramnThe starting index within the block, default 0*/
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
return ((cast(ulong)block[n ]) 56)
| ((cast(ulong)block[n+1]) 48)
| ((cast(ulong)block[n+2]) 40)
| ((cast(ulong)block[n+3]) 32)
| ((cast(ulong)block[n+4]) 24)
| ((cast(ulong)block[n+5]) 16)
| ((cast(ulong)block[n+6]) 8)
| (cast(ulong)block[n+7]);
}
unittest
{
ubyte[8] a = [0,0,0,0,0,0,0,0];
assert(a[].ubytesToUlong() == 0);
a[7] = 3;
a[6] = 1;
assert(a[].ubytesToUlong() == 259);
}
Re: ubytes to ulong problem
On 12/21/2013 02:31 PM, ponce wrote:
On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote:
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote:
I was planning to ask if there were a better way to do this, but
instead I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array, and an
index of 0, it returns a value of 8, even though all the values in
the array are 0.
The error has to be somewhere in the ret = statement, but I sure
don't see it.
/**Convert 8 consecutive bytes sliced from a ubyte[] into a ulong
* @paramblockThe array from which to slice.
* @paramnThe starting index within the block*/
ulongubytesToUlong(ubyte[] block, int n)
{ulongret;
assert (n = 0);
assert (n + 8 = block.length);
writefln (n = %s, n);
writefln (block[0] = %s, cast(ulong)block[0]);
writefln (block[1] = %s, cast(ulong)block[1]);
writefln (block[2] = %s, cast(ulong)block[2]);
writefln (block[3] = %s, cast(ulong)block[3]);
writefln (block[4] = %s, cast(ulong)block[4]);
writefln (block[5] = %s, cast(ulong)block[5]);
writefln (block[6] = %s, cast(ulong)block[6]);
writefln (block[7] = %s, cast(ulong)block[7]);
ret=cast(ulong)block[n] * 2^21
+cast(ulong)block[n+1] * 2^18
+cast(ulong)block[n+2] * 2^15
+cast(ulong)block[n+3] * 2^12
+cast(ulong)block[n+4] * 2^9
+cast(ulong)block[n+5] * 2^6
+cast(ulong)block[n+6] * 2^3
+cast(ulong)block[n+7] * 2^0;
writefln (ret = %s, ret);
returnret;
}
Use the exponentiation operator which is spelled: ^^
Thanks. I was *sure* it was something stupid on my part. It's 'good'
to know that I was right about *that*.
--
Charles Hixson
Re: ubytes to ulong problem
On 12/21/2013 03:13 PM, John Colvin wrote:
Ideally the compiler will optimise your version to be fast, but you may
find you get better performance by doing the bit manipulations eplicitly:
Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:
import std.bitmanip;
import std.system;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);
if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);
} else {
return value;
}
}
Ali
Re: ubytes to ulong problem
On Saturday, 21 December 2013 at 23:52:05 UTC, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:
Ideally the compiler will optimise your version to be fast,
but you may
find you get better performance by doing the bit
manipulations eplicitly:
Assuming that the program needs to support only big endian and
little endian systems (i.e. excluding systems where no D
compiler exists :)), the following is less wordy and should be
equally fast:
import std.bitmanip;
import std.system;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);
if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);
} else {
return value;
}
}
Ali
Nevermind equally fast, that will be much faster. -10 brain
points for me tonight...
Re: ubytes to ulong problem
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:
Ideally the compiler will optimise your version to be fast, but you may
find you get better performance by doing the bit manipulations
eplicitly:
Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:
import std.bitmanip;
import std.system;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);
if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);
} else {
return value;
}
}
Ali
Will that work even when the alignment is to odd bytes? Because that's
the case I was really worried about. The ubyte array is a packed
mixture of types, some of which are isolated bytes.
--
Charles Hixson
Re: ubytes to ulong problem
On 12/21/2013 05:44 PM, Charles Hixson wrote:
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:
Ideally the compiler will optimise your version to be fast, but you may
find you get better performance by doing the bit manipulations
eplicitly:
Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:
import std.bitmanip;
import std.system;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);
if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);
} else {
return value;
}
}
Ali
Will that work even when the alignment is to odd bytes? Because that's
the case I was really worried about. The ubyte array is a packed
mixture of types, some of which are isolated bytes.
No, it is not guaranteed to work unless the alignment is right.
How about this, then: :)
import std.array;
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n = 0);
assert (n + 8 = block.length);
}
body
{
ulong value = block.front;
block.popFront();
foreach (ub; block) {
value = 8;
value |= ub;
}
return value;
}
Ali
