Re: Fastest way to convert from a Buf to a Str?
On 2/3/19 5:26 PM, Darren Duncan wrote:
On 2019-02-02 7:22 PM, ToddAndMargo via perl6-users wrote:
I need to read a file into a buffer (NO CONVERSIONS!)
and then convert it to a string (again with no
conversions).
I think you're making an impossible request.
Don't forget that I think everywhere on this list is
a bloody genius.
If preserving exact bytes
is important, then you want to keep your data in a type that represents
a sequence of bytes, such as Blob of Buf. A Str represents a sequence
of characters, which are NOT bytes, so if you're wanting to have a Str
that is saying you don't care about the bytes. Given what you keep
saying, I'd say skip the Str and just use Buf or Blob etc full stop. --
Darren Duncan
Hi Darren,
for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
Does the trick, but it takes up to 15 seconds. Way
too slow.
I have another post looking to see if any of the other
decodes will work. So maybe...
My big issue is that the data I am looking through uses four
nuls in a row as a delimiter. If these get dropped, I
won't be able to find anything.
Your idea about just skipping Str is along the line I have
also been thinking. Brad has been helping me with "index" for
a Buf. I haven't had a shot at trying his corrections to my
code yet.
-T
Re: Fastest way to convert from a Buf to a Str?
On 2019-02-02 7:22 PM, ToddAndMargo via perl6-users wrote: I need to read a file into a buffer (NO CONVERSIONS!) and then convert it to a string (again with no conversions). I think you're making an impossible request. If preserving exact bytes is important, then you want to keep your data in a type that represents a sequence of bytes, such as Blob of Buf. A Str represents a sequence of characters, which are NOT bytes, so if you're wanting to have a Str that is saying you don't care about the bytes. Given what you keep saying, I'd say skip the Str and just use Buf or Blob etc full stop. -- Darren Duncan
Re: Fastest way to convert from a Buf to a Str?
On 2/3/19 1:55 AM, David Warring wrote:
Are all characters in the range 0-255, ie latin-1 characters?
You could then try: my $str = $buf.decode("latin-1");
There's one potential issue if your data could contain DOS end of lines
("\r\n"), which will get translated to a single logical "\n" in the
decoded string.
- David
Hi David,
It has to be an exact match. That includes all carriage returns,
line feeds, page feeds, eofs, tabs, etc.. But thank you anyway.
:-)
-T
Re: Fastest way to convert from a Buf to a Str?
Are all characters in the range 0-255, ie latin-1 characters?
You could then try: my $str = $buf.decode("latin-1");
There's one potential issue if your data could contain DOS end of lines
("\r\n"), which will get translated to a single logical "\n" in the decoded
string.
- David
On Sun, Feb 3, 2019 at 7:16 PM Brad Gilbert wrote:
> This:
>
> for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
>
> is better written as
>
> my $StrFile = $BinaryFile.map(*.chr).reduce(* ~ *);
>
> It is also exactly equivalent to just e
>
> # if $BinaryFile is a Buf
> my $StrFile = $BinaryFile.decode('latin1');
>
> # if it isn't
> my $StrFile = Buf.new($BinaryFile).decode('latin1');
>
> If you don't otherwise need $BinaryFile
>
> my $fh = open 'test', :enc('latin1');
> my $StrFile = $fh.slurp;
>
> or
>
> my $StrFile = 'test'.IO.slurp(:enc('latin1'));
>
> ---
>
> Buf and Str used to be treated more alike, and it was very confusing.
>
> There should be more methods on Buf that work like the methods on Str,
> but that is about it.
>
> Having a string act like a buffer in Modula 2 probably works fine
> because it barely supports Unicode at all.
>
> Here is an example of why it can't work like that in Perl6:
>
> my $a = 'a';
> my $b = "\c[COMBINING ACUTE ACCENT]";
>
> my $c = $a ~ $b;
> my $d = $a.encode ~ $b.encode;
> my $e = Buf.new($a.encode) ~ Buf.new($b.encode);
>
> say $a.encode; # utf8:0x<61>
> say $b.encode; # utf8:0x
>
> say $c.encode; # utf8:0x
>
> say $d; # utf8:0x<61 CC 81>
> say $e; # Buf:0x<61 CC 81>
>
> Notice that `$c.encode` and `$d` are different even though they are
> made from the same parts.
> `$d` and `$e` are similar because they are dealing with lists of
> numbers not strings.
>
> On Sat, Feb 2, 2019 at 9:23 PM ToddAndMargo via perl6-users
> wrote:
> >
> > Hi All,
> >
> > I need to read a file into a buffer (NO CONVERSIONS!)
> > and then convert it to a string (again with no
> > conversions).
> >
> > I have been doing this:
> >
> > for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
> >
> > But it takes a bit of time. What is the fastest way to do this?
> >
> > I guess there is not a way to create/declare a variable that is
> > both Buf and Str at the same time? That would mean I did not
> > have to convert anything. I use to get away with this under
> > Module 2 all the time.
> >
> > $ p6 'my $B = Buf.new(0x66, 0x66, 0x77); $B.Str ~= "z";'
> > Cannot use a Buf as a string, but you called the Str method on it
> >in block at -e line 1
> >
> > $ p6 'my $B = Buf.new(0x66, 0x66, 0x77); Str($B) ~= "z";'
> > Cannot use a Buf as a string, but you called the Str method on it
> >in block at -e line 1
> >
> >
> > Many thanks,
> > -T
> >
> > --
> >
> > A computer without Microsoft is like
> > a chocolate cake without the mustard
> >
>
Re: Fastest way to convert from a Buf to a Str?
>
> On Sat, Feb 2, 2019 at 9:23 PM ToddAndMargo via perl6-users
> wrote:
>>
>> Hi All,
>>
>> I need to read a file into a buffer (NO CONVERSIONS!)
>> and then convert it to a string (again with no
>> conversions).
>>
>> I have been doing this:
>>
>> for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
>>
>> But it takes a bit of time. What is the fastest way to do this?
>>
>> I guess there is not a way to create/declare a variable that is
>> both Buf and Str at the same time? That would mean I did not
>> have to convert anything. I use to get away with this under
>> Module 2 all the time.
>>
>> $ p6 'my $B = Buf.new(0x66, 0x66, 0x77); $B.Str ~= "z";'
>> Cannot use a Buf as a string, but you called the Str method on it
>> in block at -e line 1
>>
>> $ p6 'my $B = Buf.new(0x66, 0x66, 0x77); Str($B) ~= "z";'
>> Cannot use a Buf as a string, but you called the Str method on it
>> in block at -e line 1
>>
>>
>> Many thanks,
>> -T
On 2/2/19 10:15 PM, Brad Gilbert wrote:
This:
for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
is better written as
my $StrFile = $BinaryFile.map(*.chr).reduce(* ~ *);
It is also exactly equivalent to just e
# if $BinaryFile is a Buf
my $StrFile = $BinaryFile.decode('latin1');
# if it isn't
my $StrFile = Buf.new($BinaryFile).decode('latin1');
If you don't otherwise need $BinaryFile
my $fh = open 'test', :enc('latin1');
my $StrFile = $fh.slurp;
or
my $StrFile = 'test'.IO.slurp(:enc('latin1'));
---
Buf and Str used to be treated more alike, and it was very confusing.
There should be more methods on Buf that work like the methods on Str,
but that is about it.
Having a string act like a buffer in Modula 2 probably works fine
because it barely supports Unicode at all.
Here is an example of why it can't work like that in Perl6:
my $a = 'a';
my $b = "\c[COMBINING ACUTE ACCENT]";
my $c = $a ~ $b;
my $d = $a.encode ~ $b.encode;
my $e = Buf.new($a.encode) ~ Buf.new($b.encode);
say $a.encode; # utf8:0x<61>
say $b.encode; # utf8:0x
say $c.encode; # utf8:0x
say $d; # utf8:0x<61 CC 81>
say $e; # Buf:0x<61 CC 81>
Notice that `$c.encode` and `$d` are different even though they are
made from the same parts.
`$d` and `$e` are similar because they are dealing with lists of
numbers not strings.
Hi Brad,
Thank you!
I want ZERO decoding. I want exactly the same bytes in the
string as are in the Buffer. And it has to be done FAST.
Are you saying this the fastest way?
my $StrFile = $BinaryFile.map(*.chr).reduce(* ~ *);
Please keep in mind. NO DECODING!
-T
Re: Fastest way to convert from a Buf to a Str?
This:
for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
is better written as
my $StrFile = $BinaryFile.map(*.chr).reduce(* ~ *);
It is also exactly equivalent to just e
# if $BinaryFile is a Buf
my $StrFile = $BinaryFile.decode('latin1');
# if it isn't
my $StrFile = Buf.new($BinaryFile).decode('latin1');
If you don't otherwise need $BinaryFile
my $fh = open 'test', :enc('latin1');
my $StrFile = $fh.slurp;
or
my $StrFile = 'test'.IO.slurp(:enc('latin1'));
---
Buf and Str used to be treated more alike, and it was very confusing.
There should be more methods on Buf that work like the methods on Str,
but that is about it.
Having a string act like a buffer in Modula 2 probably works fine
because it barely supports Unicode at all.
Here is an example of why it can't work like that in Perl6:
my $a = 'a';
my $b = "\c[COMBINING ACUTE ACCENT]";
my $c = $a ~ $b;
my $d = $a.encode ~ $b.encode;
my $e = Buf.new($a.encode) ~ Buf.new($b.encode);
say $a.encode; # utf8:0x<61>
say $b.encode; # utf8:0x
say $c.encode; # utf8:0x
say $d; # utf8:0x<61 CC 81>
say $e; # Buf:0x<61 CC 81>
Notice that `$c.encode` and `$d` are different even though they are
made from the same parts.
`$d` and `$e` are similar because they are dealing with lists of
numbers not strings.
On Sat, Feb 2, 2019 at 9:23 PM ToddAndMargo via perl6-users
wrote:
>
> Hi All,
>
> I need to read a file into a buffer (NO CONVERSIONS!)
> and then convert it to a string (again with no
> conversions).
>
> I have been doing this:
>
> for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
>
> But it takes a bit of time. What is the fastest way to do this?
>
> I guess there is not a way to create/declare a variable that is
> both Buf and Str at the same time? That would mean I did not
> have to convert anything. I use to get away with this under
> Module 2 all the time.
>
> $ p6 'my $B = Buf.new(0x66, 0x66, 0x77); $B.Str ~= "z";'
> Cannot use a Buf as a string, but you called the Str method on it
>in block at -e line 1
>
> $ p6 'my $B = Buf.new(0x66, 0x66, 0x77); Str($B) ~= "z";'
> Cannot use a Buf as a string, but you called the Str method on it
>in block at -e line 1
>
>
> Many thanks,
> -T
>
> --
>
> A computer without Microsoft is like
> a chocolate cake without the mustard
>
Fastest way to convert from a Buf to a Str?
Hi All,
I need to read a file into a buffer (NO CONVERSIONS!)
and then convert it to a string (again with no
conversions).
I have been doing this:
for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
But it takes a bit of time. What is the fastest way to do this?
I guess there is not a way to create/declare a variable that is
both Buf and Str at the same time? That would mean I did not
have to convert anything. I use to get away with this under
Module 2 all the time.
$ p6 'my $B = Buf.new(0x66, 0x66, 0x77); $B.Str ~= "z";'
Cannot use a Buf as a string, but you called the Str method on it
in block at -e line 1
$ p6 'my $B = Buf.new(0x66, 0x66, 0x77); Str($B) ~= "z";'
Cannot use a Buf as a string, but you called the Str method on it
in block at -e line 1
Many thanks,
-T
--
A computer without Microsoft is like
a chocolate cake without the mustard
Fastest way to convert from a Buf to a Str?
Hi All,
I need to read a file into a buffer (NO CONVERSIONS!)
and then convert it to a string (again with no
conversions).
I have been doing this:
for ( @$BinaryFile ) -> $Char { $StrFile ~= chr($Char); }
But it takes a bit of time. What is the fastest way to do this?
I guess there is not a way to create/declare a variable that is
both Buf and Str at the same time? That would mean I did not
have to convert anything. I use to get away with this under
Module 2 all the time.
$ p6 'my $B = Buf.new(0x66, 0x66, 0x77); $B.Str ~= "z";'
Cannot use a Buf as a string, but you called the Str method on it
in block at -e line 1
$ p6 'my $B = Buf.new(0x66, 0x66, 0x77); Str($B) ~= "z";'
Cannot use a Buf as a string, but you called the Str method on it
in block at -e line 1
Many thanks,
-T
--
A computer without Microsoft is like
a chocolate cake without the mustard
