Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/19/18 7:58 PM, ToddAndMargo via perl6-users wrote:
if    $StdErr  { $proc  = run( @RunArray, :err, :out, 
:enc ); }
    elsif $StdOut  { $proc  = run( @RunArray,   :out, 
:enc ); }  # STDERR goes to the terminal
    else   { $proc  = run( @RunArray, :err, :out, 
:enc ); }  # STDIN and STDERR are silent


    if  $Code{ $RtnCode   = $proc.status; }
    if  $StdErr  { $ReturnErr = $proc.err.slurp; }
    if  $StdOut  { $ReturnStr = $proc.out.slurp; }



changed to

sub AsciiToStr ( Buf $Ascii ) {
   # masks off bits above 0H7F
   # change 0H0D 0H0A to \n
   # change lone 0H0D to \n
   # return the corrected string

   my Str $String = "";
   my Str $CrLf   = chr(0x0d) ~ chr(0x0a);
   my Str $Cr = chr(0x0d);

   for $Ascii[0..*] -> $I { $String ~= chr( Int( $I ) +& 0x7F ); }
   $String ~~ s:global/ $CrLf /\n/;
   $String ~~ s:global/ $Cr /\n/;

   # print( "AsciiToStr <" ~ $String ~ "\n" );

   return $String;
}


   if  $StdErr  { $BufStdErr = $proc.err.slurp; $ReturnErr = 
AsciiToStr( $BufStdErr ); }


   if  $StdOut  { $BufStdOut = $proc.out.slurp; $ReturnStr = 
AsciiToStr( $BufStdOut ); }

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/19/18 7:58 PM, ToddAndMargo via perl6-users wrote:
sub RunNoShellAll( Str $RunString, Bool $StdOut, Bool $StdErr, Bool 
$Code, --> List ) {


Had to remove the --> List to get rid of the random core dumps

https://github.com/rakudo/rakudo/issues/2403

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/16/18 3:59 AM, ToddAndMargo via perl6-users wrote:


This is going through a rewrite.  So, don't help me until
I clean this up.  I hope to have some time Friday, but
maybe not.  I will get back.

-T


Rewrite worked!

sub RunNoShellAll( Str $RunString, Bool $StdOut, Bool $StdErr, Bool 
$Code, --> List ) {

   # --> List returns
   #StdOut as Str $ReturnStr
   #StdErr as Str $ReturnErr
   #Return Code ($?) as Int $RtnCode

   # $RunString   Comnad line to run without a shell
   # $StdOur  read and return the STDOUT   True/False
   # $StdErr  read and return the STDERR   True/False
   # $Coderead and return the return code  True/False

   # Note: If reading the STDERR, it is PRESUMED you also will be 
reading the STDIN.
   #   If only reading the return code, it is presumed that you 
want the STDERR

   #  and STDIN to be silent.

   # place each value into a cell in an array.  Keep quoted
   # values together
   # print "Run String = <$RunString>\n";


   my Buf $BufStdErr;
   my Str $ReturnStr = "";
   my Str $ReturnErr = "";
   my Int $RtnCode = 0;
   my $proc;

   my @RunArray  = $RunString ~~ m:global/ [ '"' <-[ " ]> * '"' | \S+ ] /;

   # shells remove the quote, so you have to here as well
   for @RunArray.kv -> $index, $value { @RunArray[$index] ~~ s:g/\"//; };
   # for @RunArray.kv -> $index, $value { say "\$RunArray[$index] = 
<$value>"; }; print "\n";


   if$StdErr  { $proc  = run( @RunArray, :err, :out, 
:enc ); }
   elsif $StdOut  { $proc  = run( @RunArray,   :out, 
:enc ); }  # STDERR goes to the terminal
   else   { $proc  = run( @RunArray, :err, :out, 
:enc ); }  # STDIN and STDERR are silent


   if  $Code{ $RtnCode   = $proc.status; }
   if  $StdErr  { $ReturnErr = $proc.err.slurp; }
   if  $StdOut  { $ReturnStr = $proc.out.slurp; }

   return( $ReturnStr, $ReturnErr, $RtnCode );
}

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/15/18 6:46 PM, Curt Tilmes wrote:



On Mon, Oct 15, 2018 at 9:44 PM Brandon Allbery > wrote:


Isn't the point that it's $ReturnStr that's throwing the immutable
error, not $RunString? That's what I see in the thread history


You're right.  My thinko.  Sorry.

On Mon, Oct 15, 2018 at 9:40 PM Curt Tilmes mailto:c...@tilmes.org>> wrote:



On Mon, Oct 15, 2018 at 9:34 PM ToddAndMargo via perl6-users
mailto:perl6-users@perl.org>> wrote:

On 10/15/18 9:04 AM, Larry Wall wrote:
 > This almost certainly means that $ReturnStr is a
read-only paramater to
 > a routine.  Add "is copy" to the declaration if you want
to modify it.

I am not seeing it.

sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr,
Int $Code ) {


You are declaring $RunString as a read-only parameter.

Try this:

  sub RunNoShellAll( Str $RunString is copy, Int $StdOut, Int
$StdErr, Int $Code ) {



Guys,

This is going through a rewrite.  So, don't help me until
I clean this up.  I hope to have some time Friday, but
maybe not.  I will get back.

-T

Re: Malformed UTF-8 ???

[Curt Tilmes]

On Mon, Oct 15, 2018 at 9:44 PM Brandon Allbery  wrote:

> Isn't the point that it's $ReturnStr that's throwing the immutable error,
> not $RunString? That's what I see in the thread history
>

You're right.  My thinko.  Sorry.



> On Mon, Oct 15, 2018 at 9:40 PM Curt Tilmes  wrote:
>
>>
>>
>> On Mon, Oct 15, 2018 at 9:34 PM ToddAndMargo via perl6-users <
>> perl6-users@perl.org> wrote:
>>
>>> On 10/15/18 9:04 AM, Larry Wall wrote:
>>> > This almost certainly means that $ReturnStr is a read-only paramater to
>>> > a routine.  Add "is copy" to the declaration if you want to modify it.
>>>
>>> I am not seeing it.
>>>
>>> sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code )
>>> {
>>>
>>
>> You are declaring $RunString as a read-only parameter.
>>
>> Try this:
>>
>>  sub RunNoShellAll( Str $RunString is copy, Int $StdOut, Int $StdErr, Int
>> $Code ) {
>>
>>

Re: Malformed UTF-8 ???

[Brandon Allbery]

Isn't the point that it's $ReturnStr that's throwing the immutable error,
not $RunString? That's what I see in the thread history.

On Mon, Oct 15, 2018 at 9:40 PM Curt Tilmes  wrote:

>
>
> On Mon, Oct 15, 2018 at 9:34 PM ToddAndMargo via perl6-users <
> perl6-users@perl.org> wrote:
>
>> On 10/15/18 9:04 AM, Larry Wall wrote:
>> > This almost certainly means that $ReturnStr is a read-only paramater to
>> > a routine.  Add "is copy" to the declaration if you want to modify it.
>>
>> I am not seeing it.
>>
>> sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code ) {
>>
>
> You are declaring $RunString as a read-only parameter.
>
> Try this:
>
>  sub RunNoShellAll( Str $RunString is copy, Int $StdOut, Int $StdErr, Int
> $Code ) {
>
>

-- 
brandon s allbery kf8nh
allber...@gmail.com

Re: Malformed UTF-8 ???

[Curt Tilmes]

On Mon, Oct 15, 2018 at 9:34 PM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> On 10/15/18 9:04 AM, Larry Wall wrote:
> > This almost certainly means that $ReturnStr is a read-only paramater to
> > a routine.  Add "is copy" to the declaration if you want to modify it.
>
> I am not seeing it.
>
> sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code ) {
>

You are declaring $RunString as a read-only parameter.

Try this:

 sub RunNoShellAll( Str $RunString is copy, Int $StdOut, Int $StdErr, Int
$Code ) {

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/15/18 9:04 AM, Larry Wall wrote:

On Sun, Oct 14, 2018 at 02:03:23AM -0700, ToddAndMargo via perl6-users wrote:
: On 10/13/18 3:02 AM, ToddAndMargo via perl6-users wrote:
: >Hi All,
: >
: >  if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
: >
: >gives me
: >
: >  Malformed UTF-8
: >
: >How do I clean up $$proc.out.slurp-rest ??
: >
: >Many thanks,
: >-T
:
: This does not work:
:
: if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' ); }
:
: Can not assign to an immutable value

This almost certainly means that $ReturnStr is a read-only paramater to
a routine.  Add "is copy" to the declaration if you want to modify it.

Larry



Hi Larry,

I am not seeing it.

:'(

-T


sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code ) {
...
   my Str $ReturnStr = "";
...
   if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }

Re: Malformed UTF-8 ???

[Larry Wall]

On Sun, Oct 14, 2018 at 02:03:23AM -0700, ToddAndMargo via perl6-users wrote:
: On 10/13/18 3:02 AM, ToddAndMargo via perl6-users wrote:
: >Hi All,
: >
: >  if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
: >
: >gives me
: >
: >  Malformed UTF-8
: >
: >How do I clean up $$proc.out.slurp-rest ??
: >
: >Many thanks,
: >-T
: 
: This does not work:
: 
: if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' ); }
: 
: Can not assign to an immutable value

This almost certainly means that $ReturnStr is a read-only paramater to
a routine.  Add "is copy" to the declaration if you want to modify it.

Larry

Re: Malformed UTF-8 ???

[Ralph Mellor]

It's redundant for this code.

`$foo` means data that's to be treated as either a single item or as a
single item container containing a single item.

`$$foo` means put the single item in $foo into a single item container
containing it.

Given that this appears on the right of `=` the single item is then taken
out of the container.

Which means the extra `$` is redundant.

--
raiph


On Sun, Oct 14, 2018 at 12:36 PM Tom Browder  wrote:

> On Sun, Oct 14, 2018 at 5:13 AM Ralph Mellor 
> wrote:
> >
> > Almost certainly your problem is elsewhere.
>
> What is the meaning of the double dollar sign ($$) in the problem code?
>
> -Tom
>

Re: Malformed UTF-8 ???

[Tom Browder]

On Sun, Oct 14, 2018 at 5:13 AM Ralph Mellor  wrote:
>
> Almost certainly your problem is elsewhere.

What is the meaning of the double dollar sign ($$) in the problem code?

-Tom

Re: Malformed UTF-8 ???

[Ralph Mellor]

I see no significant difference beyond the inside of your
proc and the outside of the code you've shown.

The extra `$` in `$$proc` is redundant.

(I don't understand why you have it.)

The `(...)` is the same as my `: ...`.

It can wait until you switch to `slurp`.

I wouldn't expect it to make any difference.

We can pick things up from there.

Catch ya later.

--
raiph


On Sun, Oct 14, 2018 at 11:30 AM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> On 10/14/18 3:08 AM, Ralph Mellor wrote:
> > This code works fine:
> >
> > spurt 'foo', 'bar';
> > my Str $ReturnStr = "";
> > $ReturnStr = 'foo'.IO.open.slurp-rest: enc => 'utf8-c8';
> > say $ReturnStr; # bar
>
> Try it with my layout:
>
> $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' );
>

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/14/18 3:08 AM, Ralph Mellor wrote:

This code works fine:

spurt 'foo', 'bar';
my Str $ReturnStr = "";
$ReturnStr = 'foo'.IO.open.slurp-rest: enc => 'utf8-c8';
say $ReturnStr; # bar


Try it with my layout:

   $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' );

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

>
>
> On Sun, Oct 14, 2018 at 11:08 AM Ralph Mellor  > wrote:
>
> Are you sure the error message you showed applies to the line you
> showed?
>
> This code works fine:
>
> spurt 'foo', 'bar';
> my Str $ReturnStr = "";
> $ReturnStr = 'foo'.IO.open.slurp-rest: enc => 'utf8-c8';
> say $ReturnStr; # bar
>
> --
> raiph
>
> On Sun, Oct 14, 2018 at 10:52 AM ToddAndMargo via perl6-users
> mailto:perl6-users@perl.org>> wrote:
>
>  >> On Sun, Oct 14, 2018 at 10:35 AM ToddAndMargo via perl6-users
>  >> mailto:perl6-users@perl.org>
> >> 
wrote:

>  >>
>  >> On 10/14/18 2:29 AM, Ralph Mellor wrote:
>  >>  > In P6 "assign" means use of `=`.
>  >>  >
>  >>  > And "assign to" means to copy into the thing on the
> left of the `=`.
>  >>  >
>  >>  > And to copy into something the thing has to be a
> mutable container.
>  >>  >
>  >>  > The error message is saying that $ReturnStr is bound
> to an
>  >>  > immutable value (eg a string) not a container.
>  >>  >
>  >>  > So that's presumably a problem with whatever code 
you've

>  >>  > written that has earlier bound $ReturnStr to an
> immutable value.
>  >>  >
>  >>  > --
>  >>  > raiph
>  >>
>  >>  my Str $ReturnStr = "";
>  >>
>  >> I think slurp-rest does not support
>  >>
>  >>enc => 'utf8-c8'
>  >>
>  >> :'(
>  >>
>  >> See my other letter on this thread
>  >>
>
> On 10/14/18 2:43 AM, Ralph Mellor wrote:
>  > I just tried slurp-rest with a specified encoding and it
> worked fine.
>  >
>  > Are you sure you don't bind $ReturnStr between the
>  > declaration you've shiown and the use you've shown?
>  >
>  > --
>  > raiph
>  >
>
>
> If I do, I am not seeing it.  Here is the specific code:
>
> sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int
> $Code ) {
>
>  # $RunString   Comnad line to run without a shell
>  # $StdOur  read and return the STDOUT   1=yes, 0=no
>  # $StdErr  read and return the STDERR   1=yes, 0=no
>  # $Coderead and return the return code  1=yes, 0=no
>
>  # Note: If reading the STDERR, it is PRESUMED you also 
will be

> reading the STDIN.
>  #   If only reading the return code, it is presumed
> that you
> want the STDERR
>  #  and STDIN to be silent.
>
>  # place each value into a cell in an array.  Keep quoted
>  # values together
>  # print "Run String = <$RunString>\n";
>
>  # returns
>  #StdOut as Str $ReturnStr
>  #StdErr as Str $ReturnErr
>  #Return Code ($?) as Int $RtnCode
>
>  my Buf $BufStdErr;
>  my Str $ReturnStr = "";
>  my Str $ReturnErr = "";
>  my Int $RtnCode = 0;
>  my $proc;
>
>  my @RunArray  = $RunString ~~ m:global/ [ '"' <-[ " ]> *
> '"' | \S+ ] /;
>
>  # shells remove the quote, so you have to here as well
>  for @RunArray.kv -> $index, $value { @RunArray[$index] ~~
> s:g/\"//; };
>  # for @RunArray.kv -> $index, $value { say
> "\$RunArray[$index] =
> <$value>"; }; print "\n";
>
>  if$StdErr  { $proc  = run( @RunArray, :err, :out 
); }

>  elsif $StdOut  { $proc  = run( @RunArray,   :out );
> }  #
> STDERR goes to the terminal
>  else   { $proc  = run( @RunArray, :err, :out );
> }  #
> STDIN and STDERR are silent
>
>  if  $Code{ $RtnCode   = $proc.status; }
>  # if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest( enc =>
> 'utf8-c8' ); }
>  # if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc =>
> 'utf8-c8' ); }
>  if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest; }
>  if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
>  # $ReturnStr = $BufStdErr.decode("utf8-c8") }
>
>
>  return( $ReturnStr, $ReturnErr, $RtnCode );
> }
>

On 10/14/18 3:12 AM, Ralph Mellor wrote:

Almost certainly your problem is elsewhere.

When you refer to a variable as a *value* then you get its *value*.

So:

return $ReturnStr

returns $ReturnStr's *value*, not the variable.

Your code returns such a value, which will be an immutable string.

So I think the error 

Re: Malformed UTF-8 ???

[Ralph Mellor]

Almost certainly your problem is elsewhere.

When you refer to a variable as a *value* then you get its *value*.

So:

return $ReturnStr

returns $ReturnStr's *value*, not the variable.

Your code returns such a value, which will be an immutable string.

So I think the error message you're getting is to do with code that
you haven't yet shown.

--
raiph


On Sun, Oct 14, 2018 at 11:08 AM Ralph Mellor 
wrote:

> Are you sure the error message you showed applies to the line you showed?
>
> This code works fine:
>
> spurt 'foo', 'bar';
> my Str $ReturnStr = "";
> $ReturnStr = 'foo'.IO.open.slurp-rest: enc => 'utf8-c8';
> say $ReturnStr; # bar
>
> --
> raiph
>
> On Sun, Oct 14, 2018 at 10:52 AM ToddAndMargo via perl6-users <
> perl6-users@perl.org> wrote:
>
>> >> On Sun, Oct 14, 2018 at 10:35 AM ToddAndMargo via perl6-users
>> >> mailto:perl6-users@perl.org>> wrote:
>> >>
>> >> On 10/14/18 2:29 AM, Ralph Mellor wrote:
>> >>  > In P6 "assign" means use of `=`.
>> >>  >
>> >>  > And "assign to" means to copy into the thing on the left of the
>> `=`.
>> >>  >
>> >>  > And to copy into something the thing has to be a mutable
>> container.
>> >>  >
>> >>  > The error message is saying that $ReturnStr is bound to an
>> >>  > immutable value (eg a string) not a container.
>> >>  >
>> >>  > So that's presumably a problem with whatever code you've
>> >>  > written that has earlier bound $ReturnStr to an immutable value.
>> >>  >
>> >>  > --
>> >>  > raiph
>> >>
>> >>  my Str $ReturnStr = "";
>> >>
>> >> I think slurp-rest does not support
>> >>
>> >>enc => 'utf8-c8'
>> >>
>> >> :'(
>> >>
>> >> See my other letter on this thread
>> >>
>>
>> On 10/14/18 2:43 AM, Ralph Mellor wrote:
>> > I just tried slurp-rest with a specified encoding and it worked fine.
>> >
>> > Are you sure you don't bind $ReturnStr between the
>> > declaration you've shiown and the use you've shown?
>> >
>> > --
>> > raiph
>> >
>>
>>
>> If I do, I am not seeing it.  Here is the specific code:
>>
>> sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code ) {
>>
>> # $RunString   Comnad line to run without a shell
>> # $StdOur  read and return the STDOUT   1=yes, 0=no
>> # $StdErr  read and return the STDERR   1=yes, 0=no
>> # $Coderead and return the return code  1=yes, 0=no
>>
>> # Note: If reading the STDERR, it is PRESUMED you also will be
>> reading the STDIN.
>> #   If only reading the return code, it is presumed that you
>> want the STDERR
>> #  and STDIN to be silent.
>>
>> # place each value into a cell in an array.  Keep quoted
>> # values together
>> # print "Run String = <$RunString>\n";
>>
>> # returns
>> #StdOut as Str $ReturnStr
>> #StdErr as Str $ReturnErr
>> #Return Code ($?) as Int $RtnCode
>>
>> my Buf $BufStdErr;
>> my Str $ReturnStr = "";
>> my Str $ReturnErr = "";
>> my Int $RtnCode = 0;
>> my $proc;
>>
>> my @RunArray  = $RunString ~~ m:global/ [ '"' <-[ " ]> * '"' | \S+ ]
>> /;
>>
>> # shells remove the quote, so you have to here as well
>> for @RunArray.kv -> $index, $value { @RunArray[$index] ~~ s:g/\"//; };
>> # for @RunArray.kv -> $index, $value { say "\$RunArray[$index] =
>> <$value>"; }; print "\n";
>>
>> if$StdErr  { $proc  = run( @RunArray, :err, :out ); }
>> elsif $StdOut  { $proc  = run( @RunArray,   :out ); }  #
>> STDERR goes to the terminal
>> else   { $proc  = run( @RunArray, :err, :out ); }  #
>> STDIN and STDERR are silent
>>
>> if  $Code{ $RtnCode   = $proc.status; }
>> # if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest( enc =>
>> 'utf8-c8' ); }
>> # if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc =>
>> 'utf8-c8' ); }
>> if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest; }
>> if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
>> # $ReturnStr = $BufStdErr.decode("utf8-c8") }
>>
>>
>> return( $ReturnStr, $ReturnErr, $RtnCode );
>> }
>>
>

Re: Malformed UTF-8 ???

[Ralph Mellor]

Are you sure the error message you showed applies to the line you showed?

This code works fine:

spurt 'foo', 'bar';
my Str $ReturnStr = "";
$ReturnStr = 'foo'.IO.open.slurp-rest: enc => 'utf8-c8';
say $ReturnStr; # bar

--
raiph

On Sun, Oct 14, 2018 at 10:52 AM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> >> On Sun, Oct 14, 2018 at 10:35 AM ToddAndMargo via perl6-users
> >> mailto:perl6-users@perl.org>> wrote:
> >>
> >> On 10/14/18 2:29 AM, Ralph Mellor wrote:
> >>  > In P6 "assign" means use of `=`.
> >>  >
> >>  > And "assign to" means to copy into the thing on the left of the
> `=`.
> >>  >
> >>  > And to copy into something the thing has to be a mutable
> container.
> >>  >
> >>  > The error message is saying that $ReturnStr is bound to an
> >>  > immutable value (eg a string) not a container.
> >>  >
> >>  > So that's presumably a problem with whatever code you've
> >>  > written that has earlier bound $ReturnStr to an immutable value.
> >>  >
> >>  > --
> >>  > raiph
> >>
> >>  my Str $ReturnStr = "";
> >>
> >> I think slurp-rest does not support
> >>
> >>enc => 'utf8-c8'
> >>
> >> :'(
> >>
> >> See my other letter on this thread
> >>
>
> On 10/14/18 2:43 AM, Ralph Mellor wrote:
> > I just tried slurp-rest with a specified encoding and it worked fine.
> >
> > Are you sure you don't bind $ReturnStr between the
> > declaration you've shiown and the use you've shown?
> >
> > --
> > raiph
> >
>
>
> If I do, I am not seeing it.  Here is the specific code:
>
> sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code ) {
>
> # $RunString   Comnad line to run without a shell
> # $StdOur  read and return the STDOUT   1=yes, 0=no
> # $StdErr  read and return the STDERR   1=yes, 0=no
> # $Coderead and return the return code  1=yes, 0=no
>
> # Note: If reading the STDERR, it is PRESUMED you also will be
> reading the STDIN.
> #   If only reading the return code, it is presumed that you
> want the STDERR
> #  and STDIN to be silent.
>
> # place each value into a cell in an array.  Keep quoted
> # values together
> # print "Run String = <$RunString>\n";
>
> # returns
> #StdOut as Str $ReturnStr
> #StdErr as Str $ReturnErr
> #Return Code ($?) as Int $RtnCode
>
> my Buf $BufStdErr;
> my Str $ReturnStr = "";
> my Str $ReturnErr = "";
> my Int $RtnCode = 0;
> my $proc;
>
> my @RunArray  = $RunString ~~ m:global/ [ '"' <-[ " ]> * '"' | \S+ ] /;
>
> # shells remove the quote, so you have to here as well
> for @RunArray.kv -> $index, $value { @RunArray[$index] ~~ s:g/\"//; };
> # for @RunArray.kv -> $index, $value { say "\$RunArray[$index] =
> <$value>"; }; print "\n";
>
> if$StdErr  { $proc  = run( @RunArray, :err, :out ); }
> elsif $StdOut  { $proc  = run( @RunArray,   :out ); }  #
> STDERR goes to the terminal
> else   { $proc  = run( @RunArray, :err, :out ); }  #
> STDIN and STDERR are silent
>
> if  $Code{ $RtnCode   = $proc.status; }
> # if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest( enc =>
> 'utf8-c8' ); }
> # if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc =>
> 'utf8-c8' ); }
> if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest; }
> if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
> # $ReturnStr = $BufStdErr.decode("utf8-c8") }
>
>
> return( $ReturnStr, $ReturnErr, $RtnCode );
> }
>

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On Sun, Oct 14, 2018 at 10:35 AM ToddAndMargo via perl6-users 
mailto:perl6-users@perl.org>> wrote:


On 10/14/18 2:29 AM, Ralph Mellor wrote:
 > In P6 "assign" means use of `=`.
 >
 > And "assign to" means to copy into the thing on the left of the `=`.
 >
 > And to copy into something the thing has to be a mutable container.
 >
 > The error message is saying that $ReturnStr is bound to an
 > immutable value (eg a string) not a container.
 >
 > So that's presumably a problem with whatever code you've
 > written that has earlier bound $ReturnStr to an immutable value.
 >
 > --
 > raiph

 my Str $ReturnStr = "";

I think slurp-rest does not support

   enc => 'utf8-c8'

:'(

See my other letter on this thread



On 10/14/18 2:43 AM, Ralph Mellor wrote:

I just tried slurp-rest with a specified encoding and it worked fine.

Are you sure you don't bind $ReturnStr between the
declaration you've shiown and the use you've shown?

--
raiph




If I do, I am not seeing it.  Here is the specific code:

sub RunNoShellAll( Str $RunString, Int $StdOut, Int $StdErr, Int $Code ) {

   # $RunString   Comnad line to run without a shell
   # $StdOur  read and return the STDOUT   1=yes, 0=no
   # $StdErr  read and return the STDERR   1=yes, 0=no
   # $Coderead and return the return code  1=yes, 0=no

   # Note: If reading the STDERR, it is PRESUMED you also will be 
reading the STDIN.
   #   If only reading the return code, it is presumed that you 
want the STDERR

   #  and STDIN to be silent.

   # place each value into a cell in an array.  Keep quoted
   # values together
   # print "Run String = <$RunString>\n";

   # returns
   #StdOut as Str $ReturnStr
   #StdErr as Str $ReturnErr
   #Return Code ($?) as Int $RtnCode

   my Buf $BufStdErr;
   my Str $ReturnStr = "";
   my Str $ReturnErr = "";
   my Int $RtnCode = 0;
   my $proc;

   my @RunArray  = $RunString ~~ m:global/ [ '"' <-[ " ]> * '"' | \S+ ] /;

   # shells remove the quote, so you have to here as well
   for @RunArray.kv -> $index, $value { @RunArray[$index] ~~ s:g/\"//; };
   # for @RunArray.kv -> $index, $value { say "\$RunArray[$index] = 
<$value>"; }; print "\n";


   if$StdErr  { $proc  = run( @RunArray, :err, :out ); }
   elsif $StdOut  { $proc  = run( @RunArray,   :out ); }  # 
STDERR goes to the terminal
   else   { $proc  = run( @RunArray, :err, :out ); }  # 
STDIN and STDERR are silent


   if  $Code{ $RtnCode   = $proc.status; }
   # if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest( enc => 
'utf8-c8' ); }
   # if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 
'utf8-c8' ); }

   if  $StdErr  { $ReturnErr = $$proc.err.slurp-rest; }
   if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
   # $ReturnStr = $BufStdErr.decode("utf8-c8") }


   return( $ReturnStr, $ReturnErr, $RtnCode );
}

Re: Malformed UTF-8 ???

[Ralph Mellor]

I just tried slurp-rest with a specified encoding and it worked fine.

Are you sure you don't bind $ReturnStr between the
declaration you've shiown and the use you've shown?

--
raiph

On Sun, Oct 14, 2018 at 10:35 AM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> On 10/14/18 2:29 AM, Ralph Mellor wrote:
> > In P6 "assign" means use of `=`.
> >
> > And "assign to" means to copy into the thing on the left of the `=`.
> >
> > And to copy into something the thing has to be a mutable container.
> >
> > The error message is saying that $ReturnStr is bound to an
> > immutable value (eg a string) not a container.
> >
> > So that's presumably a problem with whatever code you've
> > written that has earlier bound $ReturnStr to an immutable value.
> >
> > --
> > raiph
>
> my Str $ReturnStr = "";
>
> I think slurp-rest does not support
>
>   enc => 'utf8-c8'
>
> :'(
>
> See my other letter on this thread
>

Re: Malformed UTF-8 ???

[Ralph Mellor]

Couple clarifications about my prior message.

> And to copy into something the thing has to be a mutable container.

"mutable" and "container" are basically redundant with each other.
Mutable means changeable. Container means data that has only one
purpose, namely to be changeable.

> that's presumably a problem with whatever code you've
written that has earlier bound $ReturnStr to an immutable value.

Or you need to use a fresh variable.

--
raiph

On Sun, Oct 14, 2018 at 10:29 AM Ralph Mellor 
wrote:

> In P6 "assign" means use of `=`.
>
> And "assign to" means to copy into the thing on the left of the `=`.
>
> And to copy into something the thing has to be a mutable container.
>
> The error message is saying that $ReturnStr is bound to an
> immutable value (eg a string) not a container.
>
> So that's presumably a problem with whatever code you've
> written that has earlier bound $ReturnStr to an immutable value.
>
> --
> raiph
>
>
> On Sun, Oct 14, 2018 at 10:19 AM ToddAndMargo via perl6-users <
> perl6-users@perl.org> wrote:
>
>> On 10/13/18 3:02 AM, ToddAndMargo via perl6-users wrote:
>> > Hi All,
>> >
>> >   if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
>> >
>> > gives me
>> >
>> >   Malformed UTF-8
>> >
>> > How do I clean up $$proc.out.slurp-rest ??
>> >
>> > Many thanks,
>> > -T
>>
>> This does not work:
>>
>> if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' ); }
>>
>> Can not assign to an immutable value
>>
>

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/14/18 2:21 AM, Ralph Mellor wrote:

OK. That makes sense.

So the program that you're running for the $proc is producing
malformed UTF8. Why don't you fix that rather than clean up
afterwards?

--
raiph


Hi Raiph,

I am reading the contents from a web page through my "curl"
interface module (CurlUtils.pm6).  The bad characters are
coming from a particular web page. I have no control over it.

:'(

But with Perl, there are 1001 ways to do everything and I
noticed on

   https://docs.perl6.org/routine/slurp-rest

   DEPRECATION NOTICE: this method will be deprecated in 6.d
   language. Do not use it for new code. Use .slurp method
   method instead.

And I also noticed on the examples of

https://docs.perl6.org/routine/slurp  and
https://docs.perl6.org/language/unicode#index-entry-UTF8-C8

That is can use an "enc" (encode?) command:

   say slurp($test-file, enc => 'utf8-c8');

And since

if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' 
); }


Can not assign to an immutable value

When I get some time to code again, I will switch my
RunNoShell.pm6 module, which CurlUtils.pm6 uses extensively,
to use "slurp" instead of "slurp-rest".  Plus I use "slurp"
extensively everywhere else, so ...

-T

Re: Malformed UTF-8 ???

[Ralph Mellor]

In P6 "assign" means use of `=`.

And "assign to" means to copy into the thing on the left of the `=`.

And to copy into something the thing has to be a mutable container.

The error message is saying that $ReturnStr is bound to an
immutable value (eg a string) not a container.

So that's presumably a problem with whatever code you've
written that has earlier bound $ReturnStr to an immutable value.

--
raiph


On Sun, Oct 14, 2018 at 10:19 AM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> On 10/13/18 3:02 AM, ToddAndMargo via perl6-users wrote:
> > Hi All,
> >
> >   if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
> >
> > gives me
> >
> >   Malformed UTF-8
> >
> > How do I clean up $$proc.out.slurp-rest ??
> >
> > Many thanks,
> > -T
>
> This does not work:
>
> if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' ); }
>
> Can not assign to an immutable value
>

Re: Malformed UTF-8 ???

[Ralph Mellor]

OK. That makes sense.

So the program that you're running for the $proc is producing
malformed UTF8. Why don't you fix that rather than clean up
afterwards?

--
raiph


On Sun, Oct 14, 2018 at 3:44 AM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> >>
> >>
> >> On Sat, Oct 13, 2018 at 11:18 AM ToddAndMargo via perl6-users
> >> mailto:perl6-users@perl.org>> wrote:
> >>
> >> Hi All,
> >>
> >>if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
> >>
> >> gives me
> >>
> >>Malformed UTF-8
> >>
> >> How do I clean up $$proc.out.slurp-rest ??
> >>
> >> Many thanks,
> >> -T
> >>
>
> On 10/13/18 7:19 AM, Ralph Mellor wrote:
> > Why are you asking about cleaning something up?
> >
> > Is it because you thought `Malformed UTF-8` was an error message?
> >
> > If so, great, because it IS an error message.
> >
> > If not, you need to know that Malformed UTF-8 is an error
> > message and we need to think about how to make that more clear.
> >
> > So that's the first thing to sort out.
> >
> > Did you realize that Malformed UTF-8 was an error message?
> >
> > --
> > raiph
>
>
> Hi Raiph,
>
> My goal was to get rid of the error.  By clean up,
> I wanted to clean up the data so it would not throw
> the error.  Sorry for the misunderstanding
>
> -T
>

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

On 10/13/18 3:02 AM, ToddAndMargo via perl6-users wrote:

Hi All,

  if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }

gives me

  Malformed UTF-8

How do I clean up $$proc.out.slurp-rest ??

Many thanks,
-T


This does not work:

if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest( enc => 'utf8-c8' ); }

Can not assign to an immutable value

Re: Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

>>
>> Hi All,
>>
>>if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
>>
>> gives me
>>
>>Malformed UTF-8
>>
>> How do I clean up $$proc.out.slurp-rest ??
>>
>> Many thanks,
>> -T

On 10/13/18 8:18 AM, Brad Gilbert wrote:

Change the encoding to `utf8-c8` to let the invalid unicode through.

or use binary methods.

or make sure it isn't in some other encoding.
On Sat, Oct 13, 2018 at 5:18 AM ToddAndMargo via perl6-users
 wrote:


How do I change

  if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }

to utf8-c8?

Re: Malformed UTF-8 ???

[Brad Gilbert]

Change the encoding to `utf8-c8` to let the invalid unicode through.

or use binary methods.

or make sure it isn't in some other encoding.
On Sat, Oct 13, 2018 at 5:18 AM ToddAndMargo via perl6-users
 wrote:
>
> Hi All,
>
>   if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
>
> gives me
>
>   Malformed UTF-8
>
> How do I clean up $$proc.out.slurp-rest ??
>
> Many thanks,
> -T

Re: Malformed UTF-8 ???

[Ralph Mellor]

Why are you asking about cleaning something up?

Is it because you thought `Malformed UTF-8` was an error message?

If so, great, because it IS an error message.

If not, you need to know that Malformed UTF-8 is an error
message and we need to think about how to make that more clear.

So that's the first thing to sort out.

Did you realize that Malformed UTF-8 was an error message?

--
raiph


On Sat, Oct 13, 2018 at 11:18 AM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

> Hi All,
>
>   if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }
>
> gives me
>
>   Malformed UTF-8
>
> How do I clean up $$proc.out.slurp-rest ??
>
> Many thanks,
> -T
>

Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

Hi All,

 if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }

gives me

 Malformed UTF-8

How do I clean up $$proc.out.slurp-rest ??

Many thanks,
-T

Malformed UTF-8 ???

[ToddAndMargo via perl6-users]

Hi All,

 if  $StdOut  { $ReturnStr = $$proc.out.slurp-rest; }

gives me

 Malformed UTF-8

How do I clean up $$proc.out.slurp-rest ??

Many thanks,
-T

Re: Malformed UTF-8?

[ToddAndMargo]

On Sat, Jun 16, 2018 at 1:09 AM ToddAndMargo <mailto:toddandma...@zoho.com>> wrote:


rakudo-pkg-Fedora28-2018.05-01.x86_64.rpm
$ perl6 -v
This is Rakudo version 2018.05 built on MoarVM version 2018.05
implementing Perl 6.c.


What did I do wrong, this time?

Malformed UTF-8
in sub RotateZipFile at /home/linuxutil/CimCheck.pl6 line 290

290:   @ReverseLogs = @Logs.sort: {my ($month, $day, $year, $hour,
$minute, $second) = .comb(/\d+/);($year // 0, $month // 0, $day // 0,
$hour // 0, $minute // 0,$second // 0, $_);}



On 06/16/2018 08:19 AM, Brandon Allbery wrote:
Something's wrong with the data file you are reading. Perl 6 is 
expecting UTF-8 encoding and getting something else (usually an ISO-8859 
encoding).




Indeed!

I mistook the error as a compiler error.

It is all abetter now.

Thank you!

Re: Malformed UTF-8?

[ToddAndMargo]

On 06/16/2018 08:37 AM, The Sidhekin wrote:
On Sat, Jun 16, 2018 at 5:19 PM, Brandon Allbery <mailto:allber...@gmail.com>> wrote:


Something's wrong with the data file you are reading. Perl 6 is
expecting UTF-8 encoding and getting something else (usually an
ISO-8859 encoding).


   Are you sure it's a data file?  I thought I recognized that code, and 
it was handling directory entries ... ah, but it used to read ls output 
from STDIN ... right ...


   Either way, the data (file or directory) has been read already, 
unless @Logs is seriously lazy.  And it has at least one entry with 
malformed UTF-8.


   So, I guess a look at how @Logs is populated is in order.


Eirik


Hi Eirik and Brandon,

I had mistaken the error as a "compiler" error, not
a run time error.

I will put some debugging and traps in to make sure I am
getting the proper data before running the sort.

Thank you!

-T

Re: Malformed UTF-8?

[The Sidhekin]

On Sat, Jun 16, 2018 at 5:19 PM, Brandon Allbery 
wrote:

> Something's wrong with the data file you are reading. Perl 6 is expecting
> UTF-8 encoding and getting something else (usually an ISO-8859 encoding).
>

  Are you sure it's a data file?  I thought I recognized that code, and it
was handling directory entries ... ah, but it used to read ls output from
STDIN ... right ...

  Either way, the data (file or directory) has been read already, unless
@Logs is seriously lazy.  And it has at least one entry with malformed
UTF-8.

  So, I guess a look at how @Logs is populated is in order.


Eirik

Re: Malformed UTF-8?

[Brandon Allbery]

Something's wrong with the data file you are reading. Perl 6 is expecting
UTF-8 encoding and getting something else (usually an ISO-8859 encoding).

On Sat, Jun 16, 2018 at 1:09 AM ToddAndMargo  wrote:

> rakudo-pkg-Fedora28-2018.05-01.x86_64.rpm
> $ perl6 -v
> This is Rakudo version 2018.05 built on MoarVM version 2018.05
> implementing Perl 6.c.
>
>
> What did I do wrong, this time?
>
> Malformed UTF-8
>in sub RotateZipFile at /home/linuxutil/CimCheck.pl6 line 290
>
> 290:   @ReverseLogs = @Logs.sort: {my ($month, $day, $year, $hour,
> $minute, $second) = .comb(/\d+/);($year // 0, $month // 0, $day // 0,
> $hour // 0, $minute // 0,$second // 0, $_);}
>


-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net

Malformed UTF-8?

[ToddAndMargo]

rakudo-pkg-Fedora28-2018.05-01.x86_64.rpm
$ perl6 -v
This is Rakudo version 2018.05 built on MoarVM version 2018.05
implementing Perl 6.c.


What did I do wrong, this time?

Malformed UTF-8
  in sub RotateZipFile at /home/linuxutil/CimCheck.pl6 line 290

290:   @ReverseLogs = @Logs.sort: {my ($month, $day, $year, $hour, 
$minute, $second) = .comb(/\d+/);($year // 0, $month // 0, $day // 0, 
$hour // 0, $minute // 0,$second // 0, $_);}