subject:"Re\: \[Mojolicious\] Re\: Mojo\:\:Log and UTF\-8"

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2020-10-31 Thread Dan Book

Any layer on standard handles will cause double encoding when a module like
Mojo::Log encodes to bytes before printing. These layers are global
unfortunately.

-Dan

On Sat, Oct 31, 2020 at 1:54 PM kon...@gmail.com  wrote:

> I found a problem. Some module did:
>
> binmode STDOUT, ':encoding(UTF-8)';
> binmode STDERR, ':encoding(UTF-8)';
>
> and, probably, when STDERR was redirected to STDOUT by Mojo::Log then
> double encoding occur.
>
> Is it worth to check that STDERR and STDOUT have not same layers before
> redirecting?
>
> On Saturday, October 31, 2020 at 6:16:55 PM UTC+2 Sebastian Riedel wrote:
>
>> Ok, data is encoded from UTF-8 into bytes when I log a message,
>>> but when this data is printed to STDERR
>>> How make it decoded from bytes to UTF-8 back?
>>> my terminal supports UTF-8 and I want see nice messages instead of
>>> Ð¥Ð°Ð±Ð
>>>
>>
>> UTF-8 *is* bytes. If you are encoding already encoded UTF-8 then you are
>> double encoding.
>>
>> --
>> sebastian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Mojolicious" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mojolicious+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mojolicious/f276d64e-f215-47d8-af7b-893d9c200ef4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/mojolicious/CABMkAVX4mqdx6m0HPR8fo5owa4B6rixb9i4bsocimop5755GEQ%40mail.gmail.com.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2020-10-31 Thread kon...@gmail.com

I found a problem. Some module did:

binmode STDOUT, ':encoding(UTF-8)';
binmode STDERR, ':encoding(UTF-8)';

and, probably, when STDERR was redirected to STDOUT by Mojo::Log then 
double encoding occur.

Is it worth to check that STDERR and STDOUT have not same layers before 
redirecting?

On Saturday, October 31, 2020 at 6:16:55 PM UTC+2 Sebastian Riedel wrote:

> Ok, data is encoded from UTF-8 into bytes when I log a message, 
>> but when this data is printed to STDERR
>> How make it decoded from bytes to UTF-8 back? 
>> my terminal supports UTF-8 and I want see nice messages instead of Ð¥Ð°Ð±Ð
>>
>
> UTF-8 *is* bytes. If you are encoding already encoded UTF-8 then you are 
> double encoding.
>
> --
> sebastian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/mojolicious/f276d64e-f215-47d8-af7b-893d9c200ef4n%40googlegroups.com.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2020-10-31 Thread kon...@gmail.com

I do nothing, except: 
$app->log->info('тест лога'); 
and get at console:
[Sat Oct  6 15:22:43 2018] [info] Ñ�ÐµÑ�Ñ� Ð»Ð¾Ð³Ð° 

Should I configure console or add some options to app?

On Saturday, October 31, 2020 at 6:16:55 PM UTC+2 Sebastian Riedel wrote:

> Ok, data is encoded from UTF-8 into bytes when I log a message, 
>> but when this data is printed to STDERR
>> How make it decoded from bytes to UTF-8 back? 
>> my terminal supports UTF-8 and I want see nice messages instead of Ð¥Ð°Ð±Ð
>>
>
> UTF-8 *is* bytes. If you are encoding already encoded UTF-8 then you are 
> double encoding.
>
> --
> sebastian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/mojolicious/de69d3d9-5823-4b4d-af0f-c20a7c656397n%40googlegroups.com.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2020-10-31 Thread Sebastian Riedel


>
> Ok, data is encoded from UTF-8 into bytes when I log a message, 
> but when this data is printed to STDERR
> How make it decoded from bytes to UTF-8 back? 
> my terminal supports UTF-8 and I want see nice messages instead of Ð¥Ð°Ð±Ð
>

UTF-8 *is* bytes. If you are encoding already encoded UTF-8 then you are 
double encoding.

--
sebastian 

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/mojolicious/9e334bec-97fd-45d7-b385-c28e813a0dd3n%40googlegroups.com.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2020-10-31 Thread kon...@gmail.com

Hi.
Ok, data is encoded from UTF-8 into bytes when I log a message, 
but when this data is printed to STDERR
How make it decoded from bytes to UTF-8 back? 
my terminal supports UTF-8 and I want see nice messages instead of Ð¥Ð°Ð±Ð

On Wednesday, October 10, 2018 at 7:08:03 PM UTC+3 gri...@gmail.com wrote:

> Mojo::Log already encodes data by default. Anything you send to stderr 
> outside of that should be encoded if necessary.
>
> -Dan
>
> On Wed, Oct 10, 2018 at 11:53 AM Alex Povolotsky  wrote:
>
>> So I should not binmode STDERR, ':utf8' if I'm using Mojo::Log and always 
>> explicitly encode() data?
>>
>> суббота, 6 октября 2018 г., 20:30:12 UTC+3 пользователь Dan Book написал:
>>>
>>> The script is not quite correct. I suspect if you add "use warnings" you 
>>> will see a "Wide character in print" warning, because you are not encoding 
>>> your unicode characters to UTF-8 before printing them. However, on the 
>>> server where you are seeing the double encoding, perhaps you have a 
>>> PERL5OPT environment variable which is setting something like "-CSAD" and 
>>> thus encoding all output handles by default (this should not be globally 
>>> set, because it can mess with modules that don't expect it like this).
>>>
>>> -Dan
>>>
>>> On Sat, Oct 6, 2018 at 10:15 AM Alex Povolotsky  
>>> wrote:
>>>
 Quite interesting. The simple script

 === cut mojolog === 

 #!/usr/bin/env perl 
 use strict; 
 use utf8; 
 use Mojo::Log; 
 my $log = new Mojo::Log; 
 print "Просто принт\n"; 
 $log->info('тест лога'); 
 === cut mojolog === 

 yields fine results on all my servers but one.

 On it, I'm getting

 % perl mojolog 
 Просто принт 
 [Sat Oct  6 15:22:43 2018] [info] Ñ�ÐµÑ�Ñ� Ð»Ð¾Ð³Ð° 

 The second line is clearly wrong and seems to be badly encoded. Same 
 version of OS, Perl, Mojolicious, same set of environment variables. What 
 else could influence unicode processing?

 суббота, 6 октября 2018 г., 13:35:48 UTC+3 пользователь Alex Povolotsky 
 написал:
>
> Hello
>
> How do I send UTF-8 constants to Mojo::Log? It forcefully encodes 
> everything so UTF-8 goes with double encoding and unreadable. Trying to 
> decode UTF-8 cat 
>
> $log->info(decode('UTF-8', 'тест лога'));
>
> results in
>
> Use of uninitialized value $_[0] in join or string at 
> /usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.
>
> Looks like something is wrong...
>
 -- 
 You received this message because you are subscribed to the Google 
 Groups "Mojolicious" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to mojolicious...@googlegroups.com.
 To post to this group, send email to mojol...@googlegroups.com.
 Visit this group at https://groups.google.com/group/mojolicious.
 For more options, visit https://groups.google.com/d/optout.

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Mojolicious" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to mojolicious...@googlegroups.com.
>> To post to this group, send email to mojol...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/mojolicious.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/mojolicious/7c222687-fa61-4588-97fb-ae3d358b365dn%40googlegroups.com.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2018-10-10 Thread Dan Book

Mojo::Log already encodes data by default. Anything you send to stderr
outside of that should be encoded if necessary.

-Dan

On Wed, Oct 10, 2018 at 11:53 AM Alex Povolotsky  wrote:

> So I should not binmode STDERR, ':utf8' if I'm using Mojo::Log and always
> explicitly encode() data?
>
> суббота, 6 октября 2018 г., 20:30:12 UTC+3 пользователь Dan Book написал:
>>
>> The script is not quite correct. I suspect if you add "use warnings" you
>> will see a "Wide character in print" warning, because you are not encoding
>> your unicode characters to UTF-8 before printing them. However, on the
>> server where you are seeing the double encoding, perhaps you have a
>> PERL5OPT environment variable which is setting something like "-CSAD" and
>> thus encoding all output handles by default (this should not be globally
>> set, because it can mess with modules that don't expect it like this).
>>
>> -Dan
>>
>> On Sat, Oct 6, 2018 at 10:15 AM Alex Povolotsky  wrote:
>>
>>> Quite interesting. The simple script
>>>
>>> === cut mojolog ===
>>>
>>> #!/usr/bin/env perl
>>> use strict;
>>> use utf8;
>>> use Mojo::Log;
>>> my $log = new Mojo::Log;
>>> print "Просто принт\n";
>>> $log->info('тест лога');
>>> === cut mojolog ===
>>>
>>> yields fine results on all my servers but one.
>>>
>>> On it, I'm getting
>>>
>>> % perl mojolog
>>> Просто принт
>>> [Sat Oct  6 15:22:43 2018] [info] Ñ�ÐµÑ�Ñ� Ð»Ð¾Ð³Ð°
>>>
>>> The second line is clearly wrong and seems to be badly encoded. Same
>>> version of OS, Perl, Mojolicious, same set of environment variables. What
>>> else could influence unicode processing?
>>>
>>> суббота, 6 октября 2018 г., 13:35:48 UTC+3 пользователь Alex Povolotsky
>>> написал:

 Hello

 How do I send UTF-8 constants to Mojo::Log? It forcefully encodes
 everything so UTF-8 goes with double encoding and unreadable. Trying to
 decode UTF-8 cat

 $log->info(decode('UTF-8', 'тест лога'));

 results in

 Use of uninitialized value $_[0] in join or string at
 /usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.

 Looks like something is wrong...

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Mojolicious" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to mojolicious...@googlegroups.com.
>>> To post to this group, send email to mojol...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/mojolicious.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Mojolicious" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mojolicious+unsubscr...@googlegroups.com.
> To post to this group, send email to mojolicious@googlegroups.com.
> Visit this group at https://groups.google.com/group/mojolicious.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

2018-10-10 Thread Alex Povolotsky

So I should not binmode STDERR, ':utf8' if I'm using Mojo::Log and always 
explicitly encode() data?

суббота, 6 октября 2018 г., 20:30:12 UTC+3 пользователь Dan Book написал:
>
> The script is not quite correct. I suspect if you add "use warnings" you 
> will see a "Wide character in print" warning, because you are not encoding 
> your unicode characters to UTF-8 before printing them. However, on the 
> server where you are seeing the double encoding, perhaps you have a 
> PERL5OPT environment variable which is setting something like "-CSAD" and 
> thus encoding all output handles by default (this should not be globally 
> set, because it can mess with modules that don't expect it like this).
>
> -Dan
>
> On Sat, Oct 6, 2018 at 10:15 AM Alex Povolotsky  > wrote:
>
>> Quite interesting. The simple script
>>
>> === cut mojolog === 
>>
>> #!/usr/bin/env perl 
>> use strict; 
>> use utf8; 
>> use Mojo::Log; 
>> my $log = new Mojo::Log; 
>> print "Просто принт\n"; 
>> $log->info('тест лога'); 
>> === cut mojolog === 
>>
>> yields fine results on all my servers but one.
>>
>> On it, I'm getting
>>
>> % perl mojolog 
>> Просто принт 
>> [Sat Oct  6 15:22:43 2018] [info] Ñ�ÐµÑ�Ñ� Ð»Ð¾Ð³Ð° 
>>
>> The second line is clearly wrong and seems to be badly encoded. Same 
>> version of OS, Perl, Mojolicious, same set of environment variables. What 
>> else could influence unicode processing?
>>
>> суббота, 6 октября 2018 г., 13:35:48 UTC+3 пользователь Alex Povolotsky 
>> написал:
>>>
>>> Hello
>>>
>>> How do I send UTF-8 constants to Mojo::Log? It forcefully encodes 
>>> everything so UTF-8 goes with double encoding and unreadable. Trying to 
>>> decode UTF-8 cat 
>>>
>>> $log->info(decode('UTF-8', 'тест лога'));
>>>
>>> results in
>>>
>>> Use of uninitialized value $_[0] in join or string at 
>>> /usr/local/lib/perl5/site_perl/Mojo/Log.pm line 55.
>>>
>>> Looks like something is wrong...
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Mojolicious" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to mojolicious...@googlegroups.com .
>> To post to this group, send email to mojol...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/mojolicious.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.

Re: [Mojolicious] Re: Mojo::Log and UTF-8

Re: [Mojolicious] Re: Mojo::Log and UTF-8

Re: [Mojolicious] Re: Mojo::Log and UTF-8

Re: [Mojolicious] Re: Mojo::Log and UTF-8

Re: [Mojolicious] Re: Mojo::Log and UTF-8

Re: [Mojolicious] Re: Mojo::Log and UTF-8

Re: [Mojolicious] Re: Mojo::Log and UTF-8

7 matches

Site Navigation

Mail list logo

Footer information