Re: Cygwin fails to utilize Unicode replacement character

[Steven Penny]

On Mon, 3 Sep 2018 23:02:58, Corinna Vinschen wrote:

I can't. I only have a limited set of fonts available in the console.


http://superuser.com/questions/390933/add-font-cmd-window-choices/956818


What I just did was calling the GetFontUnicodeRanges function
for each font, and it turns out that none of the fonts support
0xfffd "REPLACEMENT CHARACTER", but all three support 0xfffc
"OBJECT REPLACEMENT CHARACTER".  I expanded the testcase to check
for this with GetGlyphIndicesW and, lo and behold, the result
makes sense.


Here is my code if it helps:

   #include 
   #include 
   int main()
   {
 CONSOLE_FONT_INFOEX ta;
 ta.cbSize = sizeof ta;
 GetCurrentConsoleFontEx(GetStdHandle(STD_OUTPUT_HANDLE), 0, );
 HDC wh = GetDC(0);
 SelectObject(wh,
   CreateFontW(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ta.FaceName));
 WCHAR xr = 0xFFFD;
 WORD zu[1];
 GetGlyphIndicesW(wh, , 1, zu, 1);
 printf("%ls: %s\n", ta.FaceName, *zu == 0x ? "FAILURE" : "SUCCESS");
   }

Result:

   DejaVu Sans Mono: SUCCESS
   Consolas: FAILURE


On the other hand, during testing I saw a 0xfffd character printed for
these fonts.  None of them actually supports 0xfffd, so apparently the
Windows console already uses replacement fonts if possible.

I guess I just stop here and always print 0xfffd.  I seriously doubt
it makes sense to add so much code just to print a single char in a
border case.


this is not possible; most likely you were seeing the ".notdef glyph":

http://docs.microsoft.com/typography/opentype/spec/recom

for Consolas which is simlar in appearance to U+FFFD REPLACEMENT CHARACTER. The
differnce is that if you copy the ".notdef glyph" and paste it into "Notepad" or
similar, it will paste the proper character that couldnt be seen in the console,
while pasting U+FFFD into "Notepad" will just paste itself.

Expanding on the "Notepad" example, "Notepad" default font is "Lucida Console",
which doesnt have U+FFFD either. However pasting into "Notepad" will still show
U+FFFD properly because "Tahoma" has U+FFFD and "Notepad" can utilize composite
font, while it appears "cmd.exe" and similar cannot.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fails to utilize Unicode replacement character

[Corinna Vinschen]

On Sep  3 22:42, Thomas Wolff wrote:
> Am 03.09.2018 um 22:27 schrieb Corinna Vinschen:
> > On Sep  3 21:14, Corinna Vinschen wrote:
> > > On Sep  3 20:20, Thomas Wolff wrote:
> > > > Am 03.09.2018 um 19:56 schrieb Thomas Wolff:
> > > > > Am 03.09.2018 um 19:16 schrieb Corinna Vinschen:
> > > > > > On Sep  3 18:34, Thomas Wolff wrote:
> > > > > > > Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:
> > > > > > > > Does anybody have an idea what I'm doing wrong?
> > > > > > > This works in mintty, just uploaded a patch. Maybe somehow the
> > > > > > > GetConsole
> > > > > > > "dc" does not support this usage?
> > > > > > ¯\_(ツ)_/¯
> > > > > Dito; hold on, sorry, your code does *not* work inside mintty.
> > > > > Mine looks a bit different and I thought to have manually verified 
> > > > > it's
> > > > > functionally equivalent, but indeed there must be something fishy...
> > > > You still need to
> > > >    SelectObject(cdc, f);
> > > > where f is the HFONT of the font you want to check.
> > > > To compare, you may check out function win_check_glyphs in file 
> > > > wintext.c in
> > > > mintty.
> > > Thanks but I don't know how to get a HFONT for the current console font.
> > > 
> > > In the meantime I figured out why my GetCurrentConsoleFontEx call
> > > failed with error 87:
> > > 
> > > When looking again I realized there's a member called cbSize.  The MSDN
> > > docs neglect to tell that the cbSize member has to be primed with
> > > sizeof(CONSOLE_FONT_INFOEX).  As soon as I tried that, the function
> > > succeeded.
> > > 
> > > Well, it's a start.  I now have the actual font name.  No idea how to
> > > get a HFONT from there, though.  From what I can tell ATM, I'd have to
> > > call CreateFont to get a new HFONT and then destroy it again after
> > > usage.  This looks pretty wasteful.
> > Well, it still doesn't work for me.  I now have the following code:
> > 
> > = SNIP ==
> > [...]
> > = SNAP ==
> > 
> > Supposedly none of the fonts support 0xfffd:
> > 
> >$ gcc -g -o cons cons.c -lgdi32
> >$ ./cons
> >font Consolas
> >gi = 65535 879
> >rp_idx = 1
> >$ ./cons
> >font Lucida Console
> >gi = 65535 620
> >rp_idx = 1
> >$ ./cons
> >font Courier New
> >gi = 65535 372
> >rp_idx = 1
> > 
> > So I'm still doing something wrong, apparently.  Any hint?
> Test with a font that has the glyph; those 3 don't. Try DejaVu.

I can't. I only have a limited set of fonts available in the console.

But yes, you're right.

What I just did was calling the GetFontUnicodeRanges function
for each font, and it turns out that none of the fonts support
0xfffd "REPLACEMENT CHARACTER", but all three support 0xfffc
"OBJECT REPLACEMENT CHARACTER".  I expanded the testcase to check
for this with GetGlyphIndicesW and, lo and behold, the result
makes sense.

On the other hand, during testing I saw a 0xfffd character printed for
these fonts.  None of them actually supports 0xfffd, so apparently the
Windows console already uses replacement fonts if possible.

I guess I just stop here and always print 0xfffd.  I seriously doubt
it makes sense to add so much code just to print a single char in a
border case.


Thanks a lot for your help,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Cygwin fails to utilize Unicode replacement character

[Thomas Wolff]

Am 03.09.2018 um 22:27 schrieb Corinna Vinschen:

On Sep  3 21:14, Corinna Vinschen wrote:

On Sep  3 20:20, Thomas Wolff wrote:

Am 03.09.2018 um 19:56 schrieb Thomas Wolff:

Am 03.09.2018 um 19:16 schrieb Corinna Vinschen:

On Sep  3 18:34, Thomas Wolff wrote:

Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:

Does anybody have an idea what I'm doing wrong?

This works in mintty, just uploaded a patch. Maybe somehow the
GetConsole
"dc" does not support this usage?

¯\_(ツ)_/¯

Dito; hold on, sorry, your code does *not* work inside mintty.
Mine looks a bit different and I thought to have manually verified it's
functionally equivalent, but indeed there must be something fishy...

You still need to
   SelectObject(cdc, f);
where f is the HFONT of the font you want to check.
To compare, you may check out function win_check_glyphs in file wintext.c in
mintty.

Thanks but I don't know how to get a HFONT for the current console font.

In the meantime I figured out why my GetCurrentConsoleFontEx call
failed with error 87:

When looking again I realized there's a member called cbSize.  The MSDN
docs neglect to tell that the cbSize member has to be primed with
sizeof(CONSOLE_FONT_INFOEX).  As soon as I tried that, the function
succeeded.

Well, it's a start.  I now have the actual font name.  No idea how to
get a HFONT from there, though.  From what I can tell ATM, I'd have to
call CreateFont to get a new HFONT and then destroy it again after
usage.  This looks pretty wasteful.

Well, it still doesn't work for me.  I now have the following code:

= SNIP ==
#include 
#include 
#include 

int
main ()
{
   static const wchar_t replacement_char[2] =
 {
   0xfffd, /* REPLACEMENT CHARACTER */
   0x2592  /* MEDIUM SHADE */
 };

   CONSOLE_FONT_INFOEX cfi;
   HWND cwnd = GetConsoleWindow ();
   HDC cdc = GetDC (cwnd);
   int rp_idx = 1;
   WORD gi[2] = { 0, 0 };

   memset (, 0, sizeof cfi);
   cfi.cbSize = sizeof cfi;
   if (GetCurrentConsoleFontEx (GetStdHandle (STD_OUTPUT_HANDLE), FALSE, ))
 {
   printf ("font %ls\n", cfi.FaceName);
   HFONT hf = CreateFontW (cfi.dwFontSize.Y, cfi.dwFontSize.X,
  0, 0, cfi.FontWeight, FALSE, FALSE, FALSE,
  DEFAULT_CHARSET, OUT_DEFAULT_PRECIS,
  CLIP_DEFAULT_PRECIS, DEFAULT_QUALITY,
  FIXED_PITCH | FF_DONTCARE, cfi.FaceName);
   if (hf)
{
  HFONT old_f = SelectObject(cdc, hf);
  if (GetGlyphIndicesW (cdc, replacement_char, 2, gi,
GGI_MARK_NONEXISTING_GLYPHS) != GDI_ERROR)
{
  printf ("gi = %d %d\n", gi[0], gi[1]);
  if (gi[0] != 0x)
rp_idx = 0;
}
  if (old_f)
old_f = SelectObject (cdc, old_f);
  DeleteObject (hf);
}
 }

   printf ("rp_idx = %d\n", rp_idx);
   return 0;
}
= SNAP ==

Supposedly none of the fonts support 0xfffd:

   $ gcc -g -o cons cons.c -lgdi32
   $ ./cons
   font Consolas
   gi = 65535 879
   rp_idx = 1
   $ ./cons
   font Lucida Console
   gi = 65535 620
   rp_idx = 1
   $ ./cons
   font Courier New
   gi = 65535 372
   rp_idx = 1

So I'm still doing something wrong, apparently.  Any hint?

Test with a font that has the glyph; those 3 don't. Try DejaVu.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

On Sep  3 21:12, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> How feasible would it be to generate an alternate setup.ini
> >> (setup-snapshots.ini or something) and include the snapshots in the
> >> actual mirror with a switch to setup to select the alternate file?
> >> When we finally get to it with OCaml's CI, that is probably how I
> >> would configure our CI to operate (i.e. synchronise with a mirror,
> >> download the latest snapshot and then patch setup.ini), but I imagine
> >> there might be other projects which would set jobs running if it were
> >> very simple from upstream to do. 
> >
> > I'll release 2.11.1 today or tomorrow.
> 
> I almost gave the same answer when I read that yesterday.  However, I
> think David was asking for a way to let his CI pick up a snapshot when
> one is available so it would automatically give an indication if there
> was any problem with it.

The answer is no, but as soon as I have a test release, it's available
via the default setup.ini.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Cygwin fails to utilize Unicode replacement character

[Corinna Vinschen]

On Sep  3 21:14, Corinna Vinschen wrote:
> On Sep  3 20:20, Thomas Wolff wrote:
> > Am 03.09.2018 um 19:56 schrieb Thomas Wolff:
> > > Am 03.09.2018 um 19:16 schrieb Corinna Vinschen:
> > > > On Sep  3 18:34, Thomas Wolff wrote:
> > > > > Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:
> > > > > > Does anybody have an idea what I'm doing wrong?
> > > > > This works in mintty, just uploaded a patch. Maybe somehow the
> > > > > GetConsole
> > > > > "dc" does not support this usage?
> > > > ¯\_(ツ)_/¯
> > > Dito; hold on, sorry, your code does *not* work inside mintty.
> > > Mine looks a bit different and I thought to have manually verified it's
> > > functionally equivalent, but indeed there must be something fishy...
> > You still need to
> >   SelectObject(cdc, f);
> > where f is the HFONT of the font you want to check.
> > To compare, you may check out function win_check_glyphs in file wintext.c in
> > mintty.
> 
> Thanks but I don't know how to get a HFONT for the current console font.
> 
> In the meantime I figured out why my GetCurrentConsoleFontEx call
> failed with error 87:
> 
> When looking again I realized there's a member called cbSize.  The MSDN
> docs neglect to tell that the cbSize member has to be primed with
> sizeof(CONSOLE_FONT_INFOEX).  As soon as I tried that, the function
> succeeded.
> 
> Well, it's a start.  I now have the actual font name.  No idea how to
> get a HFONT from there, though.  From what I can tell ATM, I'd have to
> call CreateFont to get a new HFONT and then destroy it again after
> usage.  This looks pretty wasteful.

Well, it still doesn't work for me.  I now have the following code:

= SNIP ==
#include 
#include 
#include 

int
main ()
{
  static const wchar_t replacement_char[2] =
{
  0xfffd, /* REPLACEMENT CHARACTER */
  0x2592  /* MEDIUM SHADE */
};

  CONSOLE_FONT_INFOEX cfi;
  HWND cwnd = GetConsoleWindow ();
  HDC cdc = GetDC (cwnd);
  int rp_idx = 1;
  WORD gi[2] = { 0, 0 };

  memset (, 0, sizeof cfi);
  cfi.cbSize = sizeof cfi;
  if (GetCurrentConsoleFontEx (GetStdHandle (STD_OUTPUT_HANDLE), FALSE, ))
{
  printf ("font %ls\n", cfi.FaceName);
  HFONT hf = CreateFontW (cfi.dwFontSize.Y, cfi.dwFontSize.X,
  0, 0, cfi.FontWeight, FALSE, FALSE, FALSE,
  DEFAULT_CHARSET, OUT_DEFAULT_PRECIS,
  CLIP_DEFAULT_PRECIS, DEFAULT_QUALITY,
  FIXED_PITCH | FF_DONTCARE, cfi.FaceName);
  if (hf)
{
  HFONT old_f = SelectObject(cdc, hf);
  if (GetGlyphIndicesW (cdc, replacement_char, 2, gi,
GGI_MARK_NONEXISTING_GLYPHS) != GDI_ERROR)
{
  printf ("gi = %d %d\n", gi[0], gi[1]);
  if (gi[0] != 0x)
rp_idx = 0;
}
  if (old_f)
old_f = SelectObject (cdc, old_f);
  DeleteObject (hf);
}
}

  printf ("rp_idx = %d\n", rp_idx);
  return 0;
}
= SNAP ==

Supposedly none of the fonts support 0xfffd:

  $ gcc -g -o cons cons.c -lgdi32
  $ ./cons
  font Consolas
  gi = 65535 879
  rp_idx = 1
  $ ./cons
  font Lucida Console
  gi = 65535 620
  rp_idx = 1
  $ ./cons
  font Courier New
  gi = 65535 372
  rp_idx = 1

So I'm still doing something wrong, apparently.  Any hint?


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Cygwin fails to utilize Unicode replacement character

[Corinna Vinschen]

On Sep  3 20:20, Thomas Wolff wrote:
> Am 03.09.2018 um 19:56 schrieb Thomas Wolff:
> > Am 03.09.2018 um 19:16 schrieb Corinna Vinschen:
> > > On Sep  3 18:34, Thomas Wolff wrote:
> > > > Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:
> > > > > On Sep  3 14:46, Corinna Vinschen wrote:
> > > > > > On Sep  2 05:51, Steven Penny wrote:
> > > > > > > On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:
> > > > > > > > Actually, the width problem I suggested in my
> > > > > > > > other response (and even
> > > > > > > > referring to the wrong character) does not apply as mintty 
> > > > > > > > enforces
> > > > > > > > proper width in that case.
> > > > > > > > Also, even with fonts that do not provide the
> > > > > > > > glyph, you will usually
> > > > > > > > still see it by the Windows font fallback mechanism.
> > > > > > > > Shall I make it configurable?
> > > > > > > your call - here are the possible resolutions - in
> > > > > > > order of my preference:
> > > > > > > 
> > > > > > > 1. Change the default to U+FFFD with no option
> > > > > > > 2. Change the default to U+FFFD with option to change
> > > > > > > 3. Leave default as is with option to change
> > > > > > Ideally we could check if the current font supports a visual
> > > > > > representation of 0xfffd and if not, fall back to 0x2592.
> > > > > > 
> > > > > > Not sure how feasible that is, but it doesn't seem to be overly
> > > > > > complicated.  I'm just looking into a solution for the Cygwin
> > > > > > console.
> > > > > Only, I can't get this working.  In theory the GDI function
> > > > > GetGlyphIndicesW is supposed to allow checking if a certain character
> > > > > exists.  But I'm getting a weird result.  This code:
> > > > > 
> > > > >     static const wchar_t replacement_char[2] =
> > > > >   {
> > > > >     0xfffd, /* REPLACEMENT CHARACTER */
> > > > >     0x2592  /* MEDIUM SHADE */
> > > > >   };
> > > > >     HWND cwnd = GetConsoleWindow ();
> > > > >     HDC cdc = GetDC (cwnd);
> > > > >     int rp_idx = 0;
> > > > >     WORD gi = 0;
> > > > >     DWORD ret = GetGlyphIndicesW (cdc, replacement_char, 1, ,
> > > > > GGI_MARK_NONEXISTING_GLYPHS);
> > > > >     if (ret != GDI_ERROR && gi == 0x)
> > > > >   rp_idx = 1;
> > > > > 
> > > > > always sets rp_idx to 1 when called from inside the Cygwin DLL,
> > > > > independently of the actual console font.  And, here's the
> > > > > really weird
> > > > > thing, it always sets rp_idx to 0 when called directly from an
> > > > > application, likewise independently of the actual console font.
> > > > > 
> > > > > Does anybody have an idea what I'm doing wrong?
> > > > This works in mintty, just uploaded a patch. Maybe somehow the
> > > > GetConsole
> > > > "dc" does not support this usage?
> > > ¯\_(ツ)_/¯
> > Dito; hold on, sorry, your code does *not* work inside mintty.
> > Mine looks a bit different and I thought to have manually verified it's
> > functionally equivalent, but indeed there must be something fishy...
> You still need to
>   SelectObject(cdc, f);
> where f is the HFONT of the font you want to check.
> To compare, you may check out function win_check_glyphs in file wintext.c in
> mintty.

Thanks but I don't know how to get a HFONT for the current console font.

In the meantime I figured out why my GetCurrentConsoleFontEx call
failed with error 87:

When looking again I realized there's a member called cbSize.  The MSDN
docs neglect to tell that the cbSize member has to be primed with
sizeof(CONSOLE_FONT_INFOEX).  As soon as I tried that, the function
succeeded.

Well, it's a start.  I now have the actual font name.  No idea how to
get a HFONT from there, though.  From what I can tell ATM, I'd have to
call CreateFont to get a new HFONT and then destroy it again after
usage.  This looks pretty wasteful.


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Achim Gratz]

Corinna Vinschen writes:
>> How feasible would it be to generate an alternate setup.ini
>> (setup-snapshots.ini or something) and include the snapshots in the
>> actual mirror with a switch to setup to select the alternate file?
>> When we finally get to it with OCaml's CI, that is probably how I
>> would configure our CI to operate (i.e. synchronise with a mirror,
>> download the latest snapshot and then patch setup.ini), but I imagine
>> there might be other projects which would set jobs running if it were
>> very simple from upstream to do. 
>
> I'll release 2.11.1 today or tomorrow.

I almost gave the same answer when I read that yesterday.  However, I
think David was asking for a way to let his CI pick up a snapshot when
one is available so it would automatically give an indication if there
was any problem with it.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fails to utilize Unicode replacement character

[Thomas Wolff]

Am 03.09.2018 um 19:56 schrieb Thomas Wolff:

Am 03.09.2018 um 19:16 schrieb Corinna Vinschen:

On Sep  3 18:34, Thomas Wolff wrote:

Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:

On Sep  3 14:46, Corinna Vinschen wrote:

On Sep  2 05:51, Steven Penny wrote:

On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:
Actually, the width problem I suggested in my other response 
(and even

referring to the wrong character) does not apply as mintty enforces
proper width in that case.
Also, even with fonts that do not provide the glyph, you will 
usually

still see it by the Windows font fallback mechanism.
Shall I make it configurable?
your call - here are the possible resolutions - in order of my 
preference:


1. Change the default to U+FFFD with no option
2. Change the default to U+FFFD with option to change
3. Leave default as is with option to change

Ideally we could check if the current font supports a visual
representation of 0xfffd and if not, fall back to 0x2592.

Not sure how feasible that is, but it doesn't seem to be overly
complicated.  I'm just looking into a solution for the Cygwin
console.

Only, I can't get this working.  In theory the GDI function
GetGlyphIndicesW is supposed to allow checking if a certain character
exists.  But I'm getting a weird result.  This code:

    static const wchar_t replacement_char[2] =
  {
    0xfffd, /* REPLACEMENT CHARACTER */
    0x2592  /* MEDIUM SHADE */
  };
    HWND cwnd = GetConsoleWindow ();
    HDC cdc = GetDC (cwnd);
    int rp_idx = 0;
    WORD gi = 0;
    DWORD ret = GetGlyphIndicesW (cdc, replacement_char, 1, ,
GGI_MARK_NONEXISTING_GLYPHS);
    if (ret != GDI_ERROR && gi == 0x)
  rp_idx = 1;

always sets rp_idx to 1 when called from inside the Cygwin DLL,
independently of the actual console font.  And, here's the really 
weird

thing, it always sets rp_idx to 0 when called directly from an
application, likewise independently of the actual console font.

Does anybody have an idea what I'm doing wrong?
This works in mintty, just uploaded a patch. Maybe somehow the 
GetConsole

"dc" does not support this usage?

¯\_(ツ)_/¯

Dito; hold on, sorry, your code does *not* work inside mintty.
Mine looks a bit different and I thought to have manually verified 
it's functionally equivalent, but indeed there must be something fishy...

You still need to
  SelectObject(cdc, f);
where f is the HFONT of the font you want to check.
To compare, you may check out function win_check_glyphs in file 
wintext.c in mintty.

Thomas

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fails to utilize Unicode replacement character

[Thomas Wolff]

Am 03.09.2018 um 19:16 schrieb Corinna Vinschen:

On Sep  3 18:34, Thomas Wolff wrote:

Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:

On Sep  3 14:46, Corinna Vinschen wrote:

On Sep  2 05:51, Steven Penny wrote:

On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:

Actually, the width problem I suggested in my other response (and even
referring to the wrong character) does not apply as mintty enforces
proper width in that case.
Also, even with fonts that do not provide the glyph, you will usually
still see it by the Windows font fallback mechanism.
Shall I make it configurable?

your call - here are the possible resolutions - in order of my preference:

1. Change the default to U+FFFD with no option
2. Change the default to U+FFFD with option to change
3. Leave default as is with option to change

Ideally we could check if the current font supports a visual
representation of 0xfffd and if not, fall back to 0x2592.

Not sure how feasible that is, but it doesn't seem to be overly
complicated.  I'm just looking into a solution for the Cygwin
console.

Only, I can't get this working.  In theory the GDI function
GetGlyphIndicesW is supposed to allow checking if a certain character
exists.  But I'm getting a weird result.  This code:

static const wchar_t replacement_char[2] =
  {
0xfffd, /* REPLACEMENT CHARACTER */
0x2592  /* MEDIUM SHADE */
  };
HWND cwnd = GetConsoleWindow ();
HDC cdc = GetDC (cwnd);
int rp_idx = 0;
WORD gi = 0;
DWORD ret = GetGlyphIndicesW (cdc, replacement_char, 1, ,
  GGI_MARK_NONEXISTING_GLYPHS);
if (ret != GDI_ERROR && gi == 0x)
  rp_idx = 1;

always sets rp_idx to 1 when called from inside the Cygwin DLL,
independently of the actual console font.  And, here's the really weird
thing, it always sets rp_idx to 0 when called directly from an
application, likewise independently of the actual console font.

Does anybody have an idea what I'm doing wrong?

This works in mintty, just uploaded a patch. Maybe somehow the GetConsole
"dc" does not support this usage?

¯\_(ツ)_/¯

Dito; hold on, sorry, your code does *not* work inside mintty.
Mine looks a bit different and I thought to have manually verified it's 
functionally equivalent, but indeed there must be something fishy...


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fails to utilize Unicode replacement character

[Corinna Vinschen]

On Sep  3 18:34, Thomas Wolff wrote:
> Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:
> > On Sep  3 14:46, Corinna Vinschen wrote:
> > > On Sep  2 05:51, Steven Penny wrote:
> > > > On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:
> > > > > Actually, the width problem I suggested in my other response (and even
> > > > > referring to the wrong character) does not apply as mintty enforces
> > > > > proper width in that case.
> > > > > Also, even with fonts that do not provide the glyph, you will usually
> > > > > still see it by the Windows font fallback mechanism.
> > > > > Shall I make it configurable?
> > > > your call - here are the possible resolutions - in order of my 
> > > > preference:
> > > > 
> > > > 1. Change the default to U+FFFD with no option
> > > > 2. Change the default to U+FFFD with option to change
> > > > 3. Leave default as is with option to change
> > > Ideally we could check if the current font supports a visual
> > > representation of 0xfffd and if not, fall back to 0x2592.
> > > 
> > > Not sure how feasible that is, but it doesn't seem to be overly
> > > complicated.  I'm just looking into a solution for the Cygwin
> > > console.
> > Only, I can't get this working.  In theory the GDI function
> > GetGlyphIndicesW is supposed to allow checking if a certain character
> > exists.  But I'm getting a weird result.  This code:
> > 
> >static const wchar_t replacement_char[2] =
> >  {
> >0xfffd, /* REPLACEMENT CHARACTER */
> >0x2592  /* MEDIUM SHADE */
> >  };
> >HWND cwnd = GetConsoleWindow ();
> >HDC cdc = GetDC (cwnd);
> >int rp_idx = 0;
> >WORD gi = 0;
> >DWORD ret = GetGlyphIndicesW (cdc, replacement_char, 1, ,
> >  GGI_MARK_NONEXISTING_GLYPHS);
> >if (ret != GDI_ERROR && gi == 0x)
> >  rp_idx = 1;
> > 
> > always sets rp_idx to 1 when called from inside the Cygwin DLL,
> > independently of the actual console font.  And, here's the really weird
> > thing, it always sets rp_idx to 0 when called directly from an
> > application, likewise independently of the actual console font.
> > 
> > Does anybody have an idea what I'm doing wrong?
> This works in mintty, just uploaded a patch. Maybe somehow the GetConsole
> "dc" does not support this usage?

¯\_(ツ)_/¯

But I'm glad it works for you.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Cygwin fails to utilize Unicode replacement character

[Thomas Wolff]

Am 03.09.2018 um 16:59 schrieb Corinna Vinschen:

On Sep  3 14:46, Corinna Vinschen wrote:

On Sep  2 05:51, Steven Penny wrote:

On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:

Actually, the width problem I suggested in my other response (and even
referring to the wrong character) does not apply as mintty enforces
proper width in that case.
Also, even with fonts that do not provide the glyph, you will usually
still see it by the Windows font fallback mechanism.
Shall I make it configurable?

your call - here are the possible resolutions - in order of my preference:

1. Change the default to U+FFFD with no option
2. Change the default to U+FFFD with option to change
3. Leave default as is with option to change

Ideally we could check if the current font supports a visual
representation of 0xfffd and if not, fall back to 0x2592.

Not sure how feasible that is, but it doesn't seem to be overly
complicated.  I'm just looking into a solution for the Cygwin
console.

Only, I can't get this working.  In theory the GDI function
GetGlyphIndicesW is supposed to allow checking if a certain character
exists.  But I'm getting a weird result.  This code:

   static const wchar_t replacement_char[2] =
 {
   0xfffd, /* REPLACEMENT CHARACTER */
   0x2592  /* MEDIUM SHADE */
 };
   HWND cwnd = GetConsoleWindow ();
   HDC cdc = GetDC (cwnd);
   int rp_idx = 0;
   WORD gi = 0;
   DWORD ret = GetGlyphIndicesW (cdc, replacement_char, 1, ,
 GGI_MARK_NONEXISTING_GLYPHS);
   if (ret != GDI_ERROR && gi == 0x)
 rp_idx = 1;

always sets rp_idx to 1 when called from inside the Cygwin DLL,
independently of the actual console font.  And, here's the really weird
thing, it always sets rp_idx to 0 when called directly from an
application, likewise independently of the actual console font.

Does anybody have an idea what I'm doing wrong?
This works in mintty, just uploaded a patch. Maybe somehow the 
GetConsole "dc" does not support this usage?


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Cygwin fails to utilize Unicode replacement character

[Brian Inglis]

On 2018-09-02 06:51, Steven Penny wrote:
> On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:
>> Actually, the width problem I suggested in my other response (and even 
>> referring to the wrong character) does not apply as mintty enforces proper 
>> width in that case.
>> Also, even with fonts that do not provide the glyph, you will usually
>> still see it by the Windows font fallback mechanism.
>> Shall I make it configurable?
> your call - here are the possible resolutions - in order of my preference:

What about TTF .notdef glyph index 0?

> 1. Change the default to U+FFFD with no option
> 2. Change the default to U+FFFD with option to change
> 3. Leave default as is with option to change

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

[ANNOUNCEMENT] Updated: hwloc 1.11.11-1

[Marco Atzeri]

Version 1.11.11-1  of packages

  hwloc
  libhwloc-devel
  libhwloc5

are available in the Cygwin distribution:

CHANGES
Upstream bugfix release

Full upstream changes:
https://www.mail-archive.com/hwloc-announce@lists.open-mpi.org/msg00118.html

DESCRIPTION
The Portable Hardware Locality (hwloc) software package provides a portable
abstraction (across OS, versions, architectures, ...) of the hierarchical
topology of modern architectures, including NUMA memory nodes, sockets,
shared caches, cores and simultaneous multithreading.

HOMEPAGE
http://www.open-mpi.org/projects/hwloc/

Marco Atzeri

If you have questions or comments, please send them to the
cygwin mailing list at: cygwin (at) cygwin (dot) com .

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Updated: hwloc 1.11.11-1

[Marco Atzeri]

Version 1.11.11-1  of packages

  hwloc
  libhwloc-devel
  libhwloc5

are available in the Cygwin distribution:

CHANGES
Upstream bugfix release

Full upstream changes:
https://www.mail-archive.com/hwloc-announce@lists.open-mpi.org/msg00118.html

DESCRIPTION
The Portable Hardware Locality (hwloc) software package provides a portable
abstraction (across OS, versions, architectures, ...) of the hierarchical
topology of modern architectures, including NUMA memory nodes, sockets,
shared caches, cores and simultaneous multithreading.

HOMEPAGE
http://www.open-mpi.org/projects/hwloc/

Marco Atzeri

If you have questions or comments, please send them to the
cygwin mailing list at: cygwin (at) cygwin (dot) com .

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus

Re: Cygwin fails to utilize Unicode replacement character

[Corinna Vinschen]

On Sep  3 14:46, Corinna Vinschen wrote:
> On Sep  2 05:51, Steven Penny wrote:
> > On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:
> > > Actually, the width problem I suggested in my other response (and even
> > > referring to the wrong character) does not apply as mintty enforces
> > > proper width in that case.
> > > Also, even with fonts that do not provide the glyph, you will usually
> > > still see it by the Windows font fallback mechanism.
> > > Shall I make it configurable?
> > 
> > your call - here are the possible resolutions - in order of my preference:
> > 
> > 1. Change the default to U+FFFD with no option
> > 2. Change the default to U+FFFD with option to change
> > 3. Leave default as is with option to change
> 
> Ideally we could check if the current font supports a visual
> representation of 0xfffd and if not, fall back to 0x2592.
> 
> Not sure how feasible that is, but it doesn't seem to be overly
> complicated.  I'm just looking into a solution for the Cygwin
> console.

Only, I can't get this working.  In theory the GDI function
GetGlyphIndicesW is supposed to allow checking if a certain character
exists.  But I'm getting a weird result.  This code:

  static const wchar_t replacement_char[2] =
{
  0xfffd, /* REPLACEMENT CHARACTER */
  0x2592  /* MEDIUM SHADE */
};
  HWND cwnd = GetConsoleWindow ();
  HDC cdc = GetDC (cwnd);
  int rp_idx = 0;
  WORD gi = 0;
  DWORD ret = GetGlyphIndicesW (cdc, replacement_char, 1, ,
GGI_MARK_NONEXISTING_GLYPHS);
  if (ret != GDI_ERROR && gi == 0x)
rp_idx = 1;

always sets rp_idx to 1 when called from inside the Cygwin DLL,
independently of the actual console font.  And, here's the really weird
thing, it always sets rp_idx to 0 when called directly from an
application, likewise independently of the actual console font.

Does anybody have an idea what I'm doing wrong?

Just as side-note:

- GetTextFaceW always returns font number 7 called "System", independently
  of the actual current font set in the console.
- GetCurrentConsoleFont always returns a font number of 0, independently
  of the actual current font set in the console.
  GetCurrentConsoleFontEx always returns with error 87, "invalid
  parameter"

Something's very fishy.  Thanks for any actual help.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Cygwin fails to utilize Unicode replacement character

[Corinna Vinschen]

On Sep  2 05:51, Steven Penny wrote:
> On Sun, 2 Sep 2018 10:07:10, Thomas Wolff wrote:
> > Actually, the width problem I suggested in my other response (and even
> > referring to the wrong character) does not apply as mintty enforces
> > proper width in that case.
> > Also, even with fonts that do not provide the glyph, you will usually
> > still see it by the Windows font fallback mechanism.
> > Shall I make it configurable?
> 
> your call - here are the possible resolutions - in order of my preference:
> 
> 1. Change the default to U+FFFD with no option
> 2. Change the default to U+FFFD with option to change
> 3. Leave default as is with option to change

Ideally we could check if the current font supports a visual
representation of 0xfffd and if not, fall back to 0x2592.

Not sure how feasible that is, but it doesn't seem to be overly
complicated.  I'm just looking into a solution for the Cygwin
console.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Question for install HEG tool

[Andrey Repin]

Greetings, 王东!

> I am  so sorry for bothering you. There are some questions when I install
> HEG tool in my windows platfom. The error information is 

> ' 0 [main] bash 6340 find_fast_cwd: WARNING: Couldn't compute FAST_CWD 
> pointer.'

> So, I have no idea to solve it.

The issue has been solved over a decade ago.
Please update your Cygwin install.

> Could you please help me to solve this
> problem? Thank you for your generous help. I look forward to you reply.

https://cygwin.com/faq.html#faq.using.fixing-find_fast_cwd-warnings


-- 
With best regards,
Andrey Repin
Monday, September 3, 2018 13:22:51

Sorry for my terrible english...

can't run help_crack.py under windows environment

[Ashish Deharia]

I tried Hashcat but still no luck. It shows insufficient memory error. I am
attaching my desktop recording here. I hope you may be able to help me out
here. Cheers!
 Desktop 2018.09.03 - 09.45.13.01.mp4


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: can't run help_crack.py under windows environment

[Ashish Deharia]

I tried Hashcat but still no luck. It shows insufficient memory error. I am
attaching my desktop recording here. I hope you may be able to help me out
here. Cheers!
 Desktop 2018.09.03 - 09.45.13.01.mp4


On Mon, 3 Sep 2018 at 13:20, Ashish Deharia  wrote:

> I was trying to get help_crack.py to work in my PC. Initially it showed an
> error, Hashcat or John not found then I downloaded John and placed
> help_crack.py in the same directory where I unzipped John. but still it
> doesn't work.
>
> I am attaching a video of the same.
>

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Question for install HEG tool

[王东]

Dear Sir,

I am  so sorry for bothering you. There are some questions when I install HEG 
tool in my windows platfom. The error information is 

' 0 [main] bash 6340 find_fast_cwd: WARNING: Couldn't compute FAST_CWD pointer.'

So, I have no idea to solve it. Could you please help me to solve this problem? 
Thank you for your generous help. I look forward to you reply.

Thank you again!

Best regards!

Dong Wang
timd...@whu.edu.cn

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Bug Report: Regression in Cygwin 2.11.0-1

[Corinna Vinschen]

On Sep  2 08:37, David Allsopp wrote:
> Corinna Vinschen wrote:
> > On Sep  1 16:41, David Allsopp wrote:
> > > Corinna Vinschen wrote:
> > > > Some of the path handling is seriously crippled as soon as you start
> > > > using backslashes, and that's a delipberate decision and won't
> > > > change, even after fixing the aforementioned bug.
> > >
> > > I don't quite follow this - does that mean that that path definitely
> > > work even with a new Cygwin? i.e. our existing build system is
> > > permanently broken for Cygwin 2.11+?
> > 
> > No.  The problem we're talking about here is a regression and needs fixing, 
> > no
> > doubt about it.
> > 
> > I was just saying that win32 paths *in general* get a less complete 
> > treatment
> > than POSIX paths for hopefully obvious reasons.  Just be aware that 
> > \foo\bar is
> > not the same as /foo/bar and may even point to another file.
> >
> > As for the bug in question:  I pushed a patch which should fix this issue.  
> > I
> > created new developer snapshots and uploaded them to
> > https://cygwin.com/snapshots/.  Please give them a try.
> 
> Thanks! That snapshot does indeed fix our build.

Thanks for testing.

> How feasible would it be to generate an alternate setup.ini
> (setup-snapshots.ini or something) and include the snapshots in the
> actual mirror with a switch to setup to select the alternate file?
> When we finally get to it with OCaml's CI, that is probably how I
> would configure our CI to operate (i.e. synchronise with a mirror,
> download the latest snapshot and then patch setup.ini), but I imagine
> there might be other projects which would set jobs running if it were
> very simple from upstream to do. 

I'll release 2.11.1 today or tomorrow.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: WARNING: Couldn't compute FAST_CWD pointer

[Marco Atzeri]

Am 03.09.2018 um 07:42 schrieb Christine dbit:

  242 [main] bash 11160 find_fast_cwd: WARNING: Couldn't compute FAST_CWD
pointer.  Please report this problem to
the public mailing list cygwin@cygwin.com
my console shows this, I hope you can help me



https://cygwin.com/faq.html#faq.using.fixing-find_fast_cwd-warnings

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple