Re: [OpenJDK 2D-Dev] [9] RFR JDK-8147002:[macosx] Arabic character cannot be rendered on MacOS X

[Prasanta Sadhukhan]

Hi Phil,

I was using osx10.11 but the list I enumerated in JBS is actually the 
composite font I get in CCompositeGlyphMapper#convertToGlyph() which 
contains those 29 slots 
[CCompositeGlyphMapper#convertToGlyph(unicode=value 1575) gets the glyph 
code(909) from slot[4] which is CourierNewPSMT, for italic the list was 
taken from convertToGlyph()]


But, if we are considering the cascade list "listOfString", it contains 
.NotoSansUniversal just before  AppleColorEmoji.


However, it does not get added to CompositeFont list because 
findFont2D() returned null for it. There was another hidden font in the 
cascade list ".AppleSymbolsFB" but it was changed to "AppleSymbols" and 
that along with other fonts findFont2D() was able to find and could add 
to CompositeFont list for convertGlyph() to use.


Regards
Prasanta
On 2/15/2017 10:40 PM, Phil Race wrote:

Hi,
I don't know what versions of MacOS X either of you (Dmitry+Prasanta) 
used but I am going to guess they are different

(can you both report these, for the record).

In https://bugs.openjdk.java.net/browse/JDK-8147002, Prasanta enumerates
the cascade list for the Italic case and I don't see 
.NotoSansUniversal anywhere on that list.


So although if enumerated it likely still won't work (since it is not 
enumerated by the other APIs),
there does not appear to be any font in the cascade list reported by 
Prasanta that "would not work",

so the fix still seems fine to me as it is plugging a gap.

A fix for using "hidden" fonts in the cascade list seems like a JDK 10 
candidate unless there

is some critical case we are still missing.

-phil.



On 02/13/2017 01:56 AM, Dmitry Batrak wrote:

> Yes, we are aware this fixed it only for Oracle JDK.
> The bit I missed is that GeezaProInterface was then used on OpenJDK by JRS.
> So we could add that as well.
> If you have any other fonts you think should be added to the tail of that 
list we
> can add them too.

I ran JRSFontCreateFallbackFontForCharacters against all Unicode code 
points with Menlo-Italic as base font,
and got 63 possible fallback fonts - vs 29 present in cascade list, 
currently used by JDK 9 (given in JDK-8147002).
So the mismatch is quite large. But the interesting thing is that 
cascade list returned by
CTFontCopyDefaultCascadeListForLanguages still covers that whole 
Unicode range. The problem is that JDK
doesn't use that list fully. In particular, Arabic alef letter U+0627 
can be displayed using the font
.NotoSansUniversal from that list. This font doesn't get into 
composite font, as it's not found among the fonts
known to Java. So a better fix (and still quite small) will probably 
be using those 'hidden' fonts as part of
cascade composites - by creating a new, 'local' CFont instance if 
lookup in the known font list fails
(this, btw, seems to work fine for us). If even this is considered 
risky for JDK 9, then probably adding
.NotoSansUniversal with or instead of .GeezaProInterface as a 
hardcoded component makes sense.

Best regards,
Dmitry Batrak

On Fri, Feb 10, 2017 at 8:19 PM, Phil Race > wrote:


I think that so long as we are using JRS* in some places and
CoreText's cascading list
in others there is the likelihood of some inconsistency where JRS
introduces some magic
that we don't know about

So canDisplay() needs to be fixed to be consistent and perhaps we
also need to see
if we are always getting the glyphs from the same fonts for the
cases where there
is no missing glyph.

I do agree that getting away from JRS* support is the way to go
so we are in control.
However not at this time in JDK 9.

Yes "." fonts are not enumerated by OS X but if you can find them
they work.
This is how the UI fonts are handled.

Yes, we are aware this fixed it only for Oracle JDK.
The bit I missed is that GeezaProInterface was then used on
OpenJDK by JRS.
So we could add that as well.
If you have any other fonts you think should be added to the tail
of that list we
can add them too.

-phil.

On 02/10/2017 07:59 AM, Dmitry Batrak wrote:

Hi,

I've tried to build OpenJDK from 'client' repository with the
fix, and the
issue is still reproducible for me. Are you fixing it only
for Oracle JDK
(including Lucida Sans Regular font), and not for OpenJDK?
The problem is not
that some character cannot be rendered using a particular
font, it's that
rendering behaviour doesn't match what 'canDisplay' returns.

If you're interested I can share the way we've fixed this
issue in our
OpenJDK-based runtime.

As you've mentioned, the problem is that
JRSFontCreateFallbackFontForCharacters returns fallback font,
which is not on
the cascaded font list, and it can be absent even in the
whole list of fonts
reported 

Re: [OpenJDK 2D-Dev] [9] RFR JDK-8147002:[macosx] Arabic character cannot be rendered on MacOS X

[Phil Race]

Hi,
I don't know what versions of MacOS X either of you (Dmitry+Prasanta) 
used but I am going to guess they are different

(can you both report these, for the record).

In https://bugs.openjdk.java.net/browse/JDK-8147002, Prasanta enumerates
the cascade list for the Italic case and I don't see .NotoSansUniversal 
anywhere on that list.


So although if enumerated it likely still won't work (since it is not 
enumerated by the other APIs),
there does not appear to be any font in the cascade list reported by 
Prasanta that "would not work",

so the fix still seems fine to me as it is plugging a gap.

A fix for using "hidden" fonts in the cascade list seems like a JDK 10 
candidate unless there

is some critical case we are still missing.

-phil.



On 02/13/2017 01:56 AM, Dmitry Batrak wrote:

> Yes, we are aware this fixed it only for Oracle JDK.
> The bit I missed is that GeezaProInterface was then used on OpenJDK by JRS.
> So we could add that as well.
> If you have any other fonts you think should be added to the tail of that 
list we
> can add them too.

I ran JRSFontCreateFallbackFontForCharacters against all Unicode code 
points with Menlo-Italic as base font,
and got 63 possible fallback fonts - vs 29 present in cascade list, 
currently used by JDK 9 (given in JDK-8147002).
So the mismatch is quite large. But the interesting thing is that 
cascade list returned by
CTFontCopyDefaultCascadeListForLanguages still covers that whole 
Unicode range. The problem is that JDK
doesn't use that list fully. In particular, Arabic alef letter U+0627 
can be displayed using the font
.NotoSansUniversal from that list. This font doesn't get into 
composite font, as it's not found among the fonts
known to Java. So a better fix (and still quite small) will probably 
be using those 'hidden' fonts as part of
cascade composites - by creating a new, 'local' CFont instance if 
lookup in the known font list fails
(this, btw, seems to work fine for us). If even this is considered 
risky for JDK 9, then probably adding
.NotoSansUniversal with or instead of .GeezaProInterface as a 
hardcoded component makes sense.

Best regards,
Dmitry Batrak

On Fri, Feb 10, 2017 at 8:19 PM, Phil Race > wrote:


I think that so long as we are using JRS* in some places and
CoreText's cascading list
in others there is the likelihood of some inconsistency where JRS
introduces some magic
that we don't know about

So canDisplay() needs to be fixed to be consistent and perhaps we
also need to see
if we are always getting the glyphs from the same fonts for the
cases where there
is no missing glyph.

I do agree that getting away from JRS* support is the way to go so
we are in control.
However not at this time in JDK 9.

Yes "." fonts are not enumerated by OS X but if you can find them
they work.
This is how the UI fonts are handled.

Yes, we are aware this fixed it only for Oracle JDK.
The bit I missed is that GeezaProInterface was then used on
OpenJDK by JRS.
So we could add that as well.
If you have any other fonts you think should be added to the tail
of that list we
can add them too.

-phil.

On 02/10/2017 07:59 AM, Dmitry Batrak wrote:

Hi,

I've tried to build OpenJDK from 'client' repository with the
fix, and the
issue is still reproducible for me. Are you fixing it only for
Oracle JDK
(including Lucida Sans Regular font), and not for OpenJDK? The
problem is not
that some character cannot be rendered using a particular
font, it's that
rendering behaviour doesn't match what 'canDisplay' returns.

If you're interested I can share the way we've fixed this
issue in our
OpenJDK-based runtime.

As you've mentioned, the problem is that
JRSFontCreateFallbackFontForCharacters returns fallback font,
which is not on
the cascaded font list, and it can be absent even in the whole
list of fonts
reported by the system. On my machine, with OpenJDK, it's
.GeezaProInterface
(with name starting with dot). I guess such fonts are treated
by macOS as
'hidden' (just like files with names starting with dots),
still they seem to be
usable from application if requested by their name.

As JRSFontCreateFallbackFontForCharacters is a 'black box',
it's hard to make
sure cascade list we're building matches its internal logic,
so we've chosen to
build the list dynamically based on the output of
JRSFontCreateFallbackFontForCharacters itself. I can try to
prepare
corresponding webrev, if you think that can be useful.

Alternative solution (and, probably, a better one
strategically), would be not
using JRSFontCreateFallbackFontForCharacters 

[OpenJDK 2D-Dev] [9] Review request for 8175025: The copyright section in the test/java/awt/font/TextLayout/DiacriticsDrawingTest.java should be updated

[Dmitry Markov]

Hello,

Could you review a simple fix for jdk9, please?

bug: https://bugs.openjdk.java.net/browse/JDK-8175025
webrev: http://cr.openjdk.java.net/~dmarkov/8175025/webrev.00/

Changed the license in the test to GPL v2.

Thanks,
Dmitry

Re: [OpenJDK 2D-Dev] RFR: 8170552: [macosx] Wrong rendering of diacritics on macOS

[dmitry markov]

I have opened JDK-8175025 
 for that. Will send 
review soon.


Thanks,
Dmitry
On 15/02/2017 18:36, Philip Race wrote:

You are right !
I missed that. OpenJDK rules rquire it be contributed under GPL v2 
(not classpath)

The test needs to be updated or deleted ASAP (today would be good).

-phil.

On 2/15/17, 4:47 AM, Sergey Bylokhov wrote:
I am not sure, but Is the apache license in the test can be accepted 
in the openjdk? I had an impression that all code should by under GPL 
v2 or  GPL v2 + the Classpath exception.




+1

Vadim

On 11/02/2017 16:15, dmitry markov wrote:

Hi Dmitry,

The fix looks good to me too, but I am not a reviewer. So you need 
one more +1 from someone else with reviewer status.

I will sponsor integration of the fix once it is finally approved.

Thanks,
Dmitry
On 11/02/2017 02:48, Phil Race wrote:

Looks good to me.
It used to be a pain keeping several places in sync, hence the 
single util function.
I guess that pre-dated the Apple code and we didn't realise it had 
its own copy ..

So much code ... so little time ...

-phil.

On 02/10/2017 01:07 AM, Dmitry Batrak wrote:

Hello,

I'd like to propose a fix for JDK-8170552.
  bug: https://bugs.openjdk.java.net/browse/JDK-8170552
  webrev: 
http://cr.openjdk.java.net/~dbatrak/8170552/webrev.00/


I don't have a Committer status, so I'll require a sponsor.

I've proposed this patch earlier via support ticket (it's also 
attached to JIRA
issue). It brings the code used to detect complex text when 
working with system
fonts on macOS on par with similar code used for other fonts and 
platforms.


As compared to the previously provided patch, I've also added a 
test case. It
depends on specifics of the font being used (Menlo), but I 
couldn't think of

another way to create an automated test for this issue.

Best regards,
Dmitry Batrak

Re: [OpenJDK 2D-Dev] RFR: 8170552: [macosx] Wrong rendering of diacritics on macOS

[Philip Race]

You are right !
I missed that. OpenJDK rules rquire it be contributed under GPL v2 (not 
classpath)

The test needs to be updated or deleted ASAP (today would be good).

-phil.

On 2/15/17, 4:47 AM, Sergey Bylokhov wrote:

I am not sure, but Is the apache license in the test can be accepted in the 
openjdk? I had an impression that all code should by under GPL v2 or  GPL v2 + 
the Classpath exception.



+1

Vadim

On 11/02/2017 16:15, dmitry markov wrote:

Hi Dmitry,

The fix looks good to me too, but I am not a reviewer. So  you need one more +1 
from someone else with reviewer status.
I will sponsor integration of the fix once it is finally approved.

Thanks,
Dmitry
On 11/02/2017 02:48, Phil Race wrote:

Looks good to me.
It used to be a pain keeping several places in sync, hence the single util 
function.
I guess that pre-dated the Apple code and we didn't realise it had its own copy 
..
So much code ... so little time ...

-phil.

On 02/10/2017 01:07 AM, Dmitry Batrak wrote:

Hello,

I'd like to propose a fix for JDK-8170552.
  bug: https://bugs.openjdk.java.net/browse/JDK-8170552
  webrev: 
http://cr.openjdk.java.net/~dbatrak/8170552/webrev.00/

I don't have a Committer status, so I'll require a sponsor.

I've proposed this patch earlier via support ticket (it's also attached to JIRA
issue). It brings the code used to detect complex text when working with system
fonts on macOS on par with similar code used for other fonts and platforms.

As compared to the previously provided patch, I've also added a test case. It
depends on specifics of the font being used (Menlo), but I couldn't think of
another way to create an automated test for this issue.

Best regards,
Dmitry Batrak

Re: [OpenJDK 2D-Dev] RFR: 8170552: [macosx] Wrong rendering of diacritics on macOS

[Sergey Bylokhov]

I am not sure, but Is the apache license in the test can be accepted in the 
openjdk? I had an impression that all code should by under GPL v2 or  GPL v2 + 
the Classpath exception. 

> 
> 
> +1
> 
> Vadim
> 
> On 11/02/2017 16:15, dmitry markov wrote:
>> Hi Dmitry,
>> 
>> The fix looks good to me too, but I am not a reviewer. So  you need one more 
>> +1 from someone else with reviewer status.
>> I will sponsor integration of the fix once it is finally approved.
>> 
>> Thanks,
>> Dmitry
>> On 11/02/2017 02:48, Phil Race wrote:
>>> Looks good to me.
>>> It used to be a pain keeping several places in sync, hence the single util 
>>> function.
>>> I guess that pre-dated the Apple code and we didn't realise it had its own 
>>> copy ..
>>> So much code ... so little time ...
>>> 
>>> -phil.
>>> 
>>> On 02/10/2017 01:07 AM, Dmitry Batrak wrote:
 Hello,
 
 I'd like to propose a fix for JDK-8170552.
  bug: https://bugs.openjdk.java.net/browse/JDK-8170552
  webrev: http://cr.openjdk.java.net/~dbatrak/8170552/webrev.00/ 
 
 
 I don't have a Committer status, so I'll require a sponsor.
 
 I've proposed this patch earlier via support ticket (it's also attached to 
 JIRA
 issue). It brings the code used to detect complex text when working with 
 system
 fonts on macOS on par with similar code used for other fonts and platforms.
 
 As compared to the previously provided patch, I've also added a test case. 
 It
 depends on specifics of the font being used (Menlo), but I couldn't think 
 of
 another way to create an automated test for this issue.
 
 Best regards,
 Dmitry Batrak
>>> 
>> 
>