Re: How to Add Beams to Notes

[Richard Wordingham via Unicode]

On Fri, 5 May 2017 18:46:17 +
Michael Bear via Unicode  wrote:

> But
> if the cry for space gets REALLY desperate, I’ll merge identical
> glyphs into one glyph. Obviously, I won’t do this for more
> problematic merges, only glyphs in similar scripts with similar
> features. (e.g. I would represent Latin small letter o, Greek small
> letter omicron, Cyrillic small letter o, Armenian letter oh, and
> Georgian labial sign with one glyph, while Hebrew letter samekh and
> Arabic letter ae, despite also being circular, would be two separate
> glyphs.) But I’ll only do this if I really need to.

That could cause problems with extracting text from PDFs generated
using the font.

My interest was in whether a pan-BMP font was still possible.  As you
haven't done the counting (which is ill-defined for scripts with
conjuncts, and possibly even also for old Hangul support), you can't
tell me yet.

Richard.

Re: How to Add Beams to Notes

[James Kass via Unicode]

Philippe Verdy wrote,

> Code2000 ... uses the same font-wide strategy for hinting also
> creates lots of caveats: ...

Code2000 does not have hinting instructions; that's the font-wide strategy.

> Finally the bad thing about Code2000 is about font metrics, notably
> baselines: while you want to unify these baselines and line-heights,
> you'll reach the point where some scripts are ridiculously too small
> or improperly aligned ...

Do you have an example of either?  Is it possible that any improper
alignment or disproportionate glyphs in your display are being caused
by something other than the font?

> Trying tro fix these metrics for one script will break another one
> in that font ...

Trying to fix something which isn't broken is generally a bad plan.  I
wonder if the bizarre behavior you're reporting might have been caused
by some third party "fixing" something in the font.

In a pan-Unicode font, the base of the CJK ideographs wouldn't be
expected to match the baseline of alphabetic scripts.  Likewise, the
base of the stems used in Indic scripts shouldn't be expected to match
the baseline of alphabetic scripts as Indic scripts don't use
baselines.  Rather, the glyphs in such a font might be designed so
that, even with reasonable above and below marks/diacritics, there
would be no excessive line gaps generated for the other scripts
covered in the font.

A font which made, for example, Tibetan base letters the same size as
Latin letters would work just fine... as long as you don't mind that
runs of Latin text displayed with the font would appear to have two or
three line feeds inserted between each line.

Best regards,

James Kass

Re: How to Add Beams to Notes

[Hans Åberg via Unicode]

> On 1 May 2017, at 21:12, Michael Bear via Unicode  wrote:
> 
> I am trying to make a music notation font. It will use the Musical Symbols 
> block in Unicode (1D100-1D1FF), but, since that block has a bad rep for not 
> being very complete, I added some extra characters...

SMuFL has a rather comprehensive set of musical symbols.

http://www.smufl.org/
http://www.smufl.org/version/latest/
http://www.smufl.org/fonts/

RE: How to Add Beams to Notes

[Michael Bear via Unicode]

Additionally, I will only do OT features that are absolutely necessary for a 
certain script, not unnecessary (although stylish!) features, e.g. I will 
include things like mark positioning and init/medi/fina forms for Arabic, while 
leaving out small caps, swashes, and extensive ligatures. (In an earlier post, 
I might have said I’ll do ALL of the possible OT features. If so, I misspoke.)
But if the cry for space gets REALLY desperate, I’ll merge identical glyphs 
into one glyph. Obviously, I won’t do this for more problematic merges, only 
glyphs in similar scripts with similar features. (e.g. I would represent Latin 
small letter o, Greek small letter omicron, Cyrillic small letter o, Armenian 
letter oh, and Georgian labial sign with one glyph, while Hebrew letter samekh 
and Arabic letter ae, despite also being circular, would be two separate 
glyphs.) But I’ll only do this if I really need to.

Sent from Mail for Windows 10

Re: How to Add Beams to Notes

[Richard Wordingham via Unicode]

On Thu, 4 May 2017 23:13:08 +
Michael Bear via Unicode  wrote:

> I plan to do everything in the plane EXCEPT for the surrogates, which
> you’re not supposed to encode in fonts anyway, which leaves room for
> about 2,048 more glyphs for OpenType features.

There are, if I avoided double counting errors, 56,251 assigned
characters in the BMP in Unicode 10.0.0.  There are 1008 standardised
variation sequences, all in the BMP.  Indic scripts require more glyphs
than they have characters - usually at least twice as many. You have
the read chapter on Devanagari, haven't you?

Richard.

RE: How to Add Beams to Notes

[Michael Bear via Unicode]

“How much margin do you have for the BMP?  There are a fair few
variation sequences, on top of all the contextual forms and conjuncts.”

I plan to do everything in the plane EXCEPT for the surrogates, which you’re 
not supposed to encode in fonts anyway, which leaves room for about 2,048 more 
glyphs for OpenType features.

Sent from Mail for Windows 10

Re: How to Add Beams to Notes

[Philippe Verdy via Unicode]

You cannot cover a full plane with a single font. There are other factors
such as total size the also limit severely their use. We have to live with
the limitations of OpenType. In addition a giant font is hard to maintain,
version and update without breaking usages.

Font auhtors should focus their efforts on separatings scripts within a
collection a collection of related fonts (like what the Noto project did):
the rest will use font linking (which can be and already is used by
renderers, and can also be parametered by users for accessibility, or to
use prefered variants in some domains).

Also not all scripts have the same kinds of style variants
(serif/sans-serif, 2 or more distincive weights, straight/italic/oblique,
plain/hollow/shadowed), and trying to synthetize these styles will break
the nature of the script (notably for many symbols): you'll need separate
fonts for separate styles for specific scripts, other scripts may support
synthtic styles or not alter at all their rendering.

Code2000 is then just useful as a last resort font, but its glyphs are
still very poor compared to other fonts, and the fact it uses the same
font-wide strategy for hinting also creates lots of caveats: you cannot
hint Sinograms like Latin or Greek and symbols have their separate
requirements (notably geometric shapes and line drawing).

Finally the bad thing about Code2000 is about font metrics, notably
baselines: while you want to unify these baselines and line-heights, you'll
reach the point where some scripts are ridiculously too small or improperly
aligned: its much easier to separate them and tune these metrics
separately. Trying tro fix these metrics for one script will break another
one in that font, and finally you cannot create a comprehensive coverage
test and get stable results because there are contradicting objectives for
different uses: it's much easier to conciliate the possible choices by
separating scripts, so that you can more easily create additonal variants
for a few of them, and then create a separate rendering engine which will
use some parametered rules for selecting the most appropriate fonts. Adn
then it's much easier to update only one of these fonts when there are
improvements, without breaking all the rest.


2017-05-04 9:26 GMT+02:00 Richard Wordingham via Unicode <
unicode@unicode.org>:

> On Thu, 4 May 2017 05:01:17 +0200
> Philippe Verdy via Unicode  wrote:
>
> > Rendering Devanagari with OpenType does not require any PUA
> > assignment in that font for variants. The sequences are mapped
> > directly using subtables and the rules defined in OpenType for that
> > script. Fonts just use their own internal glyph ID's without having
> > to assign them any Unicode mapping, using Glyph processing rules.
>
> > Same remark about Arabic (though some encoded compatibility
> > characters will map to some of these glyphs... without using any PUA).
>
> The OP's plan is to use one font for the BMP, one font for the SMP, and
> one font for the rest.  However, the BMP font Code2000, which only
> goes, incompletely, up to Unicode 5.2, uses 63,546 glyphs, which is
> very close to the limit of 65,535.  There is the slight margin that it
> included a few small scripts with standardised (ConScript
> Unicode Registry) PUA allocations.
>
> Richard.
>

Re: How to Add Beams to Notes

[Richard Wordingham via Unicode]

On Thu, 4 May 2017 05:01:17 +0200
Philippe Verdy via Unicode  wrote:

> Rendering Devanagari with OpenType does not require any PUA
> assignment in that font for variants. The sequences are mapped
> directly using subtables and the rules defined in OpenType for that
> script. Fonts just use their own internal glyph ID's without having
> to assign them any Unicode mapping, using Glyph processing rules.

> Same remark about Arabic (though some encoded compatibility
> characters will map to some of these glyphs... without using any PUA).

The OP's plan is to use one font for the BMP, one font for the SMP, and
one font for the rest.  However, the BMP font Code2000, which only
goes, incompletely, up to Unicode 5.2, uses 63,546 glyphs, which is
very close to the limit of 65,535.  There is the slight margin that it
included a few small scripts with standardised (ConScript
Unicode Registry) PUA allocations.

Richard.

Re: How to Add Beams to Notes

[Philippe Verdy via Unicode]

2017-05-03 9:49 GMT+02:00 Richard Wordingham via Unicode <
unicode@unicode.org>:

> On Tue, 2 May 2017 05:08:27 +0200
> Philippe Verdy via Unicode  wrote:
>
> > Consider also that the BMP is almost full, the remaining few holes
> > are kept for isolated characters that may be added to existing
> > scripts, or permanently reserved to avoid clashes with legacy
> > softwares using simple code remappings between distinct blocks, or to
> > perform simple case conversions (e.g. in Greek) for internal purposes
> > (these positions are not interoperable and may clash with future
> > versions of the UCS and I18n tools/libraries like ICU)
> >
> > You should abstain using any currently unassigned positions in the
> > existing Unicode blocks: use PUA if you have nothing else; there are
> > plenty of space available, in the BMP (most common usage in fonts
> > that need to map additional glyphs) or in the two last planes.
>
> It isn't codepoints that is the constraint; one must consider the
> number of glyphs without dedicated one-character codes.
>

 Glyph processing use requires internal glyph ids in fonts. The limit is on
the total number of glyphs you can put it that font without exceeding the
maximum size of glyph id's. Traditionally this is solved by creating
coherent (but complete enough) subsets so that all glyphs within the same
script can fit. The other solution, nobaly for sinograms, is to use font
linking

The Arabic script (and other cursively connected scripts) has similar
> expansions, even if one goes for a typewritten style.
>
> Devanagari explodes when one considers just the conjuncts prescribed for
> Hindi.
>

Rendering Devanagari with OpenType does not require any PUA assignment in
that font for variants. The sequences are mapped directly using subtables
and the rules defined in OpenType for that script. Fonts just use their own
internal glyph ID's without having to assign them any Unicode mapping,
using Glyph processing rules.

Same remark about Arabic (though some encoded compatibility characters will
map to some of these glyphs... without using any PUA).

>
> I think it's also necessary to avoid splitting likely grapheme
> clusters between fonts. Which of the three fonts will support U+1F3F4
> U+E0067 U+E0062 U+E0065 U+E006E U+E0067 U+E007F (English flag) and
> which U+261D U+1F3FF (index pointing up: dark skin tone)?
>
> Now, the BMP has headroom provided by the surrogate characters and the
> PUA, which will not have mappings, but I'm not sure that it's enough.
>
>
For your question, the solution is to create corent subsets of symbols and
create fonts from this subset. For the  case of country/region flags, they
could all be separated in a specific font. As well you can create separate
fonts for persons/animals/plants, and another one for unanimated objects
(including planets, game pieces...) Traditional punctuation-like symbols
used in typography and normally without any emoji style can fit a generic
symbols fonts (along with geométric shapes, line drawing symbols).

Re: English flag (from Re: How to Add Beams to Notes)

[William_J_G Overington via Unicode]

Ken Whistler wrote:

> I suggest the following:

> 10BEDE for an English flag (reminding one of Bede the Venerable)
> 10CADF for a Welsh flag (harking to Cadfan ap Iago, King of Gwynedd)
> 10A1BA for a Scottish flag (for Alba, of course)

> Surely those would work for you!

Thank you for your reply. Nicely! Those code points each have a helpful 
mnemonic.

I had not known of Cadfan ap Iago until I read your post.

I found the following.

https://en.wikipedia.org/wiki/Cadfan_ap_Iago

I opine that we need to make it clear, for the benefit of some people new to 
Unicode who may be reading this thread, that those code points are in one of 
the Private Use Areas, namely Supplementary Private Use Area-B, so there could 
be problems using them in some circumstances due to lack of uniqueness in the 
use of those code points.

http://www.unicode.org/charts/PDF/U10.pdf

William Overington

Wednesday 3 May 2017

Re: English flag (from Re: How to Add Beams to Notes)

[Ken Whistler via Unicode]

On 5/3/2017 3:20 AM, William_J_G Overington via Unicode wrote:

Surely a single code point could be found. Single code points are being found 
for various emoji items on a continuing basis. Why pull up the ladder on 
encoding some flags each with a single code point?

Yes, a single code point for an English flag please. And one for a Welsh flag 
too please. And one for a Scottish flag too please. And some others please, if 
that is what end users want.


I suggest the following:

10BEDE for an English flag (reminding one of Bede the Venerable)
10CADF for a Welsh flag (harking to Cadfan ap Iago, King of Gwynedd)
10A1BA for a Scottish flag (for Alba, of course)

Surely those would work for you!

--Ken

Re: English flag (from Re: How to Add Beams to Notes)

[David Faulks via Unicode]

I think the UTC just wants to avoid the controversy and blame of deciding which flags are worthy of inclusion.However, I think that emoji users who are aware of Unicode will blame Unicode for whatever the vendors pickDavid Faulks

English flag (from Re: How to Add Beams to Notes)

[William_J_G Overington via Unicode]

Richard Wordingham wrote:

 U+1F3F4 U+E0067 U+E0062 U+E0065 U+E006E U+E0067 U+E007F (English flag) 

I looked at that and I realized that although I had effectively seen that 
encoding in http://www.unicode.org/reports/tr51/tr51-11.html though expressed 
differently, it was only when I saw it expressed as above that I realized that 
there is something gone wrong with encoding policy.

There are at present ten totally unused planes in the Unicode code point map 
and yet that seven character sequence is needed for encoding an English flag.

Surely a single code point could be found. Single code points are being found 
for various emoji items on a continuing basis. Why pull up the ladder on 
encoding some flags each with a single code point?

Yes, a single code point for an English flag please. And one for a Welsh flag 
too please. And one for a Scottish flag too please. And some others please, if 
that is what end users want.

William Overington

Wednesday 3 May 2017

Re: How to Add Beams to Notes

[Richard Wordingham via Unicode]

On Tue, 2 May 2017 05:08:27 +0200
Philippe Verdy via Unicode  wrote:

> Consider also that the BMP is almost full, the remaining few holes
> are kept for isolated characters that may be added to existing
> scripts, or permanently reserved to avoid clashes with legacy
> softwares using simple code remappings between distinct blocks, or to
> perform simple case conversions (e.g. in Greek) for internal purposes
> (these positions are not interoperable and may clash with future
> versions of the UCS and I18n tools/libraries like ICU)
> 
> You should abstain using any currently unassigned positions in the
> existing Unicode blocks: use PUA if you have nothing else; there are
> plenty of space available, in the BMP (most common usage in fonts
> that need to map additional glyphs) or in the two last planes.

It isn't codepoints that is the constraint; one must consider the
number of glyphs without dedicated one-character codes.  For example,
U+1000 MYANMAR LETTER KA needs glyphs for:

1000
1000 FE00
1039 1000 (and probably at two different widths)
1039 1000 FE00 (do.)

There are a few CJK ideographs with similar needs:

537F
537F FE00 (= CJK COMPATIBILITY IDEOGRAPH-2F831)
537F FE01 (= CJK COMPATIBILITY IDEOGRAPH-2F832)
537F FE02 (= CJK COMPATIBILITY IDEOGRAPH-2F833)

There's also the Japanese ideographic variation sequence , which should probably have its own glyph even if it's the
same as one of the above. 

The Arabic script (and other cursively connected scripts) has similar
expansions, even if one goes for a typewritten style.

Devanagari explodes when one considers just the conjuncts prescribed for
Hindi.

I think it's also necessary to avoid splitting likely grapheme
clusters between fonts. Which of the three fonts will support U+1F3F4
U+E0067 U+E0062 U+E0065 U+E006E U+E0067 U+E007F (English flag) and
which U+261D U+1F3FF (index pointing up: dark skin tone)?

Now, the BMP has headroom provided by the surrogate characters and the
PUA, which will not have mappings, but I'm not sure that it's enough.
That's why I asked the question.

Richard.

Re: How to Add Beams to Notes

[Philippe Verdy via Unicode]

Consider also that the BMP is almost full, the remaining few holes are kept
for isolated characters that may be added to existing scripts, or
permanently reserved to avoid clashes with legacy softwares using simple
code remappings between distinct blocks, or to perform simple case
conversions (e.g. in Greek) for internal purposes (these positions are not
interoperable and may clash with future versions of the UCS and I18n
tools/libraries like ICU)

You should abstain using any currently unassigned positions in the existing
Unicode blocks: use PUA if you have nothing else; there are plenty of space
available, in the BMP (most common usage in fonts that need to map
additional glyphs) or in the two last planes. The PUA block in the BMP is
large enough for most apps and almost all fonts that need private glyphs
for internal purposes, or for still unencoded characters or for your own
encoded variants such as slanted symbols, rotated symbols, inverted
symbols, or symbols with multiple sizes, or at different positions on the
musical score, or using distinct styles (e.g. between different players or
singers, or various symbols for percusive instruments or specific
intruments, or extra annotations).

Many new symbols have been encoded first as PUAs in early fonts used to
create proposals (then rendered to a PDF, or embedded fonts in a rich text
document, or webfonts loaded from static versioned URLs on a repository
like GitHub or on a public cloud). Later the proposal passed the early
steps for reviewing the repertoire and choosing more relevant positions,
then characters were encoded and standardized and these fonts were updated
to map their glyphs to not just their existing PUAs but also the nex
standard positions (or encoded variants)

2017-05-02 2:27 GMT+02:00 Richard Wordingham via Unicode <
unicode@unicode.org>:

> On Mon, 1 May 2017 23:03:53 +
> Michael Bear via Unicode  wrote:
>
> > “Rather than using "unused code positions", I would always recommend
> > to use some of the Private Use code points.” Consider it done.
> >
> > “What is the intended usage of your font? Music score
> > applications? others?” I am simply going to make a series of full
> > Unicode fonts (which, due to the 65,535-character limit in fonts,
> > each of the 3 fonts covers different planes: The first font does the
> > BMP,
>
> How much margin do you have for the BMP?  There are a fair few
> variation sequences, on top of all the contextual forms and conjuncts.
>

Re: How to Add Beams to Notes

[Richard Wordingham via Unicode]

On Mon, 1 May 2017 23:03:53 +
Michael Bear via Unicode  wrote:

> “Rather than using "unused code positions", I would always recommend
> to use some of the Private Use code points.” Consider it done.
> 
> “What is the intended usage of your font? Music score
> applications? others?” I am simply going to make a series of full
> Unicode fonts (which, due to the 65,535-character limit in fonts,
> each of the 3 fonts covers different planes: The first font does the
> BMP,

How much margin do you have for the BMP?  There are a fair few
variation sequences, on top of all the contextual forms and conjuncts.

Richard.

Re: How to Add Beams to Notes

[John W Kennedy via Unicode]

> On May 1, 2017, at 3:12 PM, Michael Bear via Unicode  
> wrote:
> 
> I am trying to make a music notation font. It will use the Musical Symbols 
> block in Unicode (1D100-1D1FF), but, since that block has a bad rep for not 
> being very complete, I added some extra characters in the unmapped positions 
> of that block (e.g. U+1D127 inverts the stem of the previous note, U+1D1E9 is 
> a ledger line, U+1D1EA is the "TAB" clef, U+1D1F0-U+1D1FC position the note 
> along the staff, etc.) I've had no problem so far, but now I need to do 
> beamed notes. The Unicode block has control characters for beginning and 
> ending a series of beamed notes (U+1D173 and U+1D174, respectively), but I'm 
> not really sure how to add beams to the notes while keeping the pitch intact. 
> I know I'll obviously need OpenType for this. Slanted beams would be 
> preferred, but straight beams are acceptable. It will need to support beams 
> added on for longer notes. Can someone help me with this?
>  
> I had asked this on a High Logic Font Creator forum (here), and someone said 
> to subscribe to your mailing list and ask you guys. So here I am! Anyway, 
> help, please?

You might want to acquaint yourself with http://www.smufl.org

RE: How to Add Beams to Notes

[Michael Bear via Unicode]

“Rather than using "unused code positions", I would always recommend to use 
some of the Private Use code points.”
Consider it done.

“What is the intended usage of your font? Music score 
applications? others?”
I am simply going to make a series of full Unicode fonts (which, due to the 
65,535-character limit in fonts, each of the 3 fonts covers different planes: 
The first font does the BMP, the second one does the SMP, and the third one is 
all the other planes, which are vacant enough to fit in one font) that will 
have the necessary OpenType features of every script. And I thought “Hey, maybe 
I should do full OT for the music block that no one has really done yet! How 
awesome would that be?” So I made a test font to work it out, but I ran into 
this one pothole. That’s when I came here.

Sent from Mail for Windows 10

Re: How to Add Beams to Notes

[Stötzner Signographie via Unicode]

Bad news, I’m afraid.
What is the intended usage of your font? Music score applications? others? The 
overall problem with musical notation is, there is no comprehensive character 
encoding standard and no generally working text and layout composing method 
established. In the light of that fact it is hopeless to make fonts for this. 
The fonts are not the problem (yes they are, there is no solid encoding scheme 
available), but the lack of composing syntax is the crux you’ll hardly overcome.
If you need to cater for a specific usage scenario you’ll end up with a 
complete hack anyway (however it may look like, doesn’t matter). 
Good luck!

A. Stötzner
(Musical notation project)


Am 01.05.2017 um 21:12 schrieb Michael Bear via Unicode:

> I am trying to make a music notation font. It will use the Musical Symbols 
> block in Unicode (1D100-1D1FF), but, since that block has a bad rep for not 
> being very complete, I added some extra characters in the unmapped positions 
> of that block (e.g. U+1D127 inverts the stem of the previous note, U+1D1E9 is 
> a ledger line, U+1D1EA is the "TAB" clef, U+1D1F0-U+1D1FC position the note 
> along the staff, etc.) I've had no problem so far, but now I need to do 
> beamed notes. The Unicode block has control characters for beginning and 
> ending a series of beamed notes (U+1D173 and U+1D174, respectively), but I'm 
> not really sure how to add beams to the notes while keeping the pitch intact. 
> I know I'll obviously need OpenType for this. Slanted beams would be 
> preferred, but straight beams are acceptable. It will need to support beams 
> added on for longer notes. Can someone help me with this?
>  
> I had asked this on a High Logic Font Creator forum (here), and someone said 
> to subscribe to your mailing list and ask you guys. So here I am! Anyway, 
> help, please?
>  
> Sent from Mail for Windows 10
>  

___

Andreas Stötzner  Gestaltung Signographie Fontentwicklung
 
Haus des Buches 
Gerichtsweg 28, Raum 434
04103 Leipzig
0176-86823396

Re: How to Add Beams to Notes

[Asmus Freytag via Unicode]

  
  
On 5/1/2017 12:12 PM, Michael Bear via
  Unicode wrote:


  
  
  
  
I am trying to make a music notation font.
  It will use the Musical Symbols block in Unicode
  (1D100-1D1FF), but, since that block has a bad rep for not
  being very complete, I added some extra characters in the
  unmapped positions of that block (e.g. U+1D127 inverts the
  stem of the previous note, U+1D1E9 is a ledger line, U+1D1EA
  is the "TAB" clef, U+1D1F0-U+1D1FC position the note along the
  staff, etc.) I've had no problem so far, but now I need to do
  beamed notes. The Unicode block has control characters for
  beginning and ending a series of beamed notes (U+1D173 and
  U+1D174, respectively), but I'm not really sure how to add
  beams to the notes while keeping the pitch intact. I know I'll
  obviously need OpenType for this. Slanted beams would be
  preferred, but straight beams are acceptable. It will need to
  support beams added on for longer notes. Can someone help me
  with this?
 
I had asked this on a High Logic Font
  Creator forum (here),
  and someone said to subscribe to your mailing list and ask you
  guys. So here I am! Anyway, help, please?
 
Sent from 
Mail for Windows 10
 
  

Just an aside: rather than using "unused
code positions", I would always recommend to use some of the Private
Use code points. That way, whatever you work on would remain usable
when Unicode inevitably adds some characters that would fill
those "unused" positions.
A./