David,
I realise that in the case of memory-mapped data this is not needed (you
speak to one who has been single-stepping through the code, and modifying
it). I should have clarified my first message by explaining that I am
working with a platform where the data is stored in segmented form in Flash
memory that is subject to relocation, and thus, although the stream
interface works very fast, because it has access to the pointers to the data
segments, it cannot be dispensed with. I expect that this is a common
situation.
However, what I have done is essential for the platform I am working on, and
may well be essential for other low-RAM devices. In my current work we can
use large TrueType fonts (~2BM Japanese fonts, for example) successfully
with my changes; we cannot use them without my changes. Also, my CMAP code
is entirely non-invasive; it doesn't change the FreeType code or API at all
but is an optional add-on. (You call a function that gets the memory offset
of the charmap, stores it, then unloads the CMAP table; then you use a
replacement function instead of FT_Get_Char_Index.)
Note also that what I propose is optional; and can be tuned to affect CMAP,
LOCA, etc., tables above a certain size only; and if made a run-time option
rather than a compile-time option (I mentioned this in my first message),
can be made to work only for non-memory-mapped files.
I don't think making FT_Get_Char_Index use a stream interface would cripple
text layout performance. I have no benchmarking to back that assertion up
yet, I admit, but my experience when profiling is that TrueType rendering is
dominated by rasterization. The stream interface will often (as is the case
with my current work) have access to the data in the form of a series of
discontinuous memory blocks; that is, the file is memory-mapped but
segmented and thus requires a stream interface, but the overhead is
relatively low because the data is actually in memory.
Further, getting a glyph location from the LOCA table requires a only one
seek and two reads; there is no very good reason for copying the LOCA table
into RAM.
Here is a working and tested alternative version of tt_face_get_location; in
this code, the pointer face->glyph_locations has been replaced by an offset
face->glyph_locations_offset.
FT_LOCAL_DEF( FT_ULong )
tt_face_get_location( TT_Face face,
FT_UInt gindex,
FT_UInt *asize )
{
FT_ULong pos1, pos2;
FT_ULong p;
FT_ULong p_limit;
FT_Stream stream = face->root.stream;
FT_Error error = 0;
pos1 = pos2 = 0;
if ( gindex < face->num_locations )
{
if ( face->header.Index_To_Loc_Format != 0 )
{
p = face->glyph_locations_offset + gindex * 4;
p_limit = face->glyph_locations_offset + face->num_locations * 4;
error = FT_Stream_Seek(stream,p);
pos1 = FT_Stream_ReadLong(stream,&error);
pos2 = pos1;
if ( p + 4 <= p_limit )
pos2 = FT_Stream_ReadLong(stream,&error);
}
else
{
p = face->glyph_locations_offset + gindex * 2;
p_limit = face->glyph_locations_offset + face->num_locations * 2;
error = FT_Stream_Seek(stream,p);
pos1 = FT_Stream_ReadShort(stream,&error) & 0xFFFF;
pos2 = pos1;
if ( p + 2 <= p_limit )
pos2 = FT_Stream_ReadShort(stream,&error) & 0xFFFF;
pos1 <<= 1;
pos2 <<= 1;
}
}
/* It isn't mentioned explicitly that the `loca' table must be */
/* ordered, but implicitly it refers to the length of an entry */
/* as the difference between the current and the next position. */
/* Anyway, there do exist (malformed) fonts which don't obey */
/* this rule, so we are only able to provide an upper bound for */
/* the size. */
if ( pos2 >= pos1 )
*asize = (FT_UInt)( pos2 - pos1 );
else
*asize = (FT_UInt)( face->glyf_len - pos1 );
return pos1;
}
In conclusion I believe that what I have done is not only useful - in fact,
essential - to my current project but of potential use to FreeType users on
other devices with limited heap memory.
Best regards,
Graham
_____
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf
Of David Turner
Sent: 14 December 2007 14:17
To: Graham Asher
Cc: [email protected]
Subject: Re: [ft-devel] limiting the heap memory used by a TrueType font
Hi Graham,
first, keep in mind that the 'loca' table, as well as many other ones, are
simply never copied in the heap if the font file is already memory-mapped.
this happens if you use FT_New_Memory_Face, or even FT_New_Face if your
underlying stream implementation supports memory-mapped files (the Linux one
does). in these cases, all accesses are directly done from the original font
file memory. see the code in FT_Stream_ExtractFrame for details.
FT_Get_Char_Index is one of these functions that are heavily used by a
typical text rendering engine that uses FreeType, and modifying the
underlying cmap decoders to use a stream interface is certainly possible,
but is going to have a significant impact on text layout performance.
- David
2007/12/11, Graham Asher <[EMAIL PROTECTED]>:
TrueType fonts contain several tables that vary in a linear way with the
size of the font. Some of these, for example hmtx (horizontal metrics) are
not loaded into the heap, but are accessed using the stream interface.
Others, like cmap (which maps character codes to glyph indexes) and loca
(which maps glyph indexes to glyph data) are loaded in their entirety into
the heap.
The fact that some of these tables are loaded into the heap means that for
devices with a small amount of RAM there is always a font big enough to
cause an out-of-memory error; the amount of heap used is O(N) where N is the
number of glyphs. The font itself can be kept 'somewhere else' - in ROM, for
instance - so the size of the font data itself is not at issue. On small
devices RAM is typically in much shorter supply than ROM.
An ideal solution to the problem is to make it a configurable option for all
tables to be accessed via the stream interface and not loaded into the heap.
I have already experimentally done this for the cmap table and it works
perfectly, saving over 80K when loading a Japanese font. The technique is
this:
1. Allow the cmap table to be loaded as normal.
2. Note its location in the file, and any other information needed to access
it.
3. Unload the cmap table from the heap.
4. Replace FT_Get_Char_Index with a new function that reads the cmap via a
stream interface.
For the 'loca' table the only good way to prevent it from being loaded is to
unload it then override the normal way of getting glyph data, using the
incremental interface; however, it would be much nicer if there was a
compile-time or perhaps run-time option that forced stream access to all
variable-sized tables.
I have had time to do this only with rather platform-specific code which
unfortunately I can't show here because it is proprietary, but I just wanted
to give my opinion that this would be a useful new feature for FreeType. I
shall very probably not have time to do it myself, but someone more familiar
with FreeType internals might be able to code it up very quickly in a way
that fitted in with the existing architecture.
Best regards,
Graham Asher
_______________________________________________
Freetype-devel mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/freetype-devel
_______________________________________________
Freetype-devel mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/freetype-devel
