Tags

, , , , ,

A previous post discussed my motivation for having a dot matrix font that faithfully reproduced the look and feel of a simple character mode LCD display module. When working with small embedded systems and their displays, these modules aren’t the only low resolution dot based displays we encounter.

While it was easy enough to use the editor at FontStruct to create one font, it has some quirks and clumsy points that make it less ideal for my personal workflow. One thing that I’d like to see is the ability to import an existing definition of a font from a dump of its character generator ROM, or from source code that created the character images.

This post will be the first of several describing one way to achieve that goal, by creating a utility that will read a variety of textual descriptions of characters and produce a usable TrueType font.

Why TrueType?

One the things I do when starting a design for a user interface for a device with limited capabilities is to sketch it out with pencil and paper, with finished documents prepared in Word or Markdown, a general drawing tool such as Inkscape, or for a more complex UI a dedicated wireframing tool such as Balsamiq.

Most of those approaches can use a custom font, and TrueType (and its close cousin OpenType) are the most widely usable font format.

TrueType is an immensely powerful language for describing font faces, including the detailed shapes that make up each character, an ability to inject code into the font rendering process to improve its appearance at small font sizes, the ability to describe multiple mappings between the characters known to the font and character codes (ASCII, Unicode, ANSI with code pages, etc.) used by applications. All this power comes at the price of a very complex file format.

Fortunately, it is possible to ignore most of the advanced features and still get nice looking results when creating fonts which are not intended for typesetting dense running text at small sizes.

Creating TrueType

The actual binary format of a .ttf file may be quite complex, but there are several ways to make it more accessible to simple scripting. I chose the TTX tool because it provides for translation to and from an XML dialect, and XML is easy enough to generate when dealing with constrained data.

Since I intend to write most of the code in Lua, I needed to pick an XML library to hide most of the syntactic drudgery. I chose to use LuaXML, but any similar library that translates a hierarchy of Lua tables to (and from) XML will likely do. The XML produced by TTX happens to include a handful of comments hinting at some bits of the TTF tables that will be recalculated by TTX, but LuaXML makes no attempt to preserve comments and provides no means to generate them. This isn’t a problem since TTX (properly) does not care if the comments are present.

But Text is Bigger

Yes it is.

To see an example .ttx file, just use TTX to translate any .ttf you have laying around. Picking a small one will be friendlier, the version of Courier shipping with Windows 7 Pro has very complete Unicode coverage and translates to an 11MB .ttx file. The popular, pretty, and free Deja Vu Sans also has very complete Unicode coverage and also produced an 11MB .ttx file. Both fonts weigh in at only about 720KB in .ttf format.

Mitigating that is the fact that we don’t need to keep the TTX version of our fonts, it can be considered an intermediate build step and removed once a TTF is produced.

Other Tools

Most fonts are hand drawn with elaborate and powerful tools. The hinting programs that improve the font’s rendering when the number of available dots per em gets small are usually written and maintained using other (expensive) tools.

Since only a few major type foundries are earning a living from font design, the tools are very much a niche market and generally expensive. Fontographer is probably the leading tool if you have a budget.

The solid and stable free option is Fontforge which runs best on Linux but can be run on Mac and Windows.

These tools are great at the interactive graphical task of drafting individual letter shapes. The support reuse of shapes through copy and paste, as well as through the TrueType mechanism for making glyphs out of other glyphs in the same font. (Using the same shape for the acute accent and composing it with the same shape for all lower case vowels, for instance. Saves time, and makes it easier to keep all the letter forms harmonized.)

These tools are not intended to be used in an automated way, say as to transform a character generator ROM image into a collection of glyphs.

Dot Matrix Basics

The idea of making text out of similar bits that can be reused certainly goes back at least to the idea of movable type invented independently invented in China ca. 1040AD, Korea ca. 1377AD, and in Germany ca. 1450 AD by Gutenberg. But the replicated bits were whole words in China, whole syllables in Korea, and individual letters in the West. Thousands of individual pieces of type would be required to complete any significant work.

The idea of forming a single character out of an array of uniform parts is harder to trace. A 1908 patent exists on a segmented display that is remarkably similar in appearance to the seven segment LED displays found on lab instruments, clocks and wristwatches since the 1970s. While it is difficult to form more than a few letters from just seven segments, raising the segment count to fourteen or sixteen produces remarkably legible displays. Integrated displays of this sort are frequently found in places where a small amount of constrained text is needed, and typography is less important than large character size and visibility both day and night.

I have not identified the earliest dot matrix display, but the idea is already present in patent descriptions for printer technology from the late 1960s, and of course the idea of a raster scanned image goes back to at least the invention of the facsimile machine in the 1840s!

Most modern displays are technically dot matrices, but generally constructed at such a high spatial resolution that the individual dots are not seen separately. This sort of display is exactly what the hinting programs in a TrueType font are designed to improve, and are not otherwise interesting in this context.

However, small inexpensive embedded devices still have need for displays that only show a few characters, or which show a small graph along with some text, and which usually only provide a 1 bit per pixel representation.

Two rows of character cells each with an array of five by seven dots is one common form. Another is a small panel providing 64 rows of 92 to 128 dots. An extreme case is the display of a DMM, which might provide enough segments for four and a half digits or a pocket calculator providing perhaps seventy segments allowing ten digits to be displayed.

The more integrated displays provide RAM as a frame buffer either for storing character codes (usually an 8-bit extension of ASCII) or individual pixels.

Supporting Cast

Here’s some tools I’ve been using to analyze, characterize, and generally play with font files. Not all are necessary, but each has come in handy at one time or another:

  • BabelMap is a utility for browsing characters available in fonts installed on your system. It is much like the stock CharacterMap utility, but much more powerful.

  • FontStruct is a web based font design tool that allows for design of highly graphical fonts through placement of block elements on a grid. This design model matches well to the creation of dot matrix fonts.

  • fontTools is a library for manipulating fonts, written in Python. TTX is a font conversion tool based on the library that can convert a variety of fonts into an XML form, and convert the XML back into a font in a variety of formats.

  • FontUtils is a collection of font manipulation utilities written in Perl that includes fret, which draws nice reports showing detailed views of all glyphs in a font.

  • Fontforge is an open source graphical editor for individual letter forms (aka glyphs) and whole fonts. It can compile its font projects into TrueType among other formats.

When working with TrueType at this level, having documentation to refer to that can unambiguously answer detailed questions will be handy. Resources that were helpful include:

  • ISO/IEC 14496-22:2015(E) OpenType, standardized as Open Font Format. Since OTF shares many details with TTF, this covers many details of TrueType as well as OTF. The link is to the ISO/IEC directory of free standards, search that page for 14496-22 and collect everything that seems relevant.

  • TrueType according to Apple The TrueType Reference Manual from Apple. It covers most details of TTF, with strong recommendations for keeping Apple systems happy.

  • OpenType according to Microsoft All things OpenType, references, links, and recommendations from a Microsoft and Windows point of view..

How I Did It

I began by converting several well designed fonts from .ttf to .ttx to examine what features were used and how those features are represented in the .ttx file. Unfortunately, the documentation for TTX is rather vague about the details of what tables and fields are named, and what data types are expected. The safest approach seems to be to start with a file it produced and edit.

I am building up some Lua modules to manage the individual TTF tables that are needed in a working font. For a font based on TTF outlines, the required tables are cmap, glyf, head, hhea, hmtx, loca, maxp, name, postand OS/2. The first nine are mandatory for TrueType on any platform, and the last is effectively mandatory if the font will be used on Windows platforms, and strongly recommended in any case.

  • cmap maps from character code to the individual glyph. It allows any number of character codes to map to a single glyph. It also supports more than one mapping so that a single font file can be used with Unicode, ASCII, or any number of ANSI code page encodings.

  • glyf contains the shape of every glyph the font holds. Individual glyphs are described by a collection of spline curves called contours. A glyph can also be made by composing other glyphs together as with an accented letter. Glyphs are drawn in design units, with 256 to 4096 units spanning a typography em.

  • head contains general information about the font and the relationship between design units and typography units.

  • hheacontains information useful for fonts that are primarily used in horizontal layouts, such as the ascender height, descender, interline spacing, side bearings, and italic slope.

  • hmtx contains the advance width and left side bearing for each individual glyph.

  • loca contains an index into the glyf table that can and is entirely computed at run time by TTX so it is always present in a .ttx file, but is shown as empty.

  • maxp contains maximum values for a variety of measurements, intended to be useful to the font rendering engine to schedule the resources it will need to render glyphs from this font.

  • name contains all of the metadata about the font, especially the font’s name as known to the system and displayed to the user. It is a loose collection of tagged strings, each tagged by an id number, and information about the platform and character encoding it is stored in. Recommended practice appears to be to provide copies of these strings encoded for all encodings provided by the cmap table.

  • post contains information related to using the font in PostScript, including lists of the names of each glyph.

  • OS/2 contains summary details about the qualitative properties of the font which can be used to catalog fonts by similarity.

Most fields of these tables can either be well-chosen constants, or can be computed from descriptions of the font we are processing. For instance, a fixed width font has a very simple hmtx table where every entry is identical, and once we know that the font we are making is a 5×7 dot array with each dot on a 128 design unit spacing, the advance width will be exactly 640 units.

The name strings are likely a source of some trouble. Until forced to think about internationalization issues like translations of the license notice into French, Russian, and Japanese, they can be provided as part of the font design in Unicode and stored in Unicode and ASCII in the cmap table.

Foreshadowing

A public fossil repository will be created. Watch this space for an announcement of its location.

The code I have running today has created a TTF font from scratch, but it contained a single useful character, itself just a blob.

It is has framework in place for generating the font data a character at a time, but does not have any code that creates those character shapes. That is the obvious next step.

Before that, creating basic contours for a seven segment display and writing code that fills a block of the private use area with all 128 possible segment patterns would produce a font that does something recognizable without implementing any elaborate parsing of character descriptions.

Summary

I decided I needed a font that faithfully reproduced the look of a character mode LCD panel, and after failing to find one I created [my own][dots] using FontStruct.

Now I’ve decided to go further and make it possible to create fonts for any sized of fixed or variable pitch dot matrix font.

If you have a project involving embedded systems, micro-controllers, electronics design, audio, video, or more we can help. Check out our main site and call or email us with your needs. No project is too small!

+1 626 303-1602
Cheshire Engineering Corp.
710 S Myrtle Ave, #315
Monrovia, CA 91016

(Written with StackEdit.)

Advertisement