Definitions: statement delimiter: a new statement will always follow a linebreak character: a) unless the following character is the continuation character b) if the following character is another linebreak character, it is a null statement. c) statements consisting entirely of whitespace characters, comments or combinations are considered null statements. d) the parser ignores null statements. e) parser eats everything up to linebreak character and each subsequent line that begins with the continuation char in the event of a parsing error. The Construct delimiters are a statement in and of themselves and they also serve to terminate the previous statement and delimit the start of the following statement. you may think of a as equivalent to Whitespace characters: space, TAB Linebreak characters: , if both occur consecutively the pair is treated as one linebreak. White characters: either Whitespace or Linebreak characters. continuation character: + NEW DEF: when used as a continuation char the '+' must appear as the first char in a line. The parser will interpret the preceeding linebreak as whitespace. Logical linebreak: a linebreak character that is NOT followed by a continuation character. Continuation linebreak: a linebreak character followed by a continuation character. Arbitrary whitespace: Whitespace chars or comments or Continuation Linebreak Construct delimiters: curly braces { } A linebreak character is not required to preceed or proceed a construct delimiter. You may think of a as logically equivalent to Construct delimiters also delimit the scope of macro definitions. If a macro was defined within a nesting level created by a pair of construct delimiters, it remains defined only within that nesting level. Nesting level: the logical set of statements enclosed by a matching pair of construct delimiters. Statement delimiter: Logical linebreak or Construct delimiter. Keyword - value separator: colon : string delimiter: double quotes "" macro invocation: equals = comment indicator: *% Comments may be inserted immediately preceeding any logical or continuation linebreak. They may contain any characters except linebreak characters. The linebreak character terminates the comment. The comment indicator must be preceeded by a White Character. (unless the comment indicator is the first byte in the source file.) Grouping operator: () used in certain value constructs. Hex substring delimiter: <> Escape character : % used in string and parameter constructs. The above characters have reserved meanings and may not be used in any keyword, symbol name or any user defined name. Name spaces: If attributes keyword are used within other constructs say feature or global keyword inside an option, that keyword must be declared using the EXTERN_FEATURE: or EXTERN_GLOBAL: modifier. Otherwise the attribute type expected (dictionary used) will be defined by the state. the namespaces of the attribute keywords for various constructs may overlap each other, since we rely on the above rules to assign the namespace, but they cannot overlap non-attribute keywords. Keywords: predefined keywords must begin with '*' the remainder of the keyword may be comprised of 'A' to 'Z', 'a' to 'z', '0' - '9', '_' and may be terminated by an optional '?'. Symbol Keywords do not begin with '*' and may be any name defined by the user, they must be comprised of the same characters as normal keywords. Symbol Keywords are used as the name of Value macros and Font names in certain constructs. Parsing rules: Values: on some Keywords the Value is ignored by the parser. In these cases the Value (and the : delimiter) may be omitted for example: *Macros *Macros: PaperNames are both valid. Block Macro definitions: if the definition (body of a BlockMacro) contains braces, the braces must appear in pairs and the correct order. ie { must appear before }. braces may be nested within the body. Parsing Level 0: this is the outermost level of parsing. At this level the characters {, } are interpreted as construct delimiters, *% begins a comment etc. This is contrasted to the parsing rules applied to higher level objects like strings where these charcters have no special meaning. Linebreak characters may only appear in parsing level 0. Their appearance at any time terminates parsing of the current statement. parser eats everything up to linebreak character in the event of a parsing error. Statements begin with either a * or a where the keyword is a parser recognized keyword token and where is a parser unrecognized token which may represent a ValueMacroName or a TTFontName. such tokens must not begin with '*'. and will be marked as SYMBOL in the TokenMap. In general arbitary Whitespaces may appear between any entities recognized at Parsing level 0. If Whitespaces are permitted within such entities it will noted. For added robustness, macro strings will not be expanded to their binary equivalents. This prevents the insertion of random Linebreak characters in the stream. Rule for GPD authors: for maximum robustness and error recovery, place level 0 braces (Construct delimiters) on a separate line. Do not place Construct delimiters on the same line where questionable keywords and constructs are used. Heap useage: the heap will be divided into several sections, each large enough to hold whatever may come. Growth of the heap sections is not allowed. Strings, composite objects like RECTS: holds all strings, offsets referenced from beginning of string section. Arrays of various types: each type of array is assigned its own dedicated memory buffer. A master table contains pointers to each array, its size and current entry. Once data is entered into the array, the keeper of the data need only remember the index of the array the data was written into. After all parsing operations are complete, we will consolidate the arrays into one memory space and update the master table accordingly. Some composite values require an indefinite amount of storage or reside in dedicated structures. (for example strings, lists and UIconstraints). Such values are stored in 2 parts, a fixed sized link, and and the part the link refers to. This part may be variable sized and may occupy heap space or one or more dedicated structures or some combination thereof. Since the link is always of a known size, it may be stored in a field in a structure etc. The following table lists the values supported by the parser and how they are structured: value type: Strings link: ARRAYREF dwOffset field specifies heap offset of start of string dwCount field specifies string length excluding terminating NULL. body: Null terminated array of bytes stored in heap. value type: LIST value type: QualifiedName Shortcuts that cause headaches: *Command: 2 forms exist. Macros: a macroDefinition cannot be self-referencing macroDefintions cannot be forward referenced. ie only a previously defined and fully resolved macro can be referenced. scope: an Macro is defined (referenceable) only after parsing the closing brace of its definition and until encoutering a closing brace that signifies the termination of the level the macro was defined in. namespaces: since macro definitions are stored in a stack, defining a second macro with the same name does not necessarily destroy the first definition. If The first macro was defined outside of the scope of the 2nd, it will be visible once the parser leaves the scope of the 2nd Macro. ValueMacros: Only string ValueMacros may be nested. That if any valueMacro definition references another valueMacro, the parser will assume the definition is a stringMacro, and the macro being referenced is also a stringvalue. BlockMacros: a blockMacro may contain other Macrodefinitions but those definitions can only be referenced inside the block macro. They will not appear when the blockmacro is actually referenced. A BlockMacroName may NOT be substituted by a ValueMacro either in a *BlockMacro or *InsertBlock statement. ----- more parsing rules ------ the first non-null line of the root GPD sourcefile must be: *GPDSpecVersion: arbitary whitespace is allowed between tokens comprising a command parameter. arbitary whitespace is allowed anywhere within a hex substring. -------------------------------------------------------------- Currently, these are the known types of keywords: CONSTRUCTS: introduces a construct (causes a parser context change) usually followed by open brace in next statement. construct is terminated by matching close brace. *UIGroup, *Switch, *Case, *Default, *Command *FontCartridge, *TTFontSubs, *Feature, *Option *OEM, *BlockMacro, *Macros a construct may be thought of as a type of structure initialization. ONly certain keywords are may be used inside of a construct. Some of these keywords may only be using within their associated construct and no where else. LOCAL ATTRIBUTES: initializes a value in a construct. GLOBAL ATTRIBUTES: initializes a value in the global structure. local and global attributes may be subdivided into freefloating and fixed. A fixed attribute must be used in the same nesting level as the construct it is associated with. A freefloating attribute may be used within another construct as long as that construct is contained within the construct associated attribute. SPECIAL ATTRIBUTE: initializes and adds another item to a dedicated or global list or a list in construct. or has side effects requiring special processing. examples: *Installable?, - Causes an installable feature to be synthesized. but parser may deal with this after all Feature/Options have been parsed. So not really. Adds link to special tree structure: *Constraints, *InvalidCombination, *InvalidInstallableCombination, *InstalledConstraints, *NotInstalledConstraints The values introduced by these keywords are additive (like using a LIST): *Font *Command:: a shorthand *MemConfigKB a shorthand way of creating an entire memory option. LIST(,,) may be written as: .LIST(,,) if there are other types of keywords let me know. Special Parsing contexts: in which User defines new keywords simply by referencing them. *TTFontSubs: { : .... not actually a symbol, but adds a string, number pair to a list. May be implemented during construction as a symbol. } *Macros: { : } *FontCart: note the FontCart construct is ROOT_ONLY and is not multivalued. Each construct with a unique SymbolName corresponds to a dedicated FONTCART structure. If we want to make FontCarts multivalued, we introduce a new keyword *AvailFontCarts: LIST(symbol1, symbol2, symbol3) which is a FreeFloating Global. MacroProcessing: = where a value or component of a string is expected = following a symbolname following a construct Keyword. *InsertBlock: recognized value types: ORDER :==
. SYMBOLS: Any user defined (not recognized by the parser) token used to identify a statement or construct or value. are not symbols because the parser has a list of recognized Valid Unidrv commands. Non Macro Symbols may be forward referenced: ie *DefaultOption or *Constraints may reference a symbol that is defined later. where defined: Associated Keyword: *Macros: not the Group Name! *BlockMacro: *Feature: *Option: *OEM: saved in symbol tree for possible future use. *TTFontSubs: TTFontnames may be stored as symbols, but are not symbols in the strictest sense. constructs not using symbols: *TTFontSubs: predefined. *UIGroup: optional - not used by parser. *Default: optional - not used by parser. *Command: predefined. CmdSelect is a special name which triggers special processing. *FontCartridge: optional - not used by parser. Implementation hint: use macros to keep all definitions in one place. or introduce *AvailFontCart: LIST(, ) inside constructs. where referenced: *InsertMacro: *: = *: = except *BlockMacro, *InsertMacro, *Include *Switch: *Case: Currently the parser saves symbols defined in *Feature and *Option keywords and remembers symbol references made in *Switch and *Case keywords. The include keyword: must not appear within a macrodefinition must not reference a macrovalue must be terminated by a linebreak not { or } construct. --- state machine ---- The parser treates construct keywords as operators which change the state of the parser. (create state transitions.) the set of allowed transitions is defined in the table AllowedTransitions this table enforces several rules: the construct _TTFONTSUBS can only appear at the root level. no constructs may appear within OEM, FONTCART, TTFONTSUBS, COMMAND constructs. The following code fragment is a comprehensive list of the allowed state transitions: pst = astAllowedTransitions[STATE_ROOT] ; pst[CONSTRUCT_UIGROUP] = STATE_UIGROUP; pst[CONSTRUCT_FEATURE] = STATE_FEATURE; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_ROOT; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_FONTCART] = STATE_FONTCART; pst[CONSTRUCT_TTFONTSUBS] = STATE_TTFONTSUBS; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_UIGROUP] ; pst[CONSTRUCT_UIGROUP] = STATE_UIGROUP; pst[CONSTRUCT_FEATURE] = STATE_FEATURE; pst = astAllowedTransitions[STATE_FEATURE] ; pst[CONSTRUCT_OPTION] = STATE_OPTIONS; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_FEATURE; pst = astAllowedTransitions[STATE_OPTIONS] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_OPTION; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_SWITCH_ROOT] ; pst[CONSTRUCT_CASE] = STATE_CASE_ROOT; pst[CONSTRUCT_DEFAULT] = STATE_DEFAULT_ROOT; pst = astAllowedTransitions[STATE_SWITCH_FEATURE] ; pst[CONSTRUCT_CASE] = STATE_CASE_FEATURE; pst[CONSTRUCT_DEFAULT] = STATE_DEFAULT_FEATURE; pst = astAllowedTransitions[STATE_SWITCH_OPTION] ; pst[CONSTRUCT_CASE] = STATE_CASE_OPTION; pst[CONSTRUCT_DEFAULT] = STATE_DEFAULT_OPTION; pst = astAllowedTransitions[STATE_CASE_ROOT] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_ROOT; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_DEFAULT_ROOT] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_ROOT; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_CASE_FEATURE] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_FEATURE; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_DEFAULT_FEATURE] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_FEATURE; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_CASE_OPTION] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_OPTION; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; pst = astAllowedTransitions[STATE_DEFAULT_OPTION] ; pst[CONSTRUCT_SWITCH] = STATE_SWITCH_OPTION; pst[CONSTRUCT_COMMAND] = STATE_COMMAND; pst[CONSTRUCT_OEM] = STATE_OEM; --- multiple statements and redefinitions: ------ for standard attributes, if two statements containing that attribute with different values appears in the same construct, the attribute takes the latter occuring value. If the attribute is defined to be FreeFloating, it may appear multiple times in different *Option or *Case constructs. In this case if the effect of the multiple occurances is to add new branches which are compatible with the existing tree, or to reinitialize the value of a node in the existing tree that is an accepted use of multiple occuring attributes. However if the effect is to define a new branch which is incompatible with the existing tree, that is an error, and the latter initialization of the attribute is ignored. There is one exception to the rule of adding conflicting branches to the attribute tree. That exception allows default initializers to be created. If an attribute is assigned a value which is subsequently made multivalued, the initial value becomes the default initializer unless the GPD author explicitly specified a 'default' case when making the attribute multivalued. Note the order cannot be reversed. An attribute which is already defined to be multivalued cannot subsequently be defined to be fewer valued. --- state machine ---- the set of allowed transitions is defined in the table AllowedTransitions this table enforces several rules: the construct _TTFONTSUBS can only appear at the root level. no constructs may appear within OEM, FONTCART, TTFONTSUBS, COMMAND constructs. ---- use of switch/case constructs ----- The same feature must not be referenced in nested constructs. This will produce an attribute tree that contains the same feature at two different levels. similarly... an attribute tree should not be constructed piecemeal. It is an error if the tree is subsequently redefined/elaborated using a different feature nesting order. ----- Severity of errors: !!!!!: parser is non-compilable/non-functional unless this is resolved. !!!!: unfinished functionality. Some legal GPD files will cause corruption. !!!: integrity check omitted - a corrupt file may be inadvertantly generated if resource limitations are encountered. !!: syntax error in GPD may cause widespread corruption !: emit useful message for user. BUG_BUG: wish item - user friendlier error message etc. parser self-consistency check, self diagnostics. more general, elegant, faster, more complex code etc. Note: PARANOID BUG_BUGs indicate error conditions that are the result of coding errors (mistaken assumptions, incomplete code paths etc) and are not the result of improper GPD syntax, or resource constraints (overflow of fixed length buffers etc). All originating error messages should report the name of the function, name of variable or system call that is out of range or invalid. Later, if a caller function sees a failure return value, it may want to tack on an extra message say keyword or line number where error occured. A if a function returns with a failure condition, the caller may at its discretion increase the severity of the error. For example if the caller passed a string to be parsed and it failed, the string parsing function may raise a tiny error condition. But if the caller was going to use the string to open a GPD or resource file, then this suddenly becomes a major problem. A function may never reduce the severity of an error unless code was just executed which will migitate the source of the problem. Don't select ERRSEV_RESTART unless there is a handler on the next go round to solve the initial problem. An endless loop may result otherwise.