OK, time for one of my trademark ridiculously ambitious plans. I will
write a prototype RELOAD parser generator, and I will write it today
(I've started). And then James (I'm too busy obviously) will use it to
port all our crappy file formats over to RELOAD, today. Wait, maybe
that's impossible :)

You have a data structure with 30 members some of which are arrays
(HeroDef, say), and you want to read or write it to disk. The really
old way: one BLOAD/GET or BSAVE/PUT statement. The newer way: use
loadrecord/storerecord (or sinful alternatives) and read/write members
one at a time in/out of the array. The current RELOAD way: write a 50
line serialisation function, and a 100 line bugprone deserialisation
function which does a very low amount of error checking and doesn't
explicitly define what the allowed input is. Not to mention it is
enormously slower than the loadrecord way due allocating a string,
then iterating over all the children of a node and doing a string
comparison for each, for every data field you set! And even the
loadrecord way is amazingly slow. You would think that disk read times
would dwarf all of this, but worryingly, no.

The way we're doing things is really painful, won't scale well, and
betrays what RELOAD was meant to accomplish.

There are two very different possible goals: either to make
serialising and deserialising data structures easy, or to make
reading/writing RELOAD easier, for the purpose of manually writing
de/serialisers that read/write RELOAD documents (which can then look
quite different from the in-memory data structures). I had thought
that was we needed was the first --- basically an annotated TYPE
declaration from which a read and a write function to be generated ---
but after sketching use cases, it seems like that won't work well.
Reading and writing RELOAD are actually very different, so you'd need
lots of annotations for both, but more importantly it's not flexible:
RELOAD documents largely need to be read in (at least partially)
imperative fashion. For example, you have to read the map number from
an RSAV "state" node so that you can load the map offset from a
different file before you can parse the NPC data, unless you correct
all the positions in a post processing step.

So here are my current ideas. These is meant to be a quick but working
prototype, so the details are very open to change. Obviously I'm
writing this in the hope of feedback --- today :). Of course it's
crazy late/early, and this isn't a complete description.

After trying out various schemes, I think a good solution is to allow
a mix of imperative and declarative programming for parsing nodes. The
declarative parts are important: they allow efficiently iterating over
the children of a node and either processing each in turn, or raising
a warning or error if it's unexpected.

Writing the generator in Python seems best, since it's now required
for building anyway. I might use a parser generator to create my
parser generator if hand coding it is too annoying. It'll take an
.rbas input file which is written in an extension of FB with
additional syntax (RELOADBASIC, "RB") and contains lots of mostly
normal looking functions for reading or writing RELOAD nodes. The
generator spits out a .bas file. It reads variable declarations to
find out which symbols are NodePtrs. We split most of the existing
RELOAD read/write functions out to .rbas files.

If n is a NodePtr, then n."childname"[.attibutes] (a "node spec") is
an expression evaluating to the value of a childe of n, if not used in
a special context. For example, n."foo".string is  equivalent to
GetChildNodeStr(n, "foo"), but does some magic to avoid any use of
strings (so should be much much faster). The default type is integer;
n."foo" is GetChildNodeInt(n, "foo"). You can stack attributes,
n."foo".string.default("EMPTY"). I guess that's a bit ugly. Maybe
n.string("foo", "EMPTY") instead. It's just that I hate typing
parentheses. Maybe I should switch to Ruby. n."child"."grandchild" may
be used. Or maybe n."child/grandchild" instead.

A node spec is a left-value: assigning to one translates into the
equivalent of SetChildNode or AppendChildNode (Set by default, use
.multi to get Append).

Normally only node specs are used in serialising functions. They're
already pretty compact, just a little verbose, so little needs to be
done.

READNODE [parent."pat" AS] node
  ...
END READNODE
'node' must be a NodePtr. Each non-blank line of this block must be
either a nested READNODE, FORNODE, WITHNODE or LOADARRAY referring to
a child of 'node', or an arbitrary line of RB source which contains
exactly one node spec referring to a child of 'node'. This translates
to a select/switch inside a while loop iterating over the children of
'node'. Any child of 'node' which is not referred to inside the
READNODE causes an "unexpected node" warning.

FORNODE "childname" in parent as node
   ...
NEXTNODE
Contains arbitrary RB code, run once per child of the parent node
matching the name. This either translates into a loop or a select case
depending on context.

WITHNODE parent."spec" as node
  ...
END WITHNODE
Contains arbitrary RB code. Mostly for convenience (created a NodePtr
variable named 'node'), but also needed for using longer snippets of
RB code in a READNODE block.

LOADARRAY fb_array_indexed_by($var) =
expression_containing(node."child"[$var]."value")
Not sure about this one: it's an easy way to read into a FB array
rather than using a FORNODE block and doing manual bounds checking.
'var' should be an integer variable. For each child of 'node' named
"child", the integer value of each child is placed in 'var', and a
bounds check is done on 'fb_array_indexed_by'. Either a warning/error
is printed, or the expression is run.

Those are all the constructs.

All nodes default to integers, are optional with a default of 0, and
show a warning if they appear more than once, unless they're used in a
FORNODE.

Here are the attribute tags I've thought of:
.integer, .string, .double -- straight forward
.bool -- corresponds to GetChildNodeBool: default to 0 if doesn't
exist, return GetInteger(node) <> 0 otherwise
.flag -- this is what's used in most places instead of .bool: YES if
and only if the child exists. Can't be a default or be optional or
required or appear more than once.
.default(def_value) -- default value to use if missing
.ignore -- don't print a warning when this is seen. This creates a
statement instead of an expression
.required -- error if not occurring exactly once; exit the function.
.warn -- print a warning if missing
.multi -- no warning on multiple occurrences

Examples: I translated a few existing functions:



SUB load_npc_loc (byval n as NodePtr, npc as NPCInst, map_offset as
XYPair, warningfunction as FnWarn)
 DIM i as integer
 IF GetChildNodeExists(n, "id") = NO THEN
  npc.id = 0
 ELSE
  WITH npc
   WITHNODE n
    .id = n."id" + 1
    .x = n."x".required + map_offset.x * 20
    .y = n."y" + map_offset.y * 20
    .dir = n."d"
    .frame = n."fr"
    .xgo = n."xgo"
    .ygo = n."ygo"
    LOADARRAY .extra($i) = n."extra"[$i]."int"
    .ignore_walls = n."ignore_walls".flag
    .not_obstruction = n."not_obstruction".flag
    .suspend_use = n."suspend_use".flag
    .suspend_ai = n."suspend_move".flag
   END WITHNODE
  END WITH
 END IF
END SUB

SUB gamestate_state_from_reload(byval parent as Reload.NodePtr)
 DIM node as NodePtr
 node = parent."state"

 gam.map.id = node."current_map".required

 DIM map_offset as XYPair
 map_offset = load_map_pos_save_offset(gam.map.id)

 READNODE node
  FORNODE "hero" in node."caterpillar" as n
   DIM i as integer = GetInteger(n)
   SELECT CASE i
    CASE 0 TO 3
     catx(i * 5) = n."x" + map_offset.x * 20
     caty(i * 5) = n."y" + map_offset.y * 20
     catd(i * 5) = n."d"
    CASE ELSE
     rsav_warn "invalid caterpillar hero index " & i
   END SELECT
   'There's no check for unrecognised children here...
  NEXTNODE

  gam.random_battle_countdown = node."random_battle_countdown"

  WITHNODE node."camera" as ch
   mapx = ch."x"
   mapy = ch."y"
   gen(genCamera) = ch."mode"
   FOR i as integer = 0 TO 3
    gen(genCamArg1 + i) = GetChildNodeInt(ch, "arg" & i+1)  'No parser
support for variable-name nodes
   NEXT i
   'There's no check for other children here either
  END WITHNODE

  gold = node."gold"

  READNODE node."playtime" as ch
   gen(genDays) = ch."days"
   gen(genHours) = ch."hours"
   gen(genMinutes) = ch."minutes"
   gen(genSeconds) = ch."seconds"
  END READNODE

  gen(genTextboxBackdrop) = node."textbox"."backdrop"

  READNODE node."status_char" as ch
   gen(genPoison) = ch."poison"
   gen(genStun) = ch."stun"
   gen(genMute) = ch."mute"
  END READNODE

  gen(genDamageCap) = node."damage_cap"
  gen(genLevelCap) = small(node."level_cap".default(99), gen(genMaxLevel))

  FORNODE "stat" in node."stats" as ch
   DIM i as integer = GetInteger(ch)
   SELECT CASE i
    CASE 0 TO 11
     gen(genStatCap + i) = ch."cap"
    CASE ELSE
     rsav_warn "invalid stat cap index " & i
   END SELECT
  NEXTNODE

 END READNODE
END SUB


'from SUB gamestate_party_from_reload
 FORNODE "slot" in node as slot
  READNODE slot

   slot."rename_on_add".ignore
   slot."hide_empty_lists".ignore

   FORNODE "list" in slot."spell_lists" as n
    DIM j as integer = GetInteger(n)
    SELECT CASE j
     CASE 0 TO 3
      gamestate_spelllist_from_reload(i, j, n)
     CASE ELSE
      rsav_warn "invalid spell list id " & j
    END SELECT
   NEXTNODE

   ....
_______________________________________________
Ohrrpgce mailing list
[email protected]
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org

Reply via email to