On 13 November 2011 07:42, James Paige <[email protected]> wrote:
> Short reply, I'll do a longer one later. This sounds like a wonderful
> idea. I like your trademark ridiculously ambitious plans! :)
>
> I am excited about this. I won't be able to start writing .rbas files
> today, (seeing my Fiancee this afternoon, driving to my sister's to
> see my Mom for the rest of the weekend)
Aww, but I kept up my end: hero editor and tags, cleaned up, today!
Wait... that's not right...
Seems my attention span is less than 24 hours :)
> I'll read this more carefully later, but I like where you are going with
> this, and I agree with your desire to avoid parentheis in the syntax.
> Parenthesis are important if the need to be nested, but that doesn't
> seem to be the case here as far as I can see.
>
> I am also wholeheartedly behind doing this in python. We are already
> using it for the build process, and I am a hopeless python fanboy anyway
> :)
I'm pleased and surprised at the positive response, since last time I
brought this up you said something along the lines of "at first this
sounded pretty alarming".
Thought about it a bit more and realised that WithNode is just a
restricted version of ForNode, so could be removed. Also, by tracking
child nodes accessed inside a WithNode/ForNode (with a bitmask), any
nodes not accessed could still cause "unexpected" warning messages.
> ---
> James
>
> On Sun, Nov 13, 2011 at 07:12:42AM +1300, Ralph Versteegen wrote:
>> OK, time for one of my trademark ridiculously ambitious plans. I will
>> write a prototype RELOAD parser generator, and I will write it today
>> (I've started). And then James (I'm too busy obviously) will use it to
>> port all our crappy file formats over to RELOAD, today. Wait, maybe
>> that's impossible :)
>>
>> You have a data structure with 30 members some of which are arrays
>> (HeroDef, say), and you want to read or write it to disk. The really
>> old way: one BLOAD/GET or BSAVE/PUT statement. The newer way: use
>> loadrecord/storerecord (or sinful alternatives) and read/write members
>> one at a time in/out of the array. The current RELOAD way: write a 50
>> line serialisation function, and a 100 line bugprone deserialisation
>> function which does a very low amount of error checking and doesn't
>> explicitly define what the allowed input is. Not to mention it is
>> enormously slower than the loadrecord way due allocating a string,
>> then iterating over all the children of a node and doing a string
>> comparison for each, for every data field you set! And even the
>> loadrecord way is amazingly slow. You would think that disk read times
>> would dwarf all of this, but worryingly, no.
>>
>> The way we're doing things is really painful, won't scale well, and
>> betrays what RELOAD was meant to accomplish.
>>
>> There are two very different possible goals: either to make
>> serialising and deserialising data structures easy, or to make
>> reading/writing RELOAD easier, for the purpose of manually writing
>> de/serialisers that read/write RELOAD documents (which can then look
>> quite different from the in-memory data structures). I had thought
>> that was we needed was the first --- basically an annotated TYPE
>> declaration from which a read and a write function to be generated ---
>> but after sketching use cases, it seems like that won't work well.
>> Reading and writing RELOAD are actually very different, so you'd need
>> lots of annotations for both, but more importantly it's not flexible:
>> RELOAD documents largely need to be read in (at least partially)
>> imperative fashion. For example, you have to read the map number from
>> an RSAV "state" node so that you can load the map offset from a
>> different file before you can parse the NPC data, unless you correct
>> all the positions in a post processing step.
>>
>> So here are my current ideas. These is meant to be a quick but working
>> prototype, so the details are very open to change. Obviously I'm
>> writing this in the hope of feedback --- today :). Of course it's
>> crazy late/early, and this isn't a complete description.
>>
> > After trying out various schemes, I think a good solution is to allow
>> a mix of imperative and declarative programming for parsing nodes. The
>> declarative parts are important: they allow efficiently iterating over
>> the children of a node and either processing each in turn, or raising
>> a warning or error if it's unexpected.
>>
>> Writing the generator in Python seems best, since it's now required
>> for building anyway. I might use a parser generator to create my
>> parser generator if hand coding it is too annoying. It'll take an
>> .rbas input file which is written in an extension of FB with
>> additional syntax (RELOADBASIC, "RB") and contains lots of mostly
>> normal looking functions for reading or writing RELOAD nodes. The
>> generator spits out a .bas file. It reads variable declarations to
>> find out which symbols are NodePtrs. We split most of the existing
>> RELOAD read/write functions out to .rbas files.
>>
>> If n is a NodePtr, then n."childname"[.attibutes] (a "node spec") is
>> an expression evaluating to the value of a childe of n, if not used in
>> a special context. For example, n."foo".string is equivalent to
>> GetChildNodeStr(n, "foo"), but does some magic to avoid any use of
>> strings (so should be much much faster). The default type is integer;
>> n."foo" is GetChildNodeInt(n, "foo"). You can stack attributes,
>> n."foo".string.default("EMPTY"). I guess that's a bit ugly. Maybe
>> n.string("foo", "EMPTY") instead. It's just that I hate typing
>> parentheses. Maybe I should switch to Ruby. n."child"."grandchild" may
>> be used. Or maybe n."child/grandchild" instead.
>>
>> A node spec is a left-value: assigning to one translates into the
>> equivalent of SetChildNode or AppendChildNode (Set by default, use
>> .multi to get Append).
>>
>> Normally only node specs are used in serialising functions. They're
>> already pretty compact, just a little verbose, so little needs to be
>> done.
>>
>> READNODE [parent."pat" AS] node
>> ...
>> END READNODE
>> 'node' must be a NodePtr. Each non-blank line of this block must be
>> either a nested READNODE, FORNODE, WITHNODE or LOADARRAY referring to
>> a child of 'node', or an arbitrary line of RB source which contains
>> exactly one node spec referring to a child of 'node'. This translates
>> to a select/switch inside a while loop iterating over the children of
>> 'node'. Any child of 'node' which is not referred to inside the
>> READNODE causes an "unexpected node" warning.
>>
>> FORNODE "childname" in parent as node
>> ...
>> NEXTNODE
>> Contains arbitrary RB code, run once per child of the parent node
>> matching the name. This either translates into a loop or a select case
>> depending on context.
>>
>> WITHNODE parent."spec" as node
>> ...
>> END WITHNODE
>> Contains arbitrary RB code. Mostly for convenience (created a NodePtr
>> variable named 'node'), but also needed for using longer snippets of
>> RB code in a READNODE block.
>>
>> LOADARRAY fb_array_indexed_by($var) =
>> expression_containing(node."child"[$var]."value")
>> Not sure about this one: it's an easy way to read into a FB array
>> rather than using a FORNODE block and doing manual bounds checking.
>> 'var' should be an integer variable. For each child of 'node' named
>> "child", the integer value of each child is placed in 'var', and a
>> bounds check is done on 'fb_array_indexed_by'. Either a warning/error
>> is printed, or the expression is run.
>>
>> Those are all the constructs.
>>
>> All nodes default to integers, are optional with a default of 0, and
>> show a warning if they appear more than once, unless they're used in a
>> FORNODE.
>>
>> Here are the attribute tags I've thought of:
>> .integer, .string, .double -- straight forward
>> .bool -- corresponds to GetChildNodeBool: default to 0 if doesn't
>> exist, return GetInteger(node) <> 0 otherwise
>> .flag -- this is what's used in most places instead of .bool: YES if
>> and only if the child exists. Can't be a default or be optional or
>> required or appear more than once.
>> .default(def_value) -- default value to use if missing
>> .ignore -- don't print a warning when this is seen. This creates a
>> statement instead of an expression
>> .required -- error if not occurring exactly once; exit the function.
>> .warn -- print a warning if missing
>> .multi -- no warning on multiple occurrences
>>
>> Examples: I translated a few existing functions:
>>
>>
>>
>> SUB load_npc_loc (byval n as NodePtr, npc as NPCInst, map_offset as
>> XYPair, warningfunction as FnWarn)
>> DIM i as integer
>> IF GetChildNodeExists(n, "id") = NO THEN
>> npc.id = 0
>> ELSE
>> WITH npc
>> WITHNODE n
>> .id = n."id" + 1
>> .x = n."x".required + map_offset.x * 20
>> .y = n."y" + map_offset.y * 20
>> .dir = n."d"
>> .frame = n."fr"
>> .xgo = n."xgo"
>> .ygo = n."ygo"
>> LOADARRAY .extra($i) = n."extra"[$i]."int"
>> .ignore_walls = n."ignore_walls".flag
>> .not_obstruction = n."not_obstruction".flag
>> .suspend_use = n."suspend_use".flag
>> .suspend_ai = n."suspend_move".flag
>> END WITHNODE
>> END WITH
>> END IF
>> END SUB
>>
>> SUB gamestate_state_from_reload(byval parent as Reload.NodePtr)
>> DIM node as NodePtr
>> node = parent."state"
>>
>> gam.map.id = node."current_map".required
>>
>> DIM map_offset as XYPair
>> map_offset = load_map_pos_save_offset(gam.map.id)
>>
>> READNODE node
>> FORNODE "hero" in node."caterpillar" as n
>> DIM i as integer = GetInteger(n)
>> SELECT CASE i
>> CASE 0 TO 3
>> catx(i * 5) = n."x" + map_offset.x * 20
>> caty(i * 5) = n."y" + map_offset.y * 20
>> catd(i * 5) = n."d"
>> CASE ELSE
>> rsav_warn "invalid caterpillar hero index " & i
>> END SELECT
>> 'There's no check for unrecognised children here...
>> NEXTNODE
>>
>> gam.random_battle_countdown = node."random_battle_countdown"
>>
>> WITHNODE node."camera" as ch
>> mapx = ch."x"
>> mapy = ch."y"
>> gen(genCamera) = ch."mode"
>> FOR i as integer = 0 TO 3
>> gen(genCamArg1 + i) = GetChildNodeInt(ch, "arg" & i+1) 'No parser
>> support for variable-name nodes
>> NEXT i
>> 'There's no check for other children here either
>> END WITHNODE
>>
>> gold = node."gold"
>>
>> READNODE node."playtime" as ch
>> gen(genDays) = ch."days"
>> gen(genHours) = ch."hours"
>> gen(genMinutes) = ch."minutes"
>> gen(genSeconds) = ch."seconds"
>> END READNODE
>>
>> gen(genTextboxBackdrop) = node."textbox"."backdrop"
>>
>> READNODE node."status_char" as ch
>> gen(genPoison) = ch."poison"
>> gen(genStun) = ch."stun"
>> gen(genMute) = ch."mute"
>> END READNODE
>>
>> gen(genDamageCap) = node."damage_cap"
>> gen(genLevelCap) = small(node."level_cap".default(99), gen(genMaxLevel))
>>
>> FORNODE "stat" in node."stats" as ch
>> DIM i as integer = GetInteger(ch)
>> SELECT CASE i
>> CASE 0 TO 11
>> gen(genStatCap + i) = ch."cap"
>> CASE ELSE
>> rsav_warn "invalid stat cap index " & i
>> END SELECT
>> NEXTNODE
>>
>> END READNODE
>> END SUB
>>
>>
>> 'from SUB gamestate_party_from_reload
>> FORNODE "slot" in node as slot
>> READNODE slot
>>
>> slot."rename_on_add".ignore
>> slot."hide_empty_lists".ignore
>>
>> FORNODE "list" in slot."spell_lists" as n
>> DIM j as integer = GetInteger(n)
>> SELECT CASE j
>> CASE 0 TO 3
>> gamestate_spelllist_from_reload(i, j, n)
>> CASE ELSE
>> rsav_warn "invalid spell list id " & j
>> END SELECT
>> NEXTNODE
>>
>> ....
>> _______________________________________________
>> Ohrrpgce mailing list
>> [email protected]
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
> _______________________________________________
> Ohrrpgce mailing list
> [email protected]
> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>
_______________________________________________
Ohrrpgce mailing list
[email protected]
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org