> The last part of this, turning the POST response page back into a Plucker
> database becomes logistically very bizarre. What if the user wants to
> fill out the form again? What if the form data changes with each post?
> Personally, the AvantGo way makes sense to me. Fill out the form, warn
> the user the post won't be made until next sync, then go on with life.
> It makes feedback forms useful. Not many other uses for forms make sense
> for offline web viewing. Other ideas are welcome...
Architecturally, this is how it "should" be done, AvantSlow or
otherwise:
1. Sync to a URI which includes a form, which brings down an empty,
non-cast form (no values in the elements).
2. Create a database of that URI, which should *NOT* be physically
located inside any other database proper. What I mean here is that
if you grab a top-level URI, and have --maxdepth=3, and on
page3.html, there's a form, this form should *NOT* go inside
MyURI.pdb, it should be externally linked (and writeable) as
MyURI-formXX.pdb, where XX is the number of the forms in that
gather process (example is 4 pages which are linked, each with
their own forms).
Why do we want it external? So we can modify that form, save the
values, possibly delete it ("Reset" the form), POST it's values
upon sync, and sync *BACK* to it from the remote site again after
POST.
If we kept that form inside MyURI.pdb, and sync'd back, we'd have
to sync (and create) a whole new MyURI.pdb file all over again,
which could contain completely new (non-form) content (page1.html
might be new now, for example). We want to retain the ability to
keep the old content, but simply POST form values.
At this point, we've simply got a form on the Palm, and have
populated it with some values, but have not yet sync'd back to the
server.
3. Form values are saved in some writable place. Remember that
MyURI.pdb is not writeable at all in our current scheme, nor is
deletion or modification of pages themselves.
4. Run the parser again, but this time it communicates with the Palm
itself, querying for any remaining form values which show up as
"DirtyRecords" in the databases which it queries. This begs the
question of using an offline cache, but then what if I sync at home
and pull down a form, fill it out on my commute to work, then sync
again to POST those values on my work desktop? So the off-Palm
cache idea won't work for long either.
5. Once the desktop parser has the "Dirty" records in tow, it can then
build a POST response, sending it to the server, and awaiting for
responses from the server to those values. Here's where a threaded
spider that can do parallel gathers would be nice. Since
gethostbyname() is blocking in most architectures, you're going to
wait until the server responds with something, before moving onto
the next URL in your parse run for that session.
6. Content is returned from the server, parser gathers it up again,
and somehow has to push this back to the Palm, into a place where
MyURI.pdb can then locate the content as "new", and allow the user
to interact with it. This assumes, of course, that you were *ONLY*
sync'ing to move this form data around, not to create new PDB's at
this time. If you wanted to sync new form data, *AND* update the
database, it could present a problem also, or if you were creating
new databases during that sync.
7. Now you should have MyURI.pdb, the original, unmodified version,
were in MyURI-form01.pdb) on your Palm. Launching MyURI should now
let you interact with the results of those values. Remember also
that you have to POST first, then receive the response, *NOT* the
reverse. Seems trivial, but if you have 10 databases all with 10
forms each, there's some logic flow that has to happen to make that
work. It also moves us closer to supporting cookies and lockfiles
on the desktop side and in memory.
This is my first pass at this, but I hope it gives everyone some ideas
to chew on. Remember too, that "our friends" at AvantSlow are actively on this
list, listening to everything we do and say. If any of them wants to lend a
hand helping us figure this out, that would be nice. We just have to make sure
that we cover all the possible contingencies, and remove *ANY* possibility of
data loss. My concerns at this stage are:
a. Where do we store the form data on the Palm?
b. Where do we store the form data at sync time while a parser run is
gathering OTHER (non-posted form content)?
c. How do we support SSL-based forms in the current Python scheme,
without requireing the user to recompile Python with libssl?
d. Do we need to support cookies, locked into the form database on the
palm, and POST that back in the "session" we create when we sync
that content back up to the server at POST time?
e. How do we handle the precedence of forms in a depth-first vs.
breadth-first gathering process?
f. How do we handle the threading of gathers? POST form values at the
end of the parser run? (one sync required; parse, sync, pull form
data, push to server, get response, build pdb, push new data to
Palm) At the beginning? (two sync's required; one to get the form
values, disconnect, parse, sync to push new data to the Palm). If I
have a 2-meg database and am sync'ing without a cradle, I don't
want my Palm to sit there locked up over IrDA while content is
being GATHERED, I only want to wait while content is being SYNC'd.
Devil's Advocate, It's on my business cards (not really).
Hope this churns some neurons.
/d
