"David A. Desrosiers" wrote:
>
> > If the form is externally linked and writeable in it's own DB, wouldn't
> > it seem that that would be the logical place to write to?
>
> Only if the architecture allows for multiple versions of that same
> form to exist, to coincide with multiple versions of the original content
> which contains it.
What sort of situations do you envision this happening? Like a search
form on the front page on CNN.com? If the form and original content has
changed, we're dealing with a whole new page.
> > This seems like a job for the conduit rather than the parser. One of
> > our goals here is to eliminate the need for HotSyncing.
>
> The "conduit" *IS* the parser on non-Windows platforms. We have tobe
Or rather, non-Windows and non-Macintosh platforms. :)
> careful not to architect out a large section of our userbase. Or, we develop
> a conduit for the Unix folks again. Alexander and I hacked away at one quite
> a while back, which still sits in the original archive, and did a
> "real-time" sync of the records to the Palm from a local cache directory (or
> from a stream in the later version). If we rely on conduit action, we need
> to resurrect that.
I agree. The communicating part is Pyrite, right? I haven't used
Pyrite, but it looks like you're able to have pretty good control over
the syncing process. Hooray for reverse engineering. Using Pyrite,
transferring the pages to the palm is just about as fast as possible,
correct? Can you be oblivious to the "HotSync name"? i.e. if I just
wanted to send the pdb to the palm in the cradle, not caring whose Palm
was in the cradle, could it do it?
But our problem still stands.. On Windows and Mac, we don't have that
control over HotSync. PDBs only go to the HotSync name they're intended
for. All sorts of various other conduits like to do their thing as
well, making the process take longer. I don't want us to have to deal
suffer because of that. This entails either bypassing HotSync entirely
and making our own transfer program and protocol or taking some of the
Open Source sync libraries from unix and porting them to Mac and Windows.
> > Its probably not the most useful for the rest of the Plucker community,
> > but I'll share: an IE toolbar button launches a program which calls the
> > spider then serially sends the file the parser made to the palm via a
> > rudimentary protocol.
>
> Again, using IE (granted, a large percent of the population) will
> limit our target userbase to "lesser" functionality. Can this be done in
> Java? Shockwave? Something more portable?
Well, sure. Java and Shockwave don't let control the functionality of
the browser itself (i.e. installing menus in the menu bar, adding
toolbar buttons) but you could have a floating window or a frame where
clicking on a button inside of it would download a PDB of the current
page. The browser could be configured to recognize the PDBs and open it
up with a specific program, such as HotSync Manager.
> What about changing values already submitted before you sync?
>
> Do the original values get overwritten by my new values? Do they go
> in a new FormsDB for each time I fill it out? Does the form show up as
> "greyed out" when I return to it a second time, since I've already filled it
> out?
> Perhaps an Alert here?
>
> [Form already processed, overwrite with new form data?]
Yep. "Alert: you've already posted to this form. Do you want to add a
new posting or replace the one you've already made?"
> > I think the architecture we're looking for is similar to the way email
> > is handled on the Palm. When you send an email from Plucker, that email
> > is dumped into the MailDB which then has a conduit on the desktop to
> > deal with it. Forms should have their own DB, if not several, as you
> > suggested, for each page they live off of.
>
> For each *INSTANCE* of that page, including multiple versions for
> each time I filled out the form (as above). Perhaps allowing multiple
> versions of the Form to be filled out, then a way to view all versions, and
> delete the ones you don't want, before sync. Have to think about this one a
> bit more.
What do you mean by instance of a page? Plucker manages multiple
versions of a single page?
> When the sync is started, and the form data is pulled from the Palm,
> POST'ed, and then the response returned, where do we store that return
> response, while the OTHER content is being sync'd? Let's say I fill out a
> form on my Palm in Foo.pdb, and the data is stored in Foo-FormsDB-239.pdb,
> and I also am creating a new pdb for Wired.com, which is being gathered at
> the same time as the form is being processed. Where do we store that data,
> and is the Palm in a 'sync' state while content is being GATHERED, or just
> when it's being SYNC'd? (battery life, latency, etc.) Mike and I have tested
> and build several-dozen megabyte databases, which take quite a long time to
> gather and build, I don't want my Palm sitting in a cradle for 5 hours while
> these things are running. Do we require the user to sync twice; once to
> query and extract any form data to POST, and then again to actually push the
> new pdb(s) onto the Palm?
Why can't the form result pdb just be installed right then and there?
Or, at least, put into the queue to be installed. You have an "install"
directory with Pyrite/pilot-link, correct?
The second issue is important. If we gather web content at the same
time we're syncing, then we're no better off than AvantSlow (good one).
If all static content is prepared for installation before the sync and
all we process during the sync is forms, it may not be too bad since we
can gather the form response while installing the static content.
> A would wager that a large percentage of forms that are being filled
> out are behind an SSL wrapper. Online ordering, stock quote queries, bank
> statements, all of that is behind SSL (and if it's not.. I would hope nobody
> is using them). It's going to be something asked for, once people realize we
> have forms support.
Why you wouldn't want to use libssl?
> > Cookies would be nice, except they have usually have expiration times so
> > trying to use them at next sync would fail. At this point, we're going
> > to have to have *some* level of cooperation from the web site itself.
> > Either non-expiring cookies or a unique ID embedded in the form which
> > would get returned with the form results.
>
> We'll never get support from the websites, because we'll never know
> who is sync'ing what to what website. MAYBE some can detect the
> HTTP_USER_AGENT string, and use some other cookie based on that, but it
> would be a very small percentage. Cookies are used for session. We may be
> able to encode the cookie in the URI, but that too would require some more
> things Python doesn't have natively (and fortunately for me, perl does).
>
> We could also modify the cookie, not ideal, but it could be done.
>
> $expiry = $expiry + 3 months;
*gasp* Modifying the expiry of the cookie on the client side might
work on a few lazy websites, but most servers keep track of the
expiration of cookies and discard them accordingly.
> > > e. How do we handle the precedence of forms in a depth-first vs.
> > > breadth-first gathering process?
> >
> > I'm not sure what you mean here. Forms live on individual pages; what
> > sort of order is there between them?
>
> Observe the example in my previous email to Chris about the way my
> bank handles the online banking logins for a good example.
To get to the second form you're going to have to post correct
information to the first one. For a three-page form sequence, in the
most ideal case there'd be no way around syncing at least 3 times.
I'm afraid that if we try to take on all this functionality at once
we're not going to be able to get off the ground. Lets take some baby
steps here:
1) get controls embedded in Plucker documents
2) next, be able to store their results and retrieve them on a sync
3) next, be able to parse a response and install it on the palm
4) next, be able to manage multiple form submittals/responses
I'd feel a lot more comfortable planning everything else once we have a
base like this to work with.
