Hi Folks,
Last week Mimi and I sat down and talked about pain points associated
with sharing once collections are set up and working. We came up with a
list of issues and possible remedies.
The executive summary is that there are still several identified issues
causing sharing pain, broadly grouped into:
1. Failure to sync
2. Inaccurate pop-to-now
3. Pop-to-now with the byline showing the wrong editor of the item
4. Items have conflicts, but the conflicts are in error or not obviously
associated with a user-visible attribute
We came up with possible remedies to these problems, based on my
understanding of the underlying causes. It's possible that fixing just
the first few problems may get the frequency of pain low enough that we
don't need to address all issues.
Problems
========
The first problem blocks usage, the rest mainly affect people heavily
using shared collections and the dashboard view. These latter problems
aren't show-stoppers by themselves; the occasional confusing item
popping to now doesn't make Chandler useless. Unfortunately, the
frequency of inaccurate pop-to-nows is high enough to make the NOW
section's signal-to-noise ratio much lower than it could be.
1. Collection fails to sync [1]
-------------------------------
Esther's been seeing the main example of this [2]. What's happening in
her particular case is the desktop and the server have different
pictures of what an item's state is, so when the desktop sends a diff,
the server rejects it for not containing enough information. It'd be
great to figure out why this is happening, unfortunately we haven't had
much luck diagnosing the problem so far, but even if we can't completely
stomp out this problem, we should fail more gracefully.
A fix to this could be to have the server's failure include a
machine-readable error (right now there's text that contains the failing
UUID, if we put that UUID in an invisible XML element, the desktop could
try resetting that item and sending the whole item, not a small diff).
There'd be a little bit of work on the server side, plus a fair amount
of desktop sharing work.
[1] http://chandlerproject.org/Planning/OneDotZeroBugReset#CriticalBugs
[2]
http://lists.osafoundation.org/pipermail/chandler-users/2008-January/001127.html
2. Mysterious pop-to-now [3]
----------------------------
There are a wide variety of reasons this is happening, it may be OK if
we fix 90% of them rather than 99% of them. With that in mind, my
educated guess is that the lion's share of these could be fixed by
adding a whitelist to the logic for deciding whether a sharing change
should pop an item to now. Right now, changes to most attributes (even
if their values aren't visible to the user in the detail view) will
trigger a pop-to-now.
Whitelisting attributes isn't 100%, occasionally it's ambiguous whether
a change to an EIM field will cause a user-visible change, but there are
many fields that clearly don't directly effect what a user sees in the
detail view, these shouldn't be allowed to cause a pop-to-now.
A first cut at this would be fairly simple to implement on the desktop.
There are also issues with Chandler's understanding of email addresses
being case-sensitive; it's not clear to me if these issues would be
solved by white-listing, there may still be work for Brian Kirsch there.
[3] http://chandlerproject.org/Planning/OneDotZeroBugReset#PopToNowBugs
3. Inaccurate byline
--------------------
There are two main sources of inaccurate bylines I'm aware of: The first
happens when two users have non-conflicting changes, but they sync
out-of-order. The byline is chosen based on the latest change, but the
change time is based on when the user actually changed the item in
Chandler, not when they synced. So out-of-order syncs will cause the
byline to stay the same, even though a new change has been merged in,
which can be disconcerting for the person who synced first.
A solution to this would be to add additional fields to the ModifiedBy
record, sending two users after a merge, and rendering an appropriate
message in this case. This is quite straightforward, but it might
require a bit of server work.
The second main cause of in accurate byline happens when an item is
deleted. The server stores a "tombstone", recognizing that the item's
been deleted, but it doesn't know *who* deleted it. This isn't a huge
problem for normal items (although it's a little annoying). For
recurring events, though, deletion actually means a triage status
change. In these cases, it's pretty confusing to see an occurrence pop
to now with nothing but a triage status change and an inaccurate byline.
One solution to this would be to change morse code to include
information about who's deleting an item, and having the server store
that alongside the existing tombstone. That might be a lot of work, though.
The recurrence problem could be fixed by changing the way triage-only
modifications are sent to the server. Unfortunately, changing this
would either break compatibility with existing data, or require a huge
amount of effort to maintain backward compatibility, so I think that's
out. This may just be a problem we live with.
4. Spurious conflicts
---------------------
Conflict resolution aims at preventing users from losing data.
Unfortunately, there are so many false positives that I literally had
never seen a *real* conflict until collaborating on writing this
document! All the other hundreds of conflicts I've seen have been
spurious, adding noise to my Chandler use and forcing me to examine and
click through too many conflict resolution dialogs.
Many spurious conflicts appear to come from having the same item shared
in multiple collections. When a user changes a multiply shared item
locally and syncs, the first associated collection receives the item
change. The second collection then sends the same change back. This
ought to result in a no-op, but apparently it doesn't always (that's a
bug in and of itself).
While ideally we'd fix the underlying bugs that are causing spurious
conflicts, it's not clear to me how much effort it would take to solve
the problem.
A radical, alternate approach that might reduce this pain, while still
providing data preservation would be to
A) Store item state before every sync, and after every sync (we mostly
do this now)
B) Provide an HTML picture of the history of an item, similar to the
view currently available by pressing F4 but focused on reproducing only
the attributes visible in the detail view
C) Auto-resolve *all* conflicts, with the more recent change winning,
then use the banner button currently associated with conflicts to alert
users if there was a conflict on a user-visible field
Implementing this strategy would be a moderate to a large amount of
work, so it's not clear if it's worth doing, even if we agree it's
desirable. On the plus side, a clean-UI for the history of an item is
an attractive feature in it's own right, so such an approach could be
implemented incrementally.
Conclusion
==========
I prioritize these roughly in the order described. Recovering from sync
failures and whitelisting user-facing attributes seem relatively
straightforward with a high value. Expanding the ModifiedBy record to
work better with merges seems lower priority but simple and valuable.
Morse Code changes to deleted items, viewing item history and
auto-resolving conflicts seem potentially valuable but possibly expensive.
Sincerely,
Jeffrey
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
