Hi Everyone,
(This is a long email. Sorry!)
I've been taking a look at how to implement the read-only layers + ownership
transfer idea over XMPP today and I would like to continue laying out this
plan on the mailing list. Please comment on this as much as possible so we
can have a good plan before implementing. There's probably some assumptions
that I am making from a lack of knowledge about xournal implementation
details
--
Alex: one quick note about how I think about layer sharing that is different
from yours. I think that sharing every page in a document would be simpler.
Also, all layers would be shared. I don't know whether that would be a loss
of useful functionality. I would welcome any feedback on this.
The situation that I think this alternate system solves is the possibility
of user A sharing page 63, layer 5 and user B receiving it on page 1, layer
1. I could see this being pretty confusing when users communicate. Also if
we implement ownership transfer, it saves users the hassle of managing
ownership of each layers independently.
In the notes following, I am assuming we're using a 'share everything,
single layer' model. If we decide not to go with it, we can add the
necessary details afterwards.
--
XMPP challenges
In the read-only layer implementation implemented over XMPP (without
ownership transfer for now), we need to copy strokes from one client to
another (No server if we use XMPP). The challenge with XMPP is showing that
regardless of (a) missing commands or (b) reordered commands, our
implementation won't have issues. Since we don't have a server we might need
to issue IDs per client to work around the reordered commands (i.e. command
from client=2, id=4)
(a) To show we can handle dropped transmissions, we can define our messaging
'protocol' as follows (implemented on top of XMPP). For every command sent
(i.e. client A: create ITEM_STROKE id=4), follow with a confirmation (i.e.
client B: OK id=4). If client A doesn't get the confirmation, resend a
request for confirmation (i.e. client A: confirm create ITEM_STROKE id=4).
Note the whole command is in the confirmation request. If client B has
already created the item id=4, then just send OK. If not, then create the
item and send OK. Client A keeps resending the confirmation request until it
gets a response from client B or until there's a timeout. This handles the
possibility that there's a dropped message in the command or in the
confirmation.
(b) To work around the reordering of commands, we should check to see
whether the current command from the client has an ID which is incrementally
bigger (to see if it is next in the series or if it was reordered). If it
was reordered, wait for it to arrive, and store the current command.
Layer ownership transfer challenges
If we further try to implement the layer ownership model I described
earlier, any layer would only have 1 editor at a time. This is kind of like
a lock. Since we're using potentially flaky connections, we can run into
problems with abandoned locked etc. The situations we need to avoid are (a)
orphaned ownership and (b) dual ownership.
(a) The possibility of orphaned ownership arises when a user disconnects
while in possession of someone else's lock. The simple solution would be to
have the original owner implement a timeout and reclaim ownership after that
period of time. However, the issue is that if the temporary owner continues
editing and reconnects at a later time, both are in possession of the layer
(b) In addition to the timeout/reconnect possibility, dual ownership can
also arise from a dropped message. We need to make sure ownership transfers
implement the 'protocol' as well. As Alex suggested, perhaps the best way to
resolve these types of conflict are to let the users decide whose changes to
keep. We'll need to implement a way of deleting the unimportant changes.
Commands
As Denis said in an earlier message, there are several commands we need to
implement to communicate between xournal clients. However if we use a
read-only layer implementation, we do not need to implement every command in
the same way. I've taken the liberty of splitting them according to whether
they interact with a specific layer.
==layer dependent interaction==
#define ITEM_STROKE 0
#define ITEM_PASTE 5
#define ITEM_TEXT 18
#define ITEM_TEMP_TEXT 19
#define ITEM_ERASURE 2
#define ITEM_MOVESEL 4
#define ITEM_REPAINTSEL 15
#define ITEM_TEXT_EDIT 20
#define ITEM_TEXT_ATTRIB 21
#define ITEM_RESIZESEL 22
#define ITEM_NEW_LAYER 6
#define ITEM_DELETE_LAYER 7
==layer independent interaction==
#define ITEM_NEW_BG_ONE 8
#define ITEM_NEW_BG_RESIZE 9
#define ITEM_PAPER_RESIZE 10
#define ITEM_NEW_DEFAULT_BG 11
#define ITEM_NEW_PAGE 13
#define ITEM_DELETE_PAGE 14
We also have some commands that do not need to be implemented as a network
command because they don't interact with the journal directly (as far as I
understand)
==local commands==
#define ITEM_RECOGNIZER 23
Finally, we will need to add a few more commands to allow for ownership
transfer of layers
==new commands==
- Request layer ownership
- Ownership request response
- Assuming layer ownership
- Most recent command seen
- Join/leave commands
- client version commands
Returning layer ownership can be implemented as an ownership request with an
automatic accept (i.e. check to see if layer id was originally owned by me,
if yes: auto accept) so there's no need for that command to be implemented
The 'most recent command seen' command could be used to sync users who join
a collaboration mid-session. (i.e. new client sends last command seen, and
other clients send every command with ID greater than that one)
Implementing commands
Each class of commands can be handled differently.
1. Sending layer dependent interactions need to be locked when there is no
ownership of the layer.
2. Receiving layer dependent commands with a layer ID should invoke an
action on the read-only layer with matching layer ID
3. Layer independent interactions should only be issued by an 'admin' user,
with exception of ITEM_NEW_PAGE.
XMPP library
I'm thinking of using gloox as the XMPP lib in question. It's a pretty
stable multi-platform (win/nix/osx/symbian) library which implements all of
XMPP core, and also most of the XMPP IM features. It's GPL licensed, and
included in the repos for Ubuntu, Fedora, Suse, and Gentoo. Has anyone heard
of this, or used it before?
http://camaya.net/gloox/features
I'm not sure what kind of XMPP features are useful to us yet. Multi-user
chat seems promising, but I think we would want to encode our strokes to hex
first before sending.
Another great feature is the offline message retrieval. We can probably use
that to take care of reconnecting users.
That's it for now. Please let me know what you think.
Matt
2011/4/21 Alexander Shumakovitch <shu...@gwu.edu>
> Hi Matthew,
>
> I would definitely be happy to share the workload with you, but I'm
> unfortunately completely ignorant in anything related to networking and
> communication protocols. Since this is exactly where most of the
> difficulties
> lie, I'm afraid I won't be of much help :( In any case, I will be
> completely
> swamped with my daily work for the next 2-3 weeks, but will start poking
> around after that. At the moment, I can't even compile Xournal++ on my
> laptop,
> so getting past this hurdle will be my first priority.
>
> Thanks,
>
> --- Alex.
>
> On Wed, Apr 20, 2011 at 02:48:28PM -0400, Matthew Chan wrote:
> > Hi everyone,
> >
> > This sounds like a great idea. We could even implement a ownership
> > transfer of layers. For example, if I wanted to edit a stroke on my
> > collaborator's layer, I could select one of his strokes and we could
> > both agree to temporarily transfer ownership of the layer to me so I
> > could recolour or resize it for example. My collaborator would be
> > unable to edit his layer until I release control of it.
> >
> > I'm not sure what happens to undo redo in this situation, but that
> > should be minor. One idea is to disallow undos further than the
> > tranfer of layer ownership
> >
> > I would be willing to work on implementing this feature if everyone
> > else thinks it is a good way to move forward.
> >
> > Alexander, If you're interested in this as well, I'm sure we can split
> > up the work. It's still a pretty large project, even with the read
> > only layer implemtation.
> >
> > Matt
> >
> > On Apr 20, 2011, at 12:27 AM, Denis Auroux <aur...@math.mit.edu> wrote:
> >
> > > Hello,
> > >
> > > Thanks for sharing this idea!
> > >
> > > * The idea of each layer having a unique owner at each time and being
> > > read-only for everyone else resembles something I suggested at some
> > > point as a limited collaborative mode, but you've fleshed out the
> > > details a bit more and quite nicely. I agree that this should be
> > > sufficient for most needs. I find the limitations a bit inelegant,
> > > so I
> > > haven't been too enthusiastic about it; but it is certainly much less
> > > troublesome in terms of conflicts, and easier to implement. Perhaps
> > > better to have a limited collaborative mode that works rather than a
> > > more ambitious one that never gets done. So, yes, if someone wants to
> > > implement this, I'm all in favor!
> > >
> > > (Note: you still have to decide who's allowed to create new pages --
> > > perhaps everyone by default? -- and to delete pages -- perhaps an
> > > answer
> > > in the spirit of your proposal is that everyone can delete pages but
> > > they only disappear from the view of the person who deletes them, and
> > > they remain available to the others?).
> > >
> > > * You wrote: "run additional threads that update the contents" for
> > > handling the remote-controlled layers. In my opinion, threads are a
> > > bad
> > > idea even in this limited-conflict environment -- then you have to
> > > handle all sorts of potential race conditions, e.g. when the user
> > > decides to "detach" a layer while the network-listening thread is
> > > trying
> > > to update it. Contrary to a common belief, reading incoming network
> > > data
> > > is NOT a blocking operation, even on a standard blocking TCP socket --
> > > at least if you're using the appropriate mechanism for listening (e.g.
> > > setting up a GTK+ callback for incoming data, which internally
> > > relies on
> > > select()/poll()) and willing to accumulate partially read messages
> > > into
> > > buffer space until there's a full message available to process. So
> > > there's no need for a separate thread, just a few event handlers.
> > >
> > > * Using an IM protocol for handshake, authentication, and going around
> > > firewalls: this is a very interesting idea, though it's not
> > > necessarily
> > > ideal for all collaboration settings. I find the idea slightly
> > > intrusive. As a user, I don't want to have to sign up for any
> > > particular
> > > IM system. But at the same time it's an elegant way of connecting the
> > > collaborators -- especially if we could make things work over a
> > > variety
> > > of IM networks. For instance, would we be able to use libpurple?
> > > (since
> > > it seems to be able to connect to pretty much every IM system out
> > > there...).
> > >
> > > One big drawback: as far as I know, most IM systems do not guarantee
> > > delivery of the messages sent through them, unlike a plain TCP socket.
> > > They might not even be a guarantee that the successive messages sent
> > > by
> > > a particular party in the chat room will arrive to everyone in the
> > > same
> > > order they were sent in (!). If one has to re-invent a whole TCP-like
> > > system for handling acknowledgements, re-sending missed packets, etc.,
> > > it's going to be a major pain. Or do any IM protocols provide a public
> > > interface to something with the same delivery guarantees as a TCP
> > > socket? Whether the IM protocol is XML-based or not is irrelevant,
> > > since the internal messages between instances of xournal need not be
> > > in
> > > any format like that of the IM system in which the communication runs.
> > >
> > > Denis
> > >
> > >
> > >
> > > On 04/19/2011 07:54 PM, Alexander Shumakovitch wrote:
> > >> Hello all,
> > >>
> > >> I've just subscribed to this mailing list and am not able to
> > >> directly reply to
> > >> the previous thread about collaboration in Xournal. Sorry.
> > >>
> > >> I was thinking about the easiest way to implement collaboration in
> > >> Xournal a
> > >> while ago and came up with a plan on how to do this in a way that
> > >> 1) doesn't
> > >> require big modifications to the existing code and 2) avoids all
> > >> the issues
> > >> with synchronization, Internet lag and so on. The idea is to
> > >> utilize the
> > >> existing infrastructure of layers.
> > >>
> > >> Here is how the process would work: collaborator A(lice) makes one
> > >> or more
> > >> layers from the page she is working on, available to selected
> > >> people (more on
> > >> implementation of this below). Her collaborator B(ob) sees that A
> > >> made some
> > >> layers available for him for viewing and incorporates them into his
> > >> own page
> > >> (most probably, below layers he is working on). These layer(s) are
> > >> available
> > >> for B read-only, but he sees every change that A makes. At the same
> > >> time, B
> > >> can make his working layer available to A so that she incorporates
> > >> it into her
> > >> own page. A and B can exchange the ownership of every shared layer
> > >> at any
> > >> moment. It should also be possible to "detach" the read-only layer
> > >> and make it
> > >> your own (before saving into a file, for example). This would
> > >> destroy the
> > >> synchronization between two copies of the layer though.
> > >>
> > >> What this scheme achieves is that for every shared layer there
> > >> exists exactly
> > >> one collaborator who can modify it, and this collaborator works on
> > >> this layer
> > >> _locally_. All other collaborators can look but don't change. But
> > >> since layers
> > >> are stacked on top of each other, they will see a superposition of
> > >> shared
> > >> layers and their own writing --- exactly what is needed. In fact,
> > >> this matches
> > >> the way I usually work with my collaborators on the same piece of
> > >> paper: I
> > >> never cross out what the other has written, but often write on top
> > >> of it.
> > >>
> > >> Advantages of this scheme:
> > >> 1) Easy to incorporate into existing structure. Basically, one
> > >> needs to allow
> > >> layers to be read-only and to run additional threads that are
> > >> going to
> > >> update their content.
> > >> 2) Each layer is edited only locally. There might be a delay with
> > >> showing the
> > >> result of the changes to a remote collaborator, but not on the
> > >> computer
> > >> where the editing is done.
> > >> 3) Easily scalable to any number of collaborators.
> > >>
> > >> Possible disadvantages:
> > >> 1) Not easy to resume synchronous collaboration after a break (when
> > >> the
> > >> document was saved on both ends). One solution could be to keep
> > >> a checksum
> > >> with each layer and if checksums for a certain layer match on
> > >> both ends,
> > >> then propose to collaborators to decide who is going to edit it.
> > >> At the
> > >> end, it would be up to humans to decide which layers to share
> > >> and how.
> > >> 2) Not possible to edit the same thing on both sides. Is this
> > >> really needed?
> > >> 3) ????
> > >>
> > >> Now, concerning the issue of authentication and sharing of the
> > >> data. I believe
> > >> that the easiest solution would be to use the existing IM
> > >> infrastructure.
> > >> Almost everyone nowadays has an account with at least one IM
> > >> network and there
> > >> are plenty of software/libraries that implement interconnections
> > >> between them.
> > >> These existing software can handle authentication. In particular,
> > >> take care of
> > >> such issues as making some content (layers) available to other
> > >> participants of
> > >> the network. Moreover, the existing XMPP protocol (used by Jabber
> > >> and others)
> > >> is based on XML, so I think it should not be overly complicated to
> > >> incorporate
> > >> Xournal primitives into it. The data would be shared stroke-by-
> > >> stroke. I guess
> > >> this should result in a reasonable latency, especially considering
> > >> that
> > >> communication for each layer would be unidirectional. I'm not any
> > >> specialist
> > >> in Internet protocols though.
> > >>
> > >> Oh, and using existing IM infrastructure would eliminate the need for
> > >> distinguishing between servers and clients and avoid problems of
> > >> having
> > >> computers behind firewalls (most of those used at home are).
> > >>
> > >> So what do you think? Does this all make sense?
> > >>
> > >> Thanks for your attention,
> > >>
> > >> --- Alex.
> > >>
> > >> ---
> > >> ---
> > >> ---
> > >> ---------------------------------------------------------------------
> > >> Benefiting from Server Virtualization: Beyond Initial Workload
> > >> Consolidation -- Increasing the use of server virtualization is a top
> > >> priority.Virtualization can reduce costs, simplify management, and
> > >> improve
> > >> application availability and disaster protection. Learn more about
> > >> boosting
> > >> the value of server virtualization.
> http://p.sf.net/sfu/vmware-sfdev2dev
> > >> _______________________________________________
> > >> Xournal-devel mailing list
> > >> Xournal-devel@lists.sourceforge.net
> > >> https://lists.sourceforge.net/lists/listinfo/xournal-devel
> > >
> > >
> > > --
> > > Denis Auroux
> > > MIT Department of Mathematics aur...@math.mit.edu
> > > (on leave) and
> > > University of California, Berkeley aur...@math.berkeley.edu
> > > Department of Mathematics Tel: 510-642-4367
> > > 817 Evans Hall # 3840 Fax: 510-642-8204
> > > Berkeley, CA 94720-3840
> > >
> > > ---
> > > ---
> > > ---
> > > ---------------------------------------------------------------------
> > > Benefiting from Server Virtualization: Beyond Initial Workload
> > > Consolidation -- Increasing the use of server virtualization is a top
> > > priority.Virtualization can reduce costs, simplify management, and
> > > improve
> > > application availability and disaster protection. Learn more about
> > > boosting
> > > the value of server virtualization.
> http://p.sf.net/sfu/vmware-sfdev2dev
> > > _______________________________________________
> > > Xournal-devel mailing list
> > > Xournal-devel@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/xournal-devel
>
------------------------------------------------------------------------------
Fulfilling the Lean Software Promise
Lean software platforms are now widely adopted and the benefits have been
demonstrated beyond question. Learn why your peers are replacing JEE
containers with lightweight application servers - and what you can gain
from the move. http://p.sf.net/sfu/vmware-sfemails
_______________________________________________
Xournal-devel mailing list
Xournal-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xournal-devel