Greetings ESME team:
In trying to design a proposed end-user web UI for ESME last week, I
realized there were too many open issues related to the ESME
dataspace (as seen from the end user's perspective), and the
functionality associated therewith. So I spent most of my time last
week thinking about those issues. I have a lot of ideas to suggest,
and questions to ask, and thought for the team to review and discuss.
It all seems to long and complicated for email, but the esme-dev
mailing list seems like the only thing we've got -- so here goes (I
do wish we had a discussion board...).
I hope this all makes sense that that you don't mind me raising so
many issues at once.
Regards,
Bill
============================================================
THOUGHTS AND QUESTIONS RE. THE END-USER DATA-MODEL AND RELATED
FUNCTIONALITY FOR ESME
by Bill Fernandez, 19feb09
============================================================
____________________________________________________________
SERVERS, DIRECTORIES, USERS, PROFILES
SCOPE
o Let's assume that we're limiting our discussion to a single server,
and that we'll deal with federations of servers in a future
discussion.
USER DATABASE ASSUMPTIONS
o I'm assuming that for each ESME server there will be one and only
one user directory server (such as an LDAP server).
o I'm assuming that an ESME server will only recognize users listed
in the directory server.
o I'm assuming that the directory server is managed by
organization-level administrators, and that only these administrators
can add, edit and delete users.
IMPLICATIONS OF THE USER DATABASE ASSUMPTIONS
o WHO can use a given ESME server is thus out of control of the ESME
server, and in control of the user directory administrator(s).
o Since the ESME server will have to keep track of a number of
ESME-specific data for each user (e.g. "who do I follow?"), the ESME
server will have to maintain a database of "preferences" that are
keyed to each user in the directory server.
o Within the ESME end-user UI, this results in there being some info
that comes from the directory server and thus that the end user CAN'T
change (e.g. name, work phone, department, mailstop, etc.) and info
that only applies within the ESME context and that the end user CAN
change (e.g. nickname, maybe his/her picture or icon, list of people
he/she follows, etc.).
QUESTIONS, ISSUES
o Are the above assumptions and implications correct?
o Do we want to allow the administrator of an ESME server to be able
to create and maintain it's own private database of authorized users
in addition to those provided by the organization's directory server?
PROs: Makes it easy to set up a test/evaluation ESME server.
Makes it possible for small organizations (those without corporate
LDAP, etc. directories) to use ESME.
CONs: Introduces the possibility of duplicate user entries. If
used extensively makes synchronization with the organizational
directory impossible. Provides an end run around policies that the
organization is counting on the organizational directory to enforce.
o What ESME-specific, end-user-defined, user-profile data do we want
to support?
ESME Nickname? -- But what if organizational policy dictates
the use of only a person's "official" user name?
Picture/Icon? -- But what if the organizational directory
server supplies one? Do we use the "official" image by default, and
let any user-defined image override it?
o Would our strategy be that the ESME administrator could
enable/disable any of these user-profile-customization fields as
required by organizational policy, that we use the
organizational-server-supplied data by default, but override it with
the user-supplied values (if any)?
____________________________________________________________
GROUPS, POOLS, USERS, PRIVILEGES, ACCESS CONTROL
David says he hates the idea of groups (though he likes "pools") and
of sending things to groups (though he likes "posting to pools").
Nevertheless I'm going to use the term "groups" because it helps me
describe certain functionality, and I hope we can decide on
terminology after the functionality's agreed to.
With that, I'd like to suggest some concepts and see how the team
feels about them.
GROUPS
o I'd like to propose that each server can support an abitrary number
of groups.
o A group is a logical construct consisting of a set of data:
-- A set of users.
-- A pool of messages.
-- A set of metadata.
-- Perhaps a set of business rules.
GROUP MEMBERSHIP
o All users are members of a group called something like "Public",
which by definition contains the set of all users of the ESME server.
o A user may be a member of as many additional groups as desired.
GROUP CREATION, DEFINITION, DELETION
o Any user can create a group (or would we like the ESME
administrator to be able to grant a "create group" privilege to only
certain users?).
o The creator of the group is it's first (and initially only) administrator.
o A group's administrator(s) can add and remove users at will, and
sets the privileges for each user.
o A group's administrator(s) can edit the group's metadata (e.g. it's
name), and edit any business rules we support for groups.
USER PRIVILEGES
o Within a group, each user has one of the following privilege levels:
-- Read. (Let's you read messages in the group's pool)
-- Read and Write. (adds the ability to post messages)
-- Read, Write and Administer (adds all admin privileges)
o The above model establishes three roles: those of viewer,
participant and administrator.
o Any user having at least READ privileges against a group has
PERMISSION to see ALL messages in the group's pool.
ADMINISTRATOR PRIVILEGES
o Anyone with admin privileges is an administrator. Thus a group can
have as many admins as desired.
o There's no concept of "ownership", only of "control". All
administrators have equal control, and the creator of the group can
drop off the set of administrators to pass control on to others.
o Over time, organizations will inevitably want finer-grained admin
privileges. Do we want to try to build them in from the beginning?
For example:
-- Moderate group (lets user delete messages?)
-- Delete group (lets user destroy the group)
-- Add/remove read and/or write users
-- Add/remove, promote/demote, admin users
MESSAGE POOLS
o Before posting a message you must choose a group. This defines the
set of users who will have permission to read the message.
o By default, all users are members of the Public group, the Public
group is the only group that is required for an ESME server, and thus
in the simplest case all messages are posted to the public pool and
accessible to everyone.
o Every message posted to a group goes into that group's message
pool; which is the set of all messages posted to/within that group.
o Which messages a given user sees depends upon what the user has
chosen to see (who he/she follows, what tags he/she tracks, etc.),
and/or what he/she searches for.
PRIVACY
o So each message exists in one and only one pool, and only users of
that pool's group have permission to access that message.
o Therefore by choosing a group before you post a message you are
choosing the scope of privacy for that message; because (a) ALL
members of that group will have a way (by searching, tracking,
following, etc.) to SEE that message, (b) NO users outside that group
will ever be able to see that message.
JUMPING THE WALLS OF PRIVACY
o Note that since a given message can only exist in one pool over its
lifetime, if you want multiple groups to recieve the same message you
must post it to each of those groups separately. From an intuitive,
end-user standpoint this appears to be a case of posting the same
message to multiple groups, but to the system each posting is a
separate logical entity (each has it's own unique ID, even if the
text of each is the same).
o So if you see a message in one pool that you want users of another
pool to see, you have to "copy" it from the one and "paste" it into
the other.
o This gives us a simple, safe privacy model, yet makes it easy and
convenient for users to circumvent the rules when necessary. It
takes the burden off the system to manage access control of
cross-pool messaging, and puts the burden of responibility on the
user.
ACCESS CONTROL
o Thus the fundamental access control model is: (a) create groups of
users that need to communicate with each other, (b) post messages to
the appropriate group, (c) the system will keep your messages private
to the group.
PRIVATE, INTERPERSONAL MESSAGING
o I think we also need to make it possible to have private
conversations between any two users without having to formally set up
a group. It should be easy to make it so that you choose a user in
the UI as the "group", post a message, and see all the messages in
this ad-hoc, private pool.
o But then, of course, the two users are going to want to bring
others in on some conversations, so maybe it would be better to make
private, ad-hoc group messaging a property of conversations.... I
don't yet have a suggestion for how to handle this anticipated
end-user desire.
____________________________________________________________
STREAMS, TRACKS, SEARCHES, VIEWS
o OK, so I've suggested that: messages be clustered into pools, each
pool has a wall of privacy around it that ony allows users in it's
group to "fish" for messages from that pool, and that a given user
may be a member of multiple groups and thus have access to multiple
pools of messages.
o So how to we convert the user's permissions against this
multi-pooled dataspace into useful and usable views?
HOW ARE MESSAGES STORED?
o I'm assuming that messages are basically records in a relational
database, and therefore that creating end-user views is
(mechanically) a matter of running a query and formatting the output.
STREAMS
o Most of the time the view a user wants is an ordered list of
messages meeting certain criteria. I call these lists "streams".
o In various contexts I have heard the use of terms like "timeline",
"mailbox", "tracking", "conversations", etc. In all cases the common
element is that they all consist of a query against the set of
messages, a results set that is limted by a user's access controls, a
sort order (usually reverse-chronological order), and the results
displayed as a list of messages. I would call all of these streams.
o Here are some common streams:
-- The Public timeline.
-- A conversation.
-- A track (a timeline defined by your tracking criteria).
-- The messages directed (via @yourname) to you.
-- Replies to your messages.
-- Each group would have its own timeline.
-- The set of messages sent by users you follow.
SYSTEM-DEFINED STREAMS
o The system should create certain predefined streams for you on
demand. You should simply be able to ask to view, and get a stream
for:
o The messages in a selected group's pool.
o The messages sent by a selected user.
o The messages with a seleted tag.
o The messages posted by users you follow.
o All messages in the Public pool.
o The messages you've sent.
o etc.
ACCESS CONTROL IN STREAMS
o In all cases, each user only gets to see those messages that their
access privileges allow them to see.
o For example, if you follow someone who posts to groups of which you
are not a member, you will never under any circumstances be able to
see THOSE messages. On the other hand, any messages posted to the
Public group (of which you ARE a member), and posted to groups of
which you are a member; you'll be able to see in the streams you
request.
DIRECTED MESSAGES
o So what happens if a poster directs a message (via @username) to a
given user, but does so when posting to a group of which the user is
not a member. In that case, by the rule of group privacy, the
targeted user would never see the message.
FILTERING STREAMS
o We should make it possible (and easy) for users to filter a stream.
o For example, let's say that a given server has only one group
defined so far, the Public group, making all messages in the server
available (in theory) to all users.
o By default, on viewing the public stream, we would display only the
most recent (say) 20 messages, and let the user page back in history
to see previous messages.
o We should also let the user apply ad-hoc filtering criteria to the
stream without having to formally do a search. So the user could ask
to see only messages with a certain tag, and/or by a certain user,
and the stream should re-display with most recent (say) 20 messages
from the now-filtered list. Underr the covers I expect this to be
implemented as an ad-hoc query, taking some search criteria from the
basic stream definition and adding some terms from the user-selected
criteria.
TRACKING
o Tracking seems to me to be nothing more than stored queries that
produce streams. So for example when you say "show me my track of
the ESME tag", the server runs a query against all the groups to
which you have access for messages tagged with ESME, sorts them in
reverse chronological order, and shows you the first (say) 20 results.
SEARCHING
o So far I've described timelines and tracking as ways to generate
streams on-demand by running stored queries. The only real
difference is that at the UI level we are calling them different
things for the user's benefit.
o I think we should also make it possible for users to explicitly "do
a search".
o users should be able to create a query by specifying:
o which group pool(s) to search.
o what tags to look for.
o what date range to search.
o whether files, URLs, etc. are attached.
o what user sent them.
o etc.
And to be able to show the results in various ways:
o list of messages.
o list of files or URLs sent as attachments.
o a graph of message volume over time.
o a chart showing the most/least active users.
o a cloud of tags, or users, or keywords, etc.
CLOUDS
o I think of a cloud as another kind of view onto the results of a
query; although a cloud is special in that for performance reasons
you need to pre-compute much of the results set (e.g. by indexing.).
o So, for example, the cloud of tags used by JohnDoe would be the
precompiled list of tags, filtered by their association with JohnDow.
o Since cloud data has to be pre-compiled, we have to define what
kinds of clouds we're interested in. We currently have tag clouds,
and that seems like it should stay. We currently have word-frequency
clouds, and I'm not sure we should keep this. If we want to have
user-participation clouds, etc. we'll need to say so up front.
CHARTS
o System administrators will need to see charts of different kinds to
monitor system loading, performance, error rates, etc.
o Maybe there are charts that end-users would like to see too. For
example, would group administrators want to see charts of activity
levels over time?
____________________________________________________________
CONVERSATIONS
o My idea for conversations is that if you reply to a message you now
have a two-message conversation: which illustrates the rule that a
conversation is a set of messages linked in a tree-shaped structure,
linked from child to parent by virtue of the child being a reply to
the parent.
o Within any conversation tree there will thus be parents, children
and siblings, and users might want to see any/all of those in
"viewing" a conversation.
o I could see one view of a conversation that is simple a
forward-chronological listing (a stream) of the messages in the
conversation.
o But I could also image users wanting to navigate the tree to follow
the threads of discourse; which means a different UI than just a list
of messages. Perhaps, then, we'd have to open conversations into
their own windows to view them...
____________________________________________________________
MESSAGES, PAYLOADS, METADATA
o So what is a message (and consequently, what can you do with it)?
o To a first approximation, a message is a short text string, posted
by a single user, to a designated message pool.
SYSTEM META-DATA
o But there is also a lot of metadata that's automatically associated
with the message:
-- Sender
-- Posting timepoint
-- Group (or pool)
-- is this a reply?
-- if so, then to what message?
-- etc.
USER-SUPPLIED METADATA
o There is also metadata that the user can add at his/her discretion:
-- tags
-- directed recipients (via @username)
-- expiration timepoint?
PAYLOAD
o I suggest that we also allow users to attach "payloads" to messages.
o One form of payload is a URL. A URL is a long text string, that
if contained in the message body would make the message too long. So
we should make it possible to select a run of text in the message to
use as the "name" of the URL, then to define the text of the URL
elsewhere. There could be two UIs for this: For the novice, let the
user type the URL in a separate text box. For the expert, let the
user use a special syntax to type the URL inline with the message
body.
o Another type of payload would me an email address...
o Another type of payload could be a file...
o In all cases we might require that each payload have a run of text
in the message body to act as the name, anchor and access point for
the payload. payload names might be styled to look like HTML links;
and clicking on them might invoke the payload (opening a link in a
new window or tab, opening an email address in the user's mail
client, downloading a file to the user's computer, etc.).
o It should thus be possible for a message to have an arbitrary
number of payloads.
o If we require that every payload have some "name" in the body of
the message, then it should be easy to copy and paste these names
(and thus the payloads) between messages. This would be useful, for
example, when you want to share a file or a link from a message you
received with another user; or to include it in a new message you are
sending out.
FILE PAYLOADS
o My first idea about payloads that are files is that the ESME server
should act like a file store and forward server. As long as there is
any message in any pool that references a given file we should store
it, and allow it to be retrieved whenever a user asks for it.
o Privacy and security is, to a first approximation, enforced by the
group pools: if a file is attached to a message, it can only be
accessed by users in that pool. If a user chooses to copy the name
of (and hence a link to) a file and to paste it into a message posted
to another pool, then it becomes available to that pool too; at the
responsibility of the user.
o One might ask how to remove a file from circulation once it's been
attached to an intial message and uploaded to the ESME server. Once
might ask if the file has an owner, who can get a list of his/her
files and force them to be deleted. Reasonable questions. But I'm
wondering if it makes sense to treat files the same way we do URLs.
Each URL points to a resource (typically a file of SOME sort) on the
internet, and once published there's no way to retract the
distribution of a URL string. So maybe we could treat files the same
way....
o Or we might say that when the original message either expires
(because the sender put an expiration timepoint on it) or is purged
from the system (because there's a server policy in place that purges
all messages older than a certain age), that only then does the file
get deleted from the file store.
TEXT MESSAGES, ETC.
o It's easy for a payload reference (in essence, perhaps just a URL)
to be embedded in the message string itself if the message string is
HTML. If so, then we embed URLs as part of anchor tags and the web
browser automagically displays just the name of the anchor while
carrying with it the full URL string.
o On the other hand, what happens when we forward a message to a
non-HTML-based system; such as cellphone text messaging (SMS)? In
these cases we'd have to strip out the payload, and perhaps just
insert a payload stub -- a special symbol that means "there used to
be a payload attached to this word", etc.
____________________________________________________________
NAMESPACES, COMMAND LINE MESSAGES
o I'd also like us to nail down the at least the basic set of special
characters and commands that can be embedded into messages.
o We use @username to make sure a user sees a message (subject to
access rules and depending on what timeline he/she's viewing).
o We also use #tagname to mark embedded tags.
o It also occurs to me that we could use $servicename to direct a
message to a service (a machine that will respond to our message)...
o Twitter has a number of letter commands, such as "d" at the
beginning of a message to make it private. I remember seeing a long
list of special commands recognized by Twitter, but they seemed: (a)
fragile, and (b) hard to extend the command set.
o I wonder about a command language with a syntax like "p: @fred @joe
here's my message" to cause a private message to be sent to only fred
and joe? Does anyone have a handle on the set of commands we
envision for our command-line language?
____________________________________________________________
____________________________________________________________
--
======================================================================
Bill Fernandez * User Interface Architect * Bill Fernandez Design
(505) 346-3080 * [email protected] * http://billfernandez.com
======================================================================
