On May 11, Matthias Felleisen wrote:
>
> On May 11, 2009, at 6:21 AM, Eli Barzilay wrote:
>
> > but this was an extreme example of "live coding", much more
> > stressful than what's usually associated with that term...
>
> To wit, it was so live that Eli was still writing the test while the
> students were taking it :-]
I forgot to include the important bits of the README file, see below.
One of the advantages I listed for doing this is that you can come to
the exam with only a sketch for some question, and write them fully as
the exam goes... In my case, I was busy hacking on the server and
client for the first ~1-2 hours, and then I wrote two question. It
was fun.
----------------------------------------------------------------------
This is a server and a client to run a paperless test. The clients
will run in "kiosk mode" -- locking out all other applications, which
is intended to prevent students from chatting, looking up stuff on the
web etc. This is currently working only on Windows. The server can
run in any platform, but it is slightly more convenient if it is
unix-like (linux or osx), preferably inside Emacs.
Why?
----
Running a paperless exam has some major advantages:
* When you get back from your exam you don't need to schlep a pile of
papers. You don't need to count them every five seconds to make
sure that they're all there. And you also don't need to worry about
leaving the pile of graded exams on the roof of your car and realize
later that you forgot them there, then spend three hours scanning
the roads and sidewalks.
* You don't need to deal with student handwriting (sometime even
intentionally obscured -- I've seen cases where students who didn't
know the answer wrote text that could barely be read).
* You can provide the students with additional materials, so they
don't need to kill half of a tree because your exam is open
materials. (However, this is limited to text files.)
* You can update files during the test: clarify questions, add hints
etc. You can even have a rough sketch of some questions and write
them up as the test goes on...
* You can provide "template files" for the students to modify. For
example, you can give them some function and ask to fix a bug.
* Grading is often a repetitive job, and I found myself starting with
writing detailed comments, and later on (when the 10th exam has the
same mistake) settle on an explanation-less cross and other obscure
markings. When you edit files, it is easy to copy+paste the same
comment.
* Grading can be more consistent if you're doing such copy+paste that
includes the grading. In addition, it is easy to grep for a
particular error that you've seen to make sure that the grading is
consistent.
* Obviously, you can easily come up with a textual markup system so
that you compute the exam grades automatically from the graded
files. I'm using a system like that for homework submission and
grading, and having exams in electronic formats fits nicely into
this system.
* The same holds for distributing the works. You can keep the
original pristine files in case students want to complain -- no need
to worry about pencil-written exam being modified post-grading, and
you can have the graded files available through your usual channels
(eg, through the PLT handin-server).
Features
--------
The server and the client were designed to be *very* robust and
secure, while being convenient to use.
* Server-client communication is done through an encrypted connection.
The server can be setup to accept connections only from some known
IP addresses (and will report attempted connections from other IPs).
* Clients do not have a "save" or a "submit" button -- the text is
continuously sent back to the server for storage. This frees
students from dealing with such details while they're stressed for
time.
* Clients do *not* have a Scheme REPL. This wouldn't be too hard to
add, but I think that this would be bad for several reasons:
students can easily get dragged to a long debugging session instead
of focusing on the test; you can get dragged to asking tougher
questions since the students can now actually try things out; and
you can get dragged to harsher grading ("if it doesn't work, you
don't get the points", versus "you got the rough idea right, so you
get the points" which is common with written exams and make them
less stressful).
* The server backs up the directories of all clients periodically, to
make things safer, and to avoid "I accidentally hit delete in the
last minute" excuses.
* Clients use a number of tricks to make the application run in "kiosk
mode": making it nearly impossible to switch to another application
or kill the client. There are cases when some of these may fail
(for example, at the NEU lab ctrl+alt+delete cannot be disabled) --
the client has a safety mechanism against this: if you hit
ctrl+alt+delete it will notice that it's not the focused window, and
then it will (a) alert the server, and (b) set itself back into
focus.
* You can set up a master password that can unlock a client. (Hit
ctrl+shift+P, then the password, then enter.) This is another
safety feature in case of some server catastrophe: in this case you
can manually unlock a station and copy the written solution out to a
different application.
* The server keeps a live connection with all clients and polls them
frequently. This is done for two reasons:
- if the client dies (either due to a bug, or to a break attempt),
the server will detect and report it,
- on each poll the client reports sends the server the updated
contents of editable buffers to save.
* The client can withstand being disconnected from the server (eg, if
the server is restarted for some reason) -- it will alert the
student, but will continue to operate as usual. (Specifically, the
student can continue reading files and writing answers during such
downtimes.) Changes are not lost -- so when a connection is
re-established, synchronization continues as usual.
* You can update client-visible files on the server -- the changes
will be sent to all clients, and the clients will visually indicate
modified files so students can easily see that they need to read
something.
* The server can require users to login to be able to edit files. In
this case, the files that the student writes will be saved in a
directory with the username. It can also work in a login-less mode,
so each client corresponds to a single student working; in this
case, you will need to make students write their names in their
answer file(s). (This sounds insecure, but works perfectly fine in
practice: as long as a responsible person is present in the room,
making sure that people do not switch stations.) You can also
require logins without having a password file: and login students on
the machine using the master password which will work with any
username, and you can even do so remotely from the server prompt.
(This is also useful if some students forgot their password.)
* If the server is set up to require logins, the client is still
functional before such login is successful -- but it will not show
editable files. This is useful, for example, if you have twice the
number of machines than the number of students: each student can
then use one machine to write their answers, and another machine to
read the exam material.
* Clients are identified with a unique client id (which you setup when
the clients are started, or leave the default as the IP number), and
after logging in, they are identified by usernames. These must be
unique, so no user can login on two clients at the same time (but if
a machine crashes, it is possible to have the student move to a
different machine). In our lab, I set each client id to correspond
to the physical location of the machine, so it is later possible to
know if two students were sitting next to each other. (When there's
suspected copying, for example.)
* If a client is disconnected, and later reconnected, it will show the
same content, including files that were modified by the logged-in
username (if logins are required) or files that were modified on the
client with the same client id -- so no edits (beyond the few
seconds of poll frequency) are lost on a client crash.
* The server controller can send messages to all clients. The message
will flash at the bottom of each client to avoid a student missing
it. The message is also logged in a "messages" buffer, so students
can read these messages later again. The controller can also send a
message to a single client.
* Clients can send a message to the server, which is done with a
chat-like interface. Together with the above, it is possible for
students to ask you questions privately, and for you to announce a
message back for the whole class. (For example, a student asks a
question, you realize that further clarification is important so you
edit a file or add a new one, and tell everyone about it.) It is
also possible to disable this feature in the configuration file.
* The server controller has extensive control over the clients through
several commands (more below). It is possible to freeze all clients
(in case you want to make sure that students listen to what you're
saying) or a specific one; force all clients (or a specific one) to
switch to the messages window (in case you want to make sure they're
reading some messages that you're sending); unlock a client (or all)
remotely (similar use case as the master password); restart a client
(or all) (eg, to recycle the machine for someone else); or kill a
client (or all) (which is useful at the end of the test).
* The clients can show the time -- which you can configure to be a
normal clock, or a countdown to a specific time. You can also
change this time at the server prompt, including to any random
(short) text (for example "1min!"). In all cases, the displayed
time is controlled by the server, so a station with a bad internal
clock will not cause problems.
* Many more aspects of both the server and the clients are
customizable. All the way down to indicator colors.
--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://www.barzilay.org/ Maze is Life!
_________________________________________________
For list-related administrative tasks:
http://list.cs.brown.edu/mailman/listinfo/plt-dev
