CS161 Programming Assignment 4: IMAP Client + jython

Preliminaries

Questions: cs161tas@cs.brown.edu
Help sessions:
    Chris - XXX
    Ron - XXX

Due date: Wednesday, November 2nd, 10pm.

Add this line to your shell startup scripts:

source /course/cs161/startups/cs161student

Make sure ~/.jython contains:

python.console=org.python.util.ReadlineConsole
python.console.readlinelib=GnuReadline
python.cachedir=/home/$USER/jpackages

Introduction

Your fourth programming task is to extend you IMAP client to allow better for complex configuration and scripting by adding an embedded language: jython

Your goal in this assignment is to develop a client that will be useful when you develop and test your (future) IMAP server. As such, your focus should be on flexibility, rather than performance. However, since one use of your client will be to simulate high loads, performance does matter, and you will, once again, use an event-based architecture to support large numbers of concurrent connections. To do so in Java, just like the last assignment, you'll use the non-blocking sockets (SocketChannel) from Java's NIO package and an event loop implented over these sockets (by a Selector). Here's the link to NIO Basics.

Your client will run commands from a file, as in Assignment 1, with a couple changes. First, you will support many new commands (see below). Second, you will support some level of concurrency even when issuing commands to a single server.

Your client must accept, as its sole argument, the name of a file to be executed as jython code. When run without an argument, your program should start an interactive interpreter.

Starting code for this assignment is at /pro/web/web/courses/cs161/lang/.

Assignment Specification

  1. You will implement "Robots" that exercise IMAP servers. These robots will be created, configured, started, and stopped in jython. The configuration should be rich enough to create robots that exhibit the behaviour you expect in various types of clients - periodic checkers, download/deleters, hoarders etc.
  2. You should turn in "performance.jy" that shows how response times are distributed as the number of clients (robots) using a single server grows.
  3. You should also implement a jython command for each IMAP command you have implemented. These commands should return tuples, the first element of which should be the complete text returned by the command. The remaining elements should containg the "natural" return values of the command. For example, the names of folders returned by the LIST command. These commands will wait for command completion before returning. You would use these commands to script interesting data dependant interaction with a server, but they are unsuitable for scripting "robots".
  4. You should turn in a new "conformance.jy" script that your client can run to determine interesting facts about IMAP implementations. You should no longer need a separate script to interpet results, you can do it in jython. Furthermore, you should be able to script more interesting interactions, since you can make decisions based on the results of commands. There are RFC "SHOULD"s and "MUST"s. It is your decision on what specific things to test. For example, you might want to test if the IMAP server understands hierarchal seperators properly.
  5. Your explanations of both scripts, and design choices you made should be placed in a README file.

Commands

  1. You should implement support for the following IMAP4 commands: CAPABILITY, NOOP, LOGOUT, LOGIN, SELECT, EXAMINE, CREATE, DELETE, RENAME, LIST, STATUS, APPEND, CLOSE, SEARCH, FETCH, COPY. That is, a user of your program should be able to exercise each of these commands on the server.
  2. You need not implement a pretty interface to SEARCH. Just accept an arbitrary string to serach on. Users of the command might choose to say something like:
    search("FLAGGED SINCE 1-Feb-1994 NOT FROM \"Smith\"")
  3. You may implement only primitive support for FETCH. You need only support FETCHes for "FULL".
  4. Your client should be able to keep state when required, for example in commands like SELECT. It would also be immensely helpful in your conformance testing script.
  5. Your jython commands should be structured, for example, your list command:
    1. Should take arguments instead of just a string of text to send:
      list <reference-name> <mailbox-name (regexp)>
    2. Should return a list of folders, not just a block of text:
      list results:
      <foldername1> <properties>
      <foldername2> <properties> ...
  6. You don't have to deal with UID commands, or parts of required commands utilizing UIDs.
  7. Your commands, when needed, should parse IMAP responses containing grammar such as "astring" in order to return reasonable results as jython objects.

    These would be the ones you most often run into. It is by no means a comprehensive list, so feel free to parse more if you feel the client should.

Output

We do not expect your client to produce output that conforms exactly to a super specific format, but your client should produce human-readable, structured, parsed responses for all commands you issue to the server, and every response the server produces. If the server doesn't respond, you should notify the user that something is wrong, and what you think the problem is.

For your scripts, performance.py should at least produce a table stating your measurements. Your table should show how response times are distributed as the number of clients (robots) using a single server grows. You should also either produce it in a format that is immediately scriptable by gnuplot/xgraph/jgraph, or should include a short script to graph your output. Graphs should be labeled properly. You are free to produce more meaningful statistics and measurements, and we will reward you for your extra effort, but make sure the TAs can easily read them.

For conformance.py, you should print out every criteria you have tested against and its result.

For example you might have:

conformance:
   ... 
  <COPY command>[sequence set syntax]    {Fail: Bad response}
      Request   "A003 COPY 2:4 MEETING" produced
      Response  "A003 OK COPY completed"
      Problem   Not all messages copied. 3:4 not copied.
  <COPY command>[nonexistant copy]       {Fail: Timeout}
      ...

Remember, document all your design decisions in your README. It would help both yourself and the TAs while grading your assignment.

Testing

You may use the same IMAP servers as in previous assignments. Don't hammer them! Just show a performance trend, not the limits of their ability.

You should also read RFC 2683 which contains advice for IMAP developers (clients and servers). It will help you write a better client, but more importantly, it will give you ideas for variant behavior that your client can provide in order to test your server.

Submitting your code

To submit your code, run the following script:

/course/cs161/bin/cs161_handin lang [dirname]

where [dirname] is the location of your code (defaults to the current directory).

Please contact the TAs if you have problems submitting. For your own sanity, don't leave your submission until the minute before the deadline.

Grading scheme

Your code will be graded on the following factors (and their weights).

Documentation and code legibility: 20%
Functionality (non-blocking, error reporting, etc): 50%
Robustness (malicious input, many connections, etc): 30%