BOSS1 Documentation

Terminology

We are acutely aware that the terminology to describe the concepts involved in an educational institution can vary. The meanings used by BOSS are as follows.

Course

A programme of study leading to a qualification (such as "BSc Computer Science"). A student is a person registered on one or more courses.

Module

A unit of study, for which an examination board might require marks. A student may be registered on individual modules.

Manager

A member of staff responsible for a module, and the named individual through which formal communications about the module will be directed. marks.

Assignment

A module will be assessed by means of one or more pieces of work (such as essays, programming assignments, examinations). An assignment is such a piece of work characterised by a deadline. We also specify two other date/times - a start date, before which students cannot submit work, and an end date after which submissions are not possible.

Component

An assignment may have multiple constituent components (think of examination rubric like "Answer question 1, either question 2 or question 3, all questions from part B, and any three from part C"). Each component may be a single thing, or may be subdivisible into further components.

Problem

A component which does not subdivide. A "single question", an essay, or a programming task, for example.

Marking Scheme

A specification of criteria (mark categories) to which marks may be individually assigned, for a single problem. This includes relative weightings for the criteria.

Marker

A person authorised to assign marks to a collection of students by following a marking scheme set by the module manager.

Moderator

Following marking of submissions for a problem by one or more markers, a moderator is a person authorised to finalise those marks as directed by the module manager.

User Categories

Users of BOSS fall into five categories. All users of BOSS must be members of the educational institution, that is, they must have an ID number which uniquely identifies them (in addition to a name).

Student

A person who would normally be registered for a course course, and for one or more modules. BOSS allows a student to submit files which constitute submission requirements for parts of a module. Optionally students may be able to run automatic tests on files submitted prior to actual submission (a "sanity check" especially useful if the files contain computer programs).

Administrator

A member of staff responsible for administration of BOSS, with global access rights to the data. A specific function of an adminstrator is to preconfigure new modules for BOSS, before the module manager adds data.

Manager

A member of staff responsible for an individual module. Normally this would mean the Lecturer in charge, but this is not prescribed. They have access rights to all data for that module, and a rich range of administrative tools. BOSS allows more than one manager to be assigned to a module.

Marker

A person responsible for marking one specific problem in an assignment. Normally a member of staff, BOSS does not exclude the possibility of (for example) a postgraduate student taking on such a rôle. They would allocate marks against specific criteria defined by the manager, and would not be aware either of the identity of a student being marked, or the relative weightings of the marking criteria. A marker might also be empowered to author feedback to students. For each assignment component one or more markers would be chosen, and each student would be assigned one or more markers from the marker pool at random.

Moderator

A person responsible for taking the marks allocated by markers for a component of an assignment, and finalising the marks for each student. The moderator might be the same person as the manager, but not necessarily. More than one moderator might be assigned to a given component.

Automatic Tests

BOSS was originally written in response to management and marking requirements for programming modules. The current version of BOSS allows automatic tests to be run on submitted work, using two different paradigms.

1: Text Input and Output

The original version of BOSS ran under UNIX, where each program would run with input from a standard input stream (and other files), with output sent to two streams - the standard output stream and the standard error stream - and possibly other files also. This allowed automatic tests to be specified as text input versus text output; a student program would be run with the given input stream, and the actual output would be compared with expected output using UNIX utilities such as diff (or hand-coded shell scripts). Although versatile, depending on the skill of the lecturer, it has several difficulties, including the necessity of being sensitive to the treatment of whitespace, control characters (including end-of-line markers) and spelling, when devising tests.

This mode of operation is still available, and may be used if BOSS is to be used for non-programming assignments.

2: Input and Output as Objects

Suppose a programming problem in the Java language can be specified as:

• a class CI which is the "input";
• a class CO which is the "output";
• a method m with signature CO m(CI)
• .

Then a test can be defined by:

• an object OI, an instance of class CI;
• an object OO, an instance of class CO, such that m(OI) returns OO.

A student's implementation of method m can then be tested by checking
m(OI).equals(OO).

The toString() method is used by BOSS to display the results of such a test within the dialogue, and naturally the equals() method may be overridden if so desired.

Since this paradigm does not specify how input or output should be presented, the lecturer may wish to provide students with the I/O code as a "template", or to leave the students to write their own I/O code.

Currently, this method of testing is provided in BOSS through integration with the JUnit framework for writing unit tests.

Submitting Work

The following is the sequence of actions that take place when a student submits work for an assignment.

1. Student "logs on" to BOSS.
2. Student chooses module.
3. Student chooses assignment for that module.
4. Student chooses problem for that assignment.
5. Student chooses file(s) they wish to submit.
6. (Optionally) student can run one or more automatic tests on the files.
7. Student confirms the choice of files to be submitted.
8. BOSS archives the files and copies them across to a data store; the submitted data is explicitly checked for integrity.
9. BOSS creates receipt (containing hash codes for all the files submitted) which is emailed to the student.

Marking Work

The following is the sequence of actions that take place when an assignment is marked.

1. For each individual problem within the assignment:
   1. Manager selects marking categories and weightings for that problem (including automatic tests, if any).
   2. BOSS runs automatic tests on all submissions.
   3. Manager selects markers and moderators.
   4. BOSS randomly allocates markers to individual students.
   5. Each marker marks the students they have been allocated (including editing a feedback file).
   6. Moderators finalise marks for each student.
   7. Manager authorises final marks and (optionally) mails marks and/or feedback to students.
2. Manager authorises final mark grid for the assignment.

Module Management

A manager for a module can perform the following administrative tasks.

• Author marking categories and relevant weightings for each component of an assignment, including automatic tests (if any).
• Select dates and times for the following events:
  • Start date for an assignment;
  • Deadline for an assignment;
  • End date for an assignment.
• Select policy on late submissions.
• Authorise deadline extensions for individual students.
• Mail feedback and/or marks to students.
• Allocate markers and moderators to individual problems.
• Authorise final mark grid for the assignments and for the whole module.
• Run plagiarism detection software.

The manager is the only person who would normally see marks in a de-anonymised form.

Feedback to Students

Feedback may be returned to students in three ways:

1. Marks for each problem.
2. A file created and edited by the markers, moderator and manager. This file is "free format" and is presented as a simple editable text box.
3. Automatic feedback can be generated. BOSS will assign a word value to a student's mark for a category. This gives students valuable information on how they were marked.
4. A global feedback function is also provided for use in a similar way to the individual feedback. This allows the course manager to provide some general feedback on the whole body of submissions for an assessment

Data Protection

Users of BOSS in the EU are reminded of the provisions of the UK Data Protection Act 1998 and other national legislation implementing EU directives. We believe that the software itself complies with the legislation; however use of the software places certain duties on the user, and allows students a right of access to the data held on them by BOSS.

Installation

Preamble

The BOSS Online Submission System is provided in two forms. One form contains a .jar file. This file is actually an installation program which can install the servers and clients in a directory of your choice and commence changing the settings files. The second version is a copy of our CVS tree used to build the installer and also to build the servers and clients for use in a production environment. Building the code from source is not difficult (Ant is used to script the entire task) but does take a little longer to set up.

Installation Preparation

Server:

BOSS requires at least one machine to run its servers on which should be able to access the database BOSS uses. It is recommended that you run the database on the same system as the servers although this does not necessarily have to be the case.

Database:

The BOSS Online Submission system currently uses a MySQL database, though previous versions have used PostgreSQL. Scripts are provided for setting up tables in either database, and database-specific code has been avoided wherever possible. It is possible to use a different database altogether, provided referential integrity is supported, but no guarantees can be made.

To prepare for database installation, check if a suitable database is installed in your system and make sure an appropriate JDBC driver is available. If a database is not installed, you can download one from http://www.mysql.com. JDBC connector drivers should be supplied with your Database installation.

Java VM (aka Java Runtime Environment)

You must have a copy of a Java VM/JRE compatible with Sun version 1.4 or higher installed. If you do not, you can download one from http://java.sun.com/j2se/. BOSS is developed and tested with Sun JDK 1.4.2.

Supporting classes

BOSS requires the following extra classes to run:

• Java Mail (mail.jar) http://java.sun.com/products/javamail/

• Java Activation (activation.jar) http://java.sun.com/products/javabeans/glasgow/jaf.php

• JUnit (junit.jar) http://www.junit.org/index.htm

• Tar file support (com.ice.tar.jar) http://www.gjt.org/pkgdoc/com/ice/tar/

If you've manually downloaded any of these, and don't have administrator privileges, you'll need to put the .jar files in the "lib" directory of BOSS once it's been installed. If they have different names to the expected ones, you may also need to alter the settings.sh file in the bin directory.

Web interface

BOSS features a web-interface written in JSP and Java. To use this you need a Java Servlet container installed. The BOSS development team use Tomcat (http://jakarta.apache.org/tomcat/). Setting up a servlet container is outside the scope of this document, however a variety of set-up guides should be available from the Servlet container authors.

For information on how to install this, have a look at the howToInstallWebClient file once BOSS is installed.

Build from source

Ant (http://ant.apache.org/) is the build tool of choice for most Java projects, because it's written in Java! It allows us to specify build rules just like a Makefile, but in XML format. It also allows for a variety of plugins including JavaCC, JUnit and IzPack. Ant will need adding to your PATH after installation.

Our installer is built with a tool called IzPack (http://www.izforge.com/). IzPack will need adding to your PATH after installation. You will also need to copy compiler.jar to Ant's lib directory (not the BOSS libs directory).

Install Instructions

Precompiled 'JAR' release

1. Download the BOSS-install.jar file from http://sourceforge.net/projects/cobalt

2. Run the installer with $ java -jar BOSS-install.jar. We will refer to your chosen installation directory as [installdir].

3. Follow the on-screen instructions. You may wish to install all components to begin with and run the clients and the servers from the same directory. Once you are happy with the operation of BOSS you can use the same installer to just install the clients or the servers.

Build from source

Once you have installed all the necessary supporting code you are ready to compile BOSS. To build everything:

$ cd [installdir]/src
$ ant

Ant will find the build.xml file in that directory and execute the all target. This will build all of the code from scratch. If you wish to perform BOSS's self-tests, you can run

$ ant test

Note that these tests are in progress and failures do not necessarily indicate a problem with your code. They are a development tool to ensure refactored code does not change its external functionality.

Once BOSS has been built, various jar files will have been placed in the [installdir]/src/jars directory.

Now you can build the installer with:

$ ant installer

which will produce a file called BOSS-install.jar. Run this file like this:

$ java -jar BOSS-install.jar

Now continue as per the binary release instructions. Note that you will need to copy the third-party libraries from the development [installdir]/src/libs directory to the new "lib" directory created by the installer. Also note that you are not restricted as to which directory or even machine you can install BOSS onto after compilation.

Database Installation

You can connect to a database either locally or over a network. However, it is recommended that you run the database on the same system as the server.

• Create a new database using your database's "create database" command.

• Import the table definitions by using your database's SQL client to run the contents of the appropriate script in [installdir]/sql.

• Now your new database should contain several tables including member, administrator and year_of_study. To ensure the correct running of the database, you must add a user into the member table and add the same user as an administrator in the administrator table. You should also add at least one record of current year into the year_of_study table.

  To do this you can run populate.sql in [installdir]/sql. This contains templates for the code needed to add the correct data to the database, but may require editing to suit your details. It's automatically filled with the details you provide at install time, assuming you've installed the "SQL Scripts" pack.

• Ensure that the database is functioning and can be accessed from the local system.

• Ensure that the database can also be accessed from the machine where server is running.

Properties modifications

You may now edit the configuration files within BOSS. There are comments included in each properties file to help you on editing. If you're just running everything from one directory on one machine, this is probably unneccessary - the install script should fill in everything needed using data supplied by you. However, if you made a mistake, want to change the config, just want to see what's there, or if your .jar files for the third party libraries are in an unexpected location, here's what you should edit:

• boss.properties in [installdir]/data/settings.

  Note: Make sure slave_server is set to equal to localhost.

• cobalt.properties in [installdir]/data/settings.

  The database_url variable is in the format of:

  database_url=jdbc:[software]://[host]/[database]
       ?user=[username]&password=[password]

  (But all on one line.)

  Note: Make sure the database-name variable is assigned to the new database you have created.

• settings.sh, base.settings.sh and (for older releases) slave.settings.sh in [installdir]/bin to match your system.

  These files are used as config files to start the BOSS servers. They should be fine, but you may need to edit the CLASSPATH settings in settings.sh.

Starting the Programs

• RMI Registry

  Assuming the above steps have been completed correctly, you should make sure an rmiregistry is currently running:

  $ rmiregistry &

• Interface servers

  Now you should be able to run the servers by running the runStaff and runStudent scripts from [installdir]/bin (you'll need to be inside the bin directory for them to run). Note that all output will be redirected to the log files located in [installdir]/data/logs. If the last log entry reads "staff server startup successful" then you have successfully started the staff server.

  To run the student server then simply run the runStudent script in [installdir]/bin and similarly, for the slave server run [installdir]/bin/runSlave.

• Clients

  You should now be able to run the staff client by running staff in the bin directory, and the student client by running student from the same place.