Distributed Systems - Summer 2010

Project Information

Project overview

In this sequence of labs, you'll build a multi-server file system called Yet-Another File System (yfs) in the spirit of Frangipani. At the end of all the labs, your file server architecture will look like this:

You'll write a file server process, labeled yfs above, using the FUSE toolkit. Each client host will run a copy of yfs. yfs will appear to local applications on the same machine by registering via FUSE to receive file system events from the operating system. The yfs extent server will store all the file system data on an extent server on the network, instead of on a local disk. yfs servers on multiple client hosts can share the file system by sharing a single extent server.

This architecture is appealing because (in principle) it shouldn't slow down very much as you add client hosts. Most of the complexity is in the per-client yfs program, so new clients make use of their own CPUs rather than competing with existing clients for the server's CPU. The extent server is shared, but hopefully it's simple and fast enough to handle a large number of clients. In contrast, a conventional NFS server is pretty complex (it has a complete file system implementation) so it's more likely to be a bottleneck when shared by many NFS clients.

Lab assignments

Lab 1 - Lock Server

Lab 2 - Basic File Server
Lab 3 - File Server: Reading, Writing and Sharing Files
Lab 4 - MKDIR, REMOVE, and Locking
Lab 5 - Caching Lock Server
Lab 6 - Caching Extent Server + Consistency
Lab 7 - Paxos
Lab 8 - Replicated lock server
Lab 9 - Project

Collaboration Policy

You must write all the code you hand in for the programming assignments, except for code that we give you as part of the assignment. You are not allowed to look at anyone else's solution. You may discuss the assignments with other students, but you may not look at or copy each others' code.

Programming Environment

You should be able to do Lab 1 on any Unix-style machine, including your own Linux/FreeBSD desktops, or MacOS X laptops.

For Labs 2 and beyond, you'll need to use a computer that has the FUSE module, library, and headers installed. You should be able to install these on your own machine by following the instructions at fuse.sourceforge.net; we outline instructions on Ubuntu/Debian machines. Most other Linux installations and many other Unix-like systems should work as well. However, the official environment for the labs is a standard Debian 5 stable installation on a x86-machine, as provided in a VirtualBox image wedescribe below.

Note that you if you have your own FreeBSD or MacOS machines that you prefer to use for programming, you should be able to use them for the majority of the work. However, there are minor annoying differences between FUSE on Linux and FUSE on other operating systems that may cause your code to fail our tests when it seems to pass for you. As an example, on Linux FUSE passes file creation operation to the file system using the MKNOD call, while on other systems it uses CREATE. Please ensure that your assignment passes the tests in the official environment, and there shouldn't be any problems when we test it.

Installing FUSE on Ubuntu/Debian

You may install FUSE on your own computer, though it is not necessary to do so. (You can also use the VMWare image described below).


To setup FUSE on your local machine, you will need the header files and the utilities. Install them like this:

sudo aptitude install libfuse2 fuse-utils libfuse-dev

Next, you need to make sure the fuse module is loaded:

alex@x61:~$ ls -l /dev/fuse 
crw-rw-rw- 1 root fuse 10, 229 2010-02-11 06:02 /dev/fuse

If your ls output matches the above output, then you can skip the modprobe step. If you do not see the above output, try running modprobe:

sudo modprobe fuse

Finally, you need to add yourself to the fuse group to be able to mount FUSE file systems:

sudo useradd -G fuse {your_user_name}

Make sure to logout of your current shell to refresh your group list. Typing groups at the command line should show fuse as one of the groups:

alex@josmp:~$ groups
alex users fuse admin

Linux VirtualBox image with FUSE

We've created a VirtualBox machine image (1.5 GB) running Debian 5 that contains everything needed for developing and testing the lab code. VirtualBox can be obtained for free from here. There is one non-root user set up in the system with the username ds and password ds. The root password is set to root.

How can I copy files from the host machine into the VM or vice versa?
Probably the easiest way to achieve this is to use scp inside the virtual machine. This requires ssh access to the machine to which the files should be copied to/from. The host machine can be accessed from inside the VM with the IP address
Alternatively, data can be shared by accessing a shared network volume (SMB or NFS) from inside the virtual machine.

Aids for working on labs

There are a number of resources available to help you with the lab portion of this course:

All the labs use the POSIX threads API (pthreads). A comprehensive guide to programming with pthreads can be found here: http://www.llnl.gov/computing/tutorials/pthreads/

The labs use the FUSE interface to plug the lab file system into the operating system. See the FUSE website for more information.

printf statements are always your friend when debugging any kind of problem in your programs. However, when programming in C/C++, you should always be familiar with gdb, the GNU debugger. You may find this gdb reference useful. Below introduces a few gdb tips for complete newbies:

If your program is crashing (segmentation fault), type gdb program core where program is the name of the binary executable to examine the core file. If you don't find the core file anywhere, type ulimited -c unlimted befor starting your program again. Once inside gdb, type bt to examine the stack trace when the segmentation fault happened.

While your programming is running, you can attach gdb to it by typing gdb program 1234. Again, program is the name of the binary executable. 1234 is the process number of your running program. Of course, you can choose to run your program with gdb from the beginning. If so, simply type gdb program. Then at the gdb prompt, type run.

While in gdb, you can set breakpoints (use gdb command b) to stop the execution at specific points, examine variable contents (use gdb command p), etc.

To apply a given gdb command to all threads in your program, prepend thread apply all to your command. For example, thread apply all bt dumps the backtrace for all threads.

Check out the GDB manual for full documentation.

W. Richard Stevens' books ``UNIX Network Programming'' Volume 1 and 2 are classic references for network programming. If you are struggling with the sockets interface it could be a helpful purchase. See the suggested books list for other helpful references.

Questions or comments regarding this course? Please use the general course mailing list or the teaching staff mailing list.

Top // Distributed Systems Summer 2010 //