Distributed Systems -
Summer 2010
Project Information
Project overview
In this sequence of labs, you'll build a multi-server file system called Yet-Another File System (yfs) in the spirit of Frangipani. At the end of all the labs, your file server architecture will look like this:
You'll write a file server process, labeled yfs above, using the FUSE toolkit. Each client host will run a copy of yfs. yfs will appear to local applications on the same machine by registering via FUSE to receive file system events from the operating system. The yfs extent server will store all the file system data on an extent server on the network, instead of on a local disk. yfs servers on multiple client hosts can share the file system by sharing a single extent server.
This architecture is appealing because (in principle) it shouldn't slow down very much as you add client hosts. Most of the complexity is in the per-client yfs program, so new clients make use of their own CPUs rather than competing with existing clients for the server's CPU. The extent server is shared, but hopefully it's simple and fast enough to handle a large number of clients. In contrast, a conventional NFS server is pretty complex (it has a complete file system implementation) so it's more likely to be a bottleneck when shared by many NFS clients.
Lab
assignments
Lab 1 - Lock Server Lab 2 - Basic File Server You
must write all the code you hand in for the programming assignments, except for
code that we give you as part of the assignment. You are not allowed to look at
anyone else's solution. You may discuss the assignments with other students,
but you may not look at or copy each others' code. You should be able to do Lab 1 on any Unix-style machine, including your own
Linux/FreeBSD desktops, or MacOS X laptops. For Labs 2 and beyond, you'll need to use a computer that has the FUSE
module, library, and headers installed. You should be able to install these on
your own machine by following the instructions at fuse.sourceforge.net; we outline
instructions on Ubuntu/Debian machines. Most other
Linux installations and many other Unix-like systems should work as well.
However, the official environment for the labs is a standard Debian 5
stable installation on a x86-machine, as provided in a VirtualBox image we� describe below. Note that you if you have your own FreeBSD or MacOS machines that you prefer
to use for programming, you should be able to use them for the majority of the
work. However, there are minor annoying differences between FUSE on Linux and
FUSE on other operating systems that may cause your code to fail our tests when
it seems to pass for you. As an example, on Linux FUSE passes file creation
operation to the file system using the MKNOD call, while on other systems it
uses CREATE. Please ensure that your assignment passes the tests in the
official environment, and there shouldn't be any problems when we test it. You
may install FUSE on your own computer, though it is not necessary to do so.
(You can also use the VMWare image described below). To
setup FUSE on your local machine, you will need the header files and the utilities.
Install them like this: Next, you need to make sure
the fuse module is loaded: If your ls output matches the above output, then you can skip the modprobe step. If you do not see the above output, try running modprobe: Finally, you need to add
yourself to the fuse group to be able to mount FUSE file systems: Make sure to logout of your
current shell to refresh your group list. Typing groups at the command line
should show fuse as one of the groups: We've
created a VirtualBox machine image (1.5 GB)
running Debian 5 that contains everything needed for developing and testing the
lab code. VirtualBox can be obtained for free from here. There is one
non-root user set up in the system with the username ds and password ds.
The root password is set to root. There are a number of
resources available to help you with the lab portion of this course: All
the labs use the POSIX threads API (pthreads). A comprehensive guide to programming
with pthreads can be found here: http://www.llnl.gov/computing/tutorials/pthreads/
The
labs use the FUSE interface to plug the lab file system into the operating system.
See the FUSE website for more
information. printf statements are always your friend
when debugging any kind of problem in your programs. However, when programming
in C/C++, you should always be familiar with gdb,
the GNU debugger. You may find this gdb reference
useful. Below introduces a few gdb tips for complete newbies: If
your program is crashing (segmentation fault), type gdb program core where program
is the name of the binary executable to examine the core file. If you don't
find the core file anywhere, type ulimited
-c unlimted befor starting your program again. Once inside gdb,
type bt to examine the stack
trace when the segmentation fault happened. While
your programming is running, you can attach gdb to it by typing gdb program 1234. Again, program is the name of the binary
executable. 1234 is the process
number of your running program. Of course, you can choose to run your program
with gdb from the beginning. If so, simply type gdb program. Then at the gdb prompt, type run. While
in gdb, you can set breakpoints (use gdb command b) to stop the execution at specific points, examine variable
contents (use gdb command p),
etc. To
apply a given gdb command to all threads in your program, prepend thread apply all to your command. For
example, thread apply all bt
dumps the backtrace for all threads. Check
out the GDB manual
for full documentation. W.
Richard Stevens' books ``UNIX Network Programming'' Volume 1 and 2 are classic
references for network programming. If you are struggling with the sockets
interface it could be a helpful purchase. See the
suggested books list for other helpful references. Questions
or comments regarding this course? Please use the general course mailing list
or the teaching staff
mailing list.
Lab 3 - File Server: Reading, Writing and Sharing Files
Lab 4 - MKDIR, REMOVE, and Locking
Lab 5 - Caching Lock Server
Lab 6 - Caching Extent Server + Consistency
Lab 7 - Paxos
Lab 8 - Replicated lock server
Lab 9 - ProjectCollaboration
Policy
Programming
Environment
Installing FUSE on
Ubuntu/Debian
sudo aptitude install libfuse2 fuse-utils libfuse-dev
alex@x61:~$ ls -l /dev/fuse
crw-rw-rw- 1 root fuse 10, 229 2010-02-11 06:02 /dev/fuse
sudo modprobe fuse
sudo useradd -G fuse {your_user_name}
alex@josmp:~$ groups
alex users fuse admin
Linux
VirtualBox image with FUSE
How can I copy files from the host machine into the VM or vice versa?
Probably the easiest way to achieve this is to use scp inside the
virtual machine. This requires ssh access to the machine to which the files
should be copied to/from. The host machine can be accessed from inside the VM
with the IP address 10.0.2.2.
Alternatively, data can be shared by accessing a shared network volume (SMB or
NFS) from inside the virtual machine. Aids for
working on labs