Before I begin with just a simple writeup that you’ll follow, let me fill you in on a few things I discovered via my own experiments so you don’t have to waste time duplicating effort. Remember the blogs name is Temporal Solutions and hopefully I’ll save all of you a lot of time. Why SCST and not just IET or another easy iSCSI Target that’s supported by default in the Linux Kernel? We’ll that’s a long and somewhat boring story, but the Linux gods decided to only include IET over SCST for their own reasons. I won’t go into them here, but enough to say that IET currently doesn’t support Persistent Reservations (SCSI3) which is required for VMware and Hyper-V. If you’re going to build a pure Linux solution, let’s say with KVM, you can probably get away with just IET. I’d test the requirements for KVM if you want to go that route. But from what I’ve read, you don’t even need centralized storage for KVM. DRBD works just fine with KVM. That will probably be the next project I work on.
Plan your project before you start. You don’t want to have to do this step more than a few times. It’s very, very, very time consuming and if you think you’ll be done in 10 min. you’ll be sadly and annoyingly disappointed. That’s why I wrote initially that you want to plan what setup you want to try. You can try all of them to see what works for you, but try not to make more work for yourself.
If you are only going to build two nodes, then definitely start with this step. I’ll show you how to test the setup before you move on. This way you don’t have two unknowns when you try to diagnose what is going on.
If you are going to build the GlusterFS Storage Nodes / iSCSI Head Units, I’d recommend to start with the iSCSI Head Units. These are the more challenging to get working. The GlusterFS setup is rather easy to setup with the newer release of the product.
I poured over the internet to find a SCSI Ubuntu setup that worked reliably. I found one at Bauer-Power “How to Install SCST on Ubuntu” which is rather good and works. Paul Bauer mentions on his site that he had trouble with GlusterFS, but he had used iET and not SCST and GlusterFS had trouble repairing itself when storing large files. I’ll go into more detail in the GlusterFS module when I get to it.
My setup was with Ubuntu which is a popular distro, but you can adjust the settings for you’re own distro. Let me tell you, there are as many ways to recompile a kernel in Linux as there are distros. The instructions are based on what is written in iscsi-scst howto. As I find time, I may include instructions for Red Hat.
I’ve updated instructions to only the packages that were required and I just finished testing it on 4/2/13.
This is performed using Ubuntu with a Kernel 3.2.xx
Now with the setup.
The first 5 commands in Bold will perform the following tasks…
Updates repositories used by apt-get after a new install of Ubunut.
At this point I’m still not sure if Kernel-Package is required, but I intend to test further. Subversion installs the svn command so you can get the latest build of SCST.
Installs dependencies required to build linux-image-*, ignore recommended packages.
Upgrades installed packages to current release.
Restart after running upgrade just to play it safe.
#sudo apt-get update
#sudo apt-get install kernel-package subversion
#sudo apt-get build-dep –no-install-recommends linux-image-$(uname -r)
#sudo apt-get upgrade
#sudo shutdown –r now
The following assumes you use /usr/src as the location for the source linux-image-* to be stored.
Also that /root will be used to store SCST.
You are free to choose your own source and SCST install directories.
#sudo apt-get source linux-image-$(uname -r)
#sudo cp -vi /boot/config-`uname -r` .config
#sudo svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk scst
The following will copy the two patches that are required to the linux source directory. As an alternative you can run the patch command pointing to the original file locations.
#sudo cp /root/scst/iscsi-scst/kernel/patches/put_page_callback-X.X.XX.patch /usr/src/linux-X.X.XX
#sudo cp /root/scst/scst/kernel/scst_exec_req_fifo-X.X.XX.patch /usr/src/linux-X.X.XX
Apply the patches.
#sudo patch -p1 < put_page_callback-X.X.XX.patch
#sudo patch -p1 < scst_exec_req_fifo-X.X.XX.patch
Now comes the command that will take a very long time to run. The “make && make modules will take a while. In case you are wondering the “&&” means in Linux to run the first command and after it finished run the next command.
#sudo make && make modules
The next two are rather quick. The 2nd command is specific to Ubuntu to rebuild the initial RamDisk.
#sudo make modules_install && make install
#sudo update-initramfs –u
Reboot into the new kernel.
#sudo shutdown –r now
After the reboot install SCST.
#make scst scst_install iscsi iscsi_install scstadm scstadm_install
This is the end for the setup I did. LCMC will control SCST so there is not need for a default /etc/scst.conf file.