SCST iSCSI Target Linux Setup

SCST Module

    Before I begin with just a simple writeup that you’ll follow, let me fill you in on a few things I discovered via my own experiments so you don’t have to waste time duplicating effort. Remember the blogs name is Temporal Solutions and hopefully I’ll save all of you a lot of time. Why SCST and not just IET or another easy iSCSI Target that’s supported by default in the Linux Kernel? We’ll that’s a long and somewhat boring story, but the Linux gods decided to only include IET over SCST for their own reasons. I won’t go into them here, but enough to say that IET currently doesn’t support Persistent Reservations (SCSI3) which is required for VMware and Hyper-V. If you’re going to build a pure Linux solution, let’s say with KVM, you can probably get away with just IET. I’d test the requirements for KVM if you want to go that route. But from what I’ve read, you don’t even need centralized storage for KVM. DRBD works just fine with KVM. That will probably be the next project I work on.

    Plan your project before you start. You don’t want to have to do this step more than a few times. It’s very, very, very time consuming and if you think you’ll be done in 10 min. you’ll be sadly and annoyingly disappointed. That’s why I wrote initially that you want to plan what setup you want to try. You can try all of them to see what works for you, but try not to make more work for yourself.
If you are only going to build two nodes, then definitely start with this step. I’ll show you how to test the setup before you move on. This way you don’t have two unknowns when you try to diagnose what is going on.

   If you are going to build the GlusterFS Storage Nodes / iSCSI Head Units, I’d recommend to start with the iSCSI Head Units. These are the more challenging to get working. The GlusterFS setup is rather easy to setup with the newer release of the product.

    I poured over the internet to find a SCSI Ubuntu setup that worked reliably. I found one at Bauer-Power “How to Install SCST on Ubuntu” which is rather good and works.  Paul Bauer mentions on his site that he had trouble with GlusterFS, but he had used iET and not SCST and GlusterFS had trouble repairing itself when storing large files.  I’ll go into more detail in the GlusterFS module when I get to it.

    My setup was with Ubuntu which is a popular distro, but you can adjust the settings for you’re own distro. Let me tell you, there are as many ways to recompile a kernel in Linux as there are distros. The instructions are based on what is written in iscsi-scst howto. As I find time, I may include instructions for Red Hat.

I’ve updated instructions to only the packages that were required and I just finished testing it on 4/2/13.

This is performed using Ubuntu with a Kernel 3.2.xx
Now with the setup.

The first 5 commands in Bold will perform the following tasks…

Updates repositories used by apt-get after a new install of Ubunut.
At this point I’m still not sure if Kernel-Package is required, but I intend to test further. Subversion installs the svn command so you can get the latest build of SCST.
Installs dependencies required to build linux-image-*, ignore recommended packages.
Upgrades installed packages to current release.
Restart after running upgrade just to play it safe.

#sudo apt-get update
#sudo apt-get install kernel-package subversion
#sudo apt-get build-dep –no-install-recommends linux-image-$(uname -r)
#sudo apt-get upgrade
#sudo shutdown –r now

The following assumes you use /usr/src as the location for the source linux-image-* to be stored.
Also that /root will be used to store SCST.
You are free to choose your own source and SCST install directories.

#cd /usr/src
#sudo apt-get source linux-image-$(uname -r)
#cd linux-X.X.XX
#sudo cp -vi /boot/config-`uname -r` .config

#cd /root
#sudo svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk scst

The following will copy the two patches that are required to the linux source directory. As an alternative you can run the patch command pointing to the original file locations.

#sudo cp /root/scst/iscsi-scst/kernel/patches/put_page_callback-X.X.XX.patch /usr/src/linux-X.X.XX
#sudo cp /root/scst/scst/kernel/scst_exec_req_fifo-X.X.XX.patch /usr/src/linux-X.X.XX

Apply the patches.

#cd /usr/src/linux-X.X.XX
#sudo patch -p1 < put_page_callback-X.X.XX.patch
#sudo patch -p1 < scst_exec_req_fifo-X.X.XX.patch

Now comes the command that will take a very long time to run. The “make && make modules will take a while. In case you are wondering the “&&” means in Linux to run the first command and after it finished run the next command.

#sudo make && make modules

The next two are rather quick. The 2nd command is specific to Ubuntu to rebuild the initial RamDisk.

#sudo make modules_install && make install
#sudo update-initramfs –u

Reboot into the new kernel.

#sudo shutdown –r now

After the reboot install SCST.

#make scst scst_install iscsi iscsi_install scstadm scstadm_install

This is the end for the setup I did. LCMC will control SCST so there is not need for a default /etc/scst.conf file.

SCST Manual Testing

Testing     At this point you have completed the SCST compilation. Before you continue, you’ll want to do a manual test of the SCST install. I’ll show you how to run the test without having to go through the trouble of setting up a scst.conf file. First, we’ll need a binary file to serve as …

View page »

4 comments

1 ping

Skip to comment form

  1. john doe

    “Why SCST and not just IET or another easy iSCSI Target that’s supported by default in the Linux Kernel? We’ll that’s a long and somewhat boring story, but the Linux gods decided to only include IET over SCST for their own reasons. I won’t go into them here, but enough to say that IET currently doesn’t support Persistent Reservations (SCSI3) which is required for VMware and Hyper-V.”
    You need to get certain facts correct
    1. LIO is the scsi target which is included in the kernel and not IET
    2. If you meant distributions, then most include SCST, STGT, IET and newer ones LIO
    3. VMWare uses SCSI RESERVE/RELEASE and not SPC Persistent Reservations. It uses VAAI ATS if available

  2. Marco

    I did a quick lookup of what others have found and I have URLs to point to.
    As to what is included in the Kernel, different version have different specs and my Ubuntu kernel supported IET natively.
    Yes, as distributions get updated SCST may natively become supported. Information always changes, that’s why (for example) medical books get updated regularly with new revisions. There may come a time when IET will support the requirements of VMware. There’s a lot of information out there, not all of it correct. All I was able to determine was that you wanted to use iSCSI SCST with VMWare (at the time I collected my data). Other iSCSI solutions proved problematic to others.

  3. Sachin

    There is an important omission here.

    After patching in the 2 scst patches, and before building the kernel you need to turn on zero_copy callback in the kernel.

    At the command prompt:
    #sudo make menuconfig

    Search for “TCP_ZERO_COPY_TRANSFER_COMPLETION_NOTIFICATION”, navigate to it,

    and then set TCP_ZERO_COPY_TRANSFER_COMPLETION_NOTIFICATION to “y”

  4. Marco

    I never came across that a year ago when I first wrote the articles.
    Thanks for the addition and I’ll have to give it a try.

  1. CentOS 6 with SCST | Qwkhyena's blog…

    Here someone linked a setup for SCSI in CentOS 6 for anyone who’s interested.

    […] http://temporalsolutions.com/linux-ha/scst-setup/ Ubuntu.. […]

Leave a Reply

Your email address will not be published. Required fields are marked *