Now Playing: co-workers talking
Topic: work
I should really be posting here more because I do something at work every day that could be posted as a tip of some sort. Usually it is something small that no one ever tells you which takes hours to figure out. Then once you know it, you'll probably never need it again so who cares?
Today I decided to split the internal Debian mirror at work off to its own machine. So I needed install a new VM with Debian 6 (Squeeze) (my OS of choice btw). I wanted to have a small root partition of 10Gb and then use iSCSI to connect to an EqualLogic SAN volume of 150Gb. I currently mirror Etch, Lenny, and Squeeze. I also plan to move my Unattended install there as well. I just sync the main repo on Lenny and Squeeze and no sources. The Etch mirror has sources but no non-free. All of this takes up 111Gb.
Enough about all that. Created the volumes/datastores needed on the EqualLogic box. I installed Debian to the smaller volume and installed open-iscsi. I did a discovery right away just to see if everything was showing up correctly with
iscsiadm -m discovery -t sendtargets -p <SAN ip>
A huge list of LUNs flew by because the SAN has multiple LUNS. I saw the one I needed in the list. I needed to configure CHAP so nano /etc/iscsi/iscsid.conf. I uncommented these lines and put in my CHAP info.
node.session.auth.authmethod = CHAP
node.session.auth.username = username
node.session.auth.password = password
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = username
discovery.sendtargets.auth.password = password
Now I may not need CHAP setup for discovery because I was able to discover just fine without CHAP. I was getting an authentication error when trying to connect so I added this. Ultimately I think my error might have been caused by an ESXi server connecting to the LUN.
I also set "node.startup = automatic" because I want this to connect on boot.
I tried to connect to the lun with
iscsiadm -m node --targetname <iqn> --portal <SAN ip> --login
I had some error messages as mentioned before. I made changes to the config file and restarted the open-iscsi daemon. When I did this open-iscsi started to connet to all the previous iqn's it found. Some of wich allow multiple connections (for vMotion to work). A quick fdisk -l showed that indeed I was connected to the volumes. I disconnected from all the volumes and edited the config again. I changed from automatic to manual. I restarted the daemon and BOOM all the nodes tried to connect again! So I'm thinking that the manual/automatic isn't working here.
I went into the /etc/iscsi/nodes folder and deleted all the nodes there. I then tried to figure out a way to do a discovery with only the node I needed to come up. I couldn't find any way to do that. So I just did another discovery. I then used this command in the /etc/iscsi/nodes folder to remove all the nodes discovered except the one I needed (which has the word he-storage).
ls | grep -v "he-storage" | xargs rm -Rf
This removed the many volumes I didn't need and left me with the one I wanted. So I then connected to that LUN successfully and proceded to partition, format, and mount the volume. I figured that now I should change back to automatic loading of the LUN. I edited the file and restated the daemon. It showed a successful disconnect but no re-connect. WTF? I double check the config and restart the daemon, same thing. So now I'm NOT getting the behavior I need when it was happening before.
I then remembered seeing a list of settings under the node folder. If you cat the iqn folder under the /etc/iscsi/nodes folder, you can tab complete out till you hit a file called "default". Once you cat "default" you see all the settings for that node. This basically looks like the config file without the comments. Right up toward the top was "node.startup = manual". It seems that the settings for the node are pulled from the config file when they are discovered, but if the config file is updated those changes won't take effect for any existing node. I edited "default" and restarted the daemon. The node came up automatically without issue.
The -R, --rescan for iscsiadm, may be a sollution to this problem, but I didn't look into it since I finally had everything working.
So there it is. The big discovery of the day was that under open-iscsi, nodes that you discover will take the settings from iscsid.conf at the time they are discovered and don't seem to update. So if you need to make changes that will effect these nodes you will need to edit the "default" file or dig more into the man page of iscsiadm.