Redundant Array of Independent Disks (RAID) is a term used to describe the technique of improving data availability through the use of arrays of disks and various data-striping methodologies. Disk arrays are groups of disk drives that work together to achieve higher data-transfer and I/O rates than those provided by single large drives. An array is a set of multiple disk drives plus a specialized controller (an array controller) that keeps track of how data is distributed across the drives. Data for a particular file is written in segments to the different drives in the array rather than being written to a single drive.

Arrays can also provide data redundancy so that no data is lost if a single drive (physical disk) in the array should fail. Depending on the RAID level, data is either mirrored or striped.

Subarrays are contained within an array subsystem. Depending on how you configure it, an array subsystem can contain one or more sub-arrays, also referred to as Logical Units (LUN). Each LUN has its own characteristics (RAID level, logical block size and logical unit size, for example). From the operating system, each subarray is seen as a single hdisk with its own unique name.

RAID algorithms can be implemented as part of the operating system's file system software, or as part of a disk device driver (common for RAID 0 and RAID 1). These algorithms can be performed by a locally embedded processor on a hardware RAID adapter. Hardware RAID adapters generally provide better performance than software RAID because embedded processors offload the main system processor by performing the complex algorithms, sometimes employing specialized circuitry for data transfer and manipulation.

RAID Levels and Their Performance Implications

Each of the RAID levels supported by disk arrays uses a different method of writing data and hence provides different benefits.

RAID 0 - For Performance

RAID 0 is also known as data striping. It is well-suited for program libraries requiring rapid loading of large tables, or more generally, applications requiring fast access to read-only data, or fast writing. RAID 0 is only designed to increase performance; there is no redundancy, so any disk failures require reloading from backups. Select RAID Level 0 for applications that would benefit from the increased performance capabilities of this RAID Level. Never use this level for critical applications that require high availability.

RAID 1 - For Availability/Good Read Response Time

RAID 1 is also known as disk mirroring. It is most suited to applications that require high data availability, good read response times, and where cost is a secondary issue. The response time for writes can be somewhat slower than for a single disk, depending on the write policy; the writes can either be executed in parallel for speed or serially for safety. Select RAID Level 1 for applications with a high percentage of read operations and where the cost is not the major concern.

RAID 2 - Rarely Used

RAID 2 is rarely used. It implements the same process as RAID 3, but can utilize multiple disk drives for parity, while RAID 3 can use only one.

RAID 3 - For CAD/CAM, Sequential Access to Large Files

RAID 3 and RAID 2 are parallel process array mechanisms, where all drives in the array operate in unison. Similar to data striping, information to be written to disk is split into chunks (a fixed amount of data), and each chunk is written out to the same physical position on separate disks (in parallel). More advanced versions of RAID 2 and 3 synchronize the disk spindles so that the reads and writes can truly occur simultaneously (minimizing rotational latency buildups between disks). This architecture requires parity information to be written for each stripe of data; the difference between RAID 2 and RAID 3 is that RAID 2 can utilize multiple disk drives for parity, while RAID 3 can use only one. The LVM does not support Raid 3; therefore, a RAID 3 array must be used as a raw device from the host system.

Performance is very good for large amounts of data but poor for small requests because every drive is always involved, and there can be no overlapped or independent operation. It is well-suited for large data objects such as CAD/CAM or image files, or applications requiring sequential access to large data files. Select RAID 3 for applications that process large blocks of data. RAID 3 provides redundancy without the high overhead incurred by mirroring in RAID 1.

RAID 4 - Less Used (Parity Volume Bottleneck)

RAID 4 addresses some of the disadvantages of RAID 3 by using larger chunks of data and striping the data across all of the drives except the one reserved for parity. Write requests require a read/modify/update cycle that creates a bottleneck at the single parity drive. Therefore, RAID 4 is not used as often as RAID 5, which implements the same process, but without the parity volume bottleneck.

RAID 5 - High Availability and Fewer Writes Than Reads

RAID 5, as has been mentioned, is very similar to RAID 4. The difference is that the parity information is distributed across the same disks used for the data, thereby eliminating the bottleneck. Parity data is never stored on the same drive as the chunks that it protects. This means that concurrent read and write operations can now be performed, and there are performance increases due to the availability of an extra disk (the disk previously used for parity). There are other enhancements possible to further increase data transfer rates, such as caching simultaneous reads from the disks and transferring that information while reading the next blocks. This can generate data transfer rates at up to the adapter speed.

RAID 5 is best used in environments requiring high availability and fewer writes than reads. Select RAID level 5 for applications that manipulate small amounts of data, such as transaction processing applications.

RAID 6 - Seldom Used

RAID 6 is similar to RAID 5, but with additional parity information written that permits data recovery if two disk drives fail. Extra parity disk drives are required, and write performance is slower than a similar implementation of RAID 5.

RAID 7 - A Definition of 3rd Parties

The RAID 7 architecture gives data and parity the same privileges. The level 7 implementation allows each individual drive to access data as fast as possible. This is achieved by three features:

• Independent control and data paths for each I/O device/interface.
• Each device/interface is connected to a high-speed data bus that has a central cache capable of supporting multiple host I/O paths.
• A real time, process-oriented operating system is embedded into the disk drive array architecture. The embedded operating system "frees" the drives by allowing each drive head to move independently of the other disk drives. Also, the RAID 7 embedded operating system is enabled to handle a heterogeneous mix of disk drive types and sizes.

RAID 10 - RAID-0+1

RAID-0+1, also known in the industry as RAID 10, implements block interleave data striping and mirroring. RAID 10 is not formally recognized by the RAID Advisory Board (RAB), but, it is an industry standard term. In RAID 10, data is striped across multiple disk drives, and then those drives are mirrored to another set of drives.

The performance of RAID 10 is approximately the same as RAID 0 for sequential I/Os. RAID 10 provides an enhanced feature for disk mirroring that stripes data and copies the data across all the drives of the array. The first stripe is the data stripe; the second stripe is the mirror (copy) of the first data stripe, but it is shifted over one drive. Because the data is mirrored, the capacity of the logical drive is 50 percent of the physical capacity of the hard disk drives in the array.
-----------------------------

In case of striping the data is evenly spread across the disks. But
when you add more disks to the volume group you may not be able to
extend your filesystem if it does not satisfy the stripe width.

Summary of RAID Levels

The advantages and disadvantages of the different RAID levels are summarized in the following table:

The most common RAID implementations are: 0, 1, 3 and 5. Levels 2, 4 and 6 have problems with performance and are functionally not better than the other ones. In most cases, RAID 5 is used instead of RAID 3 because of the bottleneck when using only one disk for parity.

RAID 0 and RAID 1 can be implemented with software support only. RAID 3, 5 and 7 require both hardware and software support (special RAID adapters or RAID array controllers).

For RAID 1 and RAID 0+1, the rebuild time increases as the disk size is increased 
but is not affected by how many member disks are in the array. If the array 
consists of   9.1 GB disks with a 16 KB strip size, the following times are
taken to rebuild after a failure if there are no concurrent I/O operations to the array:
Rebuild time 50 min.

DISK_ERR2 (Cd, Disk or R/W Optial operation failure
PERM
Failure in disk assembly hardware (for example power loss)

DISK_ERR3 (CD, disk or R/W optical operation failure
PERM
Error was detected by the SCSI adapter
(The error could have been initiated by something else on the SCSI bus which
affected the disk drives)

DISK_ERR4 (CD, Disk, or R/W optical recovered error)
TEMP
Error caused by a bad block or event of a recorded error

DISK_ERR5 (Undetermined Error)
PERM
SCSI device driver failure of unknown type

An error class value of H and an error type value of PERM indicates that the system encountered a problem with a piece of hardware and could not recover from it….

Disk errors 1, 2, and 4 will return sense data which can be analyzed by the diagnostic programs to provide extra information regarding the nature of the error and its severity.

DISK_ERR 4 is by far the most common error generated, and it is the least severe. It indicates that a bad block has been detected during a read or write request to the disk…

 1,  2,   4,  8, 16, 32, 64, 128, 256,  512, 1024

CREATING A PCI SCSI RAID
Using Command Line
scraidmgr –A –c yes –l (array) –r (raid level) –e (disk ids) –q (queue depth) –g yes
example:
scraidmgr -A -c yes -l scraid0 -r 5 -e "08 0B 0D" -q 8 –f no -g yes
hdisk1 Available
Consistency Check in progress -
1 percent complete
5 percent complete
10 percent complete
.100 percent complete
--------------------------------
Using SMIT

Enter the SMIT menu "PCI SCSI Disk Array Manager" using the smit short cut :-
smit pdam
Select "Create a PCI SCSI Disk Array" and press Enter
Select the PCI SCSI RAID Adapter required and press Enter
Select the RAID Array type required and press Enter
Use ESC+7 to select all the drives required to create the RAID Array 
and press Enter  Initialize Parity

If the parity / mirror (RAID5 / RAID1) is not initialized, then any consistency check run on the array will encounter inconsistencies. It is recommended that parity / mirror be initialized at array creation time. RAID0 is a non redundant array and therefore cannot be initialized. 
Press Enter to continue
When OK and 100% you are complete.

* Copies special array configuration information file to the new drive.
* Recalculates the data and/or parity from the data and parity on the other disks in the array.
* Writes the recalculated data and parity to the new drive.

2) Select reconstruct a PCI SCSI Disk Array and press enter

3)Select the raid array required and press enter 

4) Select the failed drive required and press enter

5)  Press f4 to display a list of drives to be used to reconstruct the array (this is if you have an extra. If you are in the same slot you are selecting the one that was failed…

6) Select drive required and press enter
    It will ask are you sure?

7) Press enter    It will say reconstruction in process…
1 percent complete
3 percent complete
etc
100 percent complete

SCSI ARRAY STATES

OPTIMAL
The Disk Array is operating normally

DEGRADED
A single drive has failed in a RAID 1 or 5 Disk Array, and the unit is now in degraded
mode. Replace the failed drive as soon as possible. Check the drive status to determine
which drive failed. The status will remain degraded until the new drive is reconstructed.

DEAD
The Disk Array is not functioning. Any data which may have been on the array is lost.
Check individual drive status. 

RECONSTRUCTING
The adapter is reconstructing drive data on the new drive.

NOTE: 
One user reported a problem when he defined RAID10 and then put it back
as undefined.... The two disks had the same PVID which had to be
cleared...the error message was 0516-304 mkvg unable to find device id
004213317ebd2b46.. in Device configuration database and 
0516-324 mkvg: Unable to find PV Identifier for physical volume...inconsistent for physical volumes.. 
The solution was to do a
chdev -l hdiskX -a pv=clear 
and
chdev -l hdiskX -a pv=yes 

There is also hardware Raid 0+1 and this can also accommodate
other facilities.

SSA ARRAY STATES

Good State
The array is online and it can be read and written. All the array components are present. 
All parity data (except that affected by recently completed write operations) is 
synchronized. No data or parity rebuilding is outstanding. The array is fully 
protected against the loss of one component.

Exposed State
One component is missing from the array. When the array is read, data can be 
reconstructed for the missing component. The first write operation causes the 
array to enter the "Degraded" state, unless there is no hot spare available that 
can be used to replace the missing component. In the "Exposed" state, the
missing component can be reintroduced or replaced. Then, after any 
necessary rebuilding, the array is returned to the "Good" state.

Degraded State
One component is missing and a write operation has been received for the array. Read and write
operations to the array are supported. However, if power is lost before all the parity data 
has been written, it might not be possible to recreate all the data for the missing component. 
The missing component is permanently excluded from the array.

Note :- While in Degraded state, an array is not protected. If another disk drive in 
the array fails, or the power fails during a write operation, data might be lost.

Rebuilding State
The array is online and it can be read and written. The full complement of array components 
are present but data and parity are being rebuilt on one of the components.

Offline State
An array enters Offline state when two or more member disk drives become missing. Read and write operations are not allowed.

---------------------------------------------------

SSA PHYSICAL DISK ASSIGNMENT

SSA/RAID Setup 

Verify that you have a pci raid adapters
lsdev -Cc adapter

(some of these are for microchannel some for pci...Make sure you 
have all the ones you need for your adapter...)

1) Do an lslpp -l | grep devices.mca.8f97 ( do for each one) 
devices.mca.8f97.com
devices.mca.8f97.diag 
devices.ssa.disk.rte
devices.ssa.IBM_raid.rte
devices.ssa.tm.rte
devices.pci.14104500.rte 
devices.pci.14104500.diag

Make sure there are there at base levels first. Then reboot machine. DO A
BACKUP... smitty mksysb (save off to the side just in case) 

2) Then load the patches….put them in a separate directory like
/tmp/patches. See Document on how to get patches 

3) Install them and then reboot machine 

4) Then we may need to do a 
rmdev -d -l (for each of the ssa logical hard disks if they exist.............
do an lsdev -C | more and look) 

5) rmdev -d -l (for ssa physical disks) and hdisks 

6) rmdev -d -l ssa0 (or whatever it is called) 

7) rmdev -d -l ssar (if it exists) 

8) reboot machine 

9) verify the microcode lscfg -vl pdisk and lscfg -vl hdisk 

10) call hardware and get them to update microcode. Or get off the
web..... 

11) smit ssaraid for non ssa the menu is smitty pdam 

smitty ssaraid 

List All Defined SSA RAID Arrays
List All Supported SSA RAID Arrays
List All SSA RAID Arrays Connected to a RAID Manager
List Status Of All Defined SSA RAID Arrays
List/Identify SSA Physical Disks
List/Delete Old RAID Arrays Recorded in SSA RAID Manager
Add an SSA RAID Array
Delete an SSA RAID Array
Change/Show Attributes of an SSA RAID Array
Change Member Disks in an SSA RAID Array
Change/Show Use of an SSA Physical Disk
Change Use of Multiple SSA Physical Disks 

12) Change Show Use of SSA Multiple Physical Disks 

13) Select disks 

14) On line for new use of ssa disks change to array candidate disks 

15) To make one disk a hot swap you must go back into smitty ssaraid 

16) Change/show physical disk drive. 

17) Select the drive to change and under new use change to hot spare.

18) Go back into smitty ssaraid 

19) Select add an SAA raid array. select your adapter. 

20) Look for field that says enable use of hot spare. Remember if you
did not do this in step 16..do not say true set to false . You will get an
error every hour if you do not set up correctly. 

21) After you run this command…. DO NOT GO FURTHER You must wait
for the parity to build or you will really mess things up…wait time is
anywhere from 20 minutes to several hours. 

22) Go into smitty ssaraid 

23) Select #4 List status of SSA raid 

24) You will see two columns..one is unbuilt and one is unsynced both
numbers will be zero. when it is done. DO NOT try to make a volume
group or anything else before they both are at zero….trust me…you 
will end up doing it over and over again….. Do a mksysb now that 
it is set up.

If you get an error on shutdown that says: /usr/sbin/shutdown[1018]:
/usr/sbin/ssadisk_shutdown: not found. you need to do the following: 
ln -s /usr/lpp/devices.ssa.disk/bin/ssadisk_shutdown
/usr/sbin/ssadisk_shutdown 
---------------------------------- 
Notes on Restore for SSA
1) All member drives and hot spares must be in same loop.

2) It takes a write to the array to start a rebuild. So if the system is only doing
reads, the rebuld will wait until it does a write.

3) If you have a hot spare....It will automatically rebuild itself, provided 
that you do a write and are in the same loop. 

4) When you have a disk replaced and are now putting it in, and the hot spare
has already rebuilt, you can just run cfgmgr and make the new drive a hot
spare. (Change SSA physical disk and select new use...change to hot.spare)
You should see it go from degraded to rebuilding.

5) If you do not have a hot spare or if it was in the wrong loop, you have to do it manually. The drive will probably be rejected from the array. You will then need to do a rmdev -d -l pdisk? (put your number in) 
and then cfgmgr Then change member disks in ssa array. Add a disk to ssa
raid array, and then it should automatically start to rebuild for you..... you
should see it change in status from degraded to rebuilding. 

Identify an SSA Physical Disk

SSARAID -I -l ssa0 -n pdisk0
name
pdisk0
id
000629CA4A3900D
class
disk
use
member
blocksize
512
size
4.5GB
state
good
fastwrite
off

It is impossible to use the system location codes to physically locate SSA disk drives    example:-
lscfg |grep disk
+ hdisk0 04-A0-00-0,0Other SCSI Disk Drive
+ pdisk0 04-08-P 4GB SSA F Physical Disk Drive
+ pdisk1 04-08-P 4GB SSA F Physical Disk Drive
+ pdisk2 04-08-P 4GB SSA F Physical Disk Drive

Also the numeric identifiers of pdisks, hdisks, and the disk drive slots are not related to each other. 

For example, pdisk1 is not necessarily configured as hdisk1 and is not necessarily in slot/bay 1.

In order to physically identify and locate an SSA disk drive, 
there is a utility to "flash" the "check light" of a named
SSA pdisk. 

To start flashing the "check light" on SSA disk pdisk0 :-
ssaidentify -l pdisk0 –y

To stop flashing the "check light" on SSA disk pdisk0 :-
ssaidentify -l pdisk0 –n

Adding a New SSA Physical Disk

cfgmgr
Or use the mkdev command :-
mkdev -c (Class) -s (Subclass) -t (Type) -p (Parent) -w (Connection)
example:
mkdev -c pdisk -s ssar -t 4000mbF -p ssar -w 000629D192FF00D
pdisk3 Available

 Mapping hdisks to pdisks

ssaxlate -l (pdisk name or hdisk name)
example:
ssaxlate -l pdisk2
hdisk1 

ssaxlate -l hdisk1
pdisk2

Changing the Use of an SSA Physical Disk

In the example below, an SSA Physical disk is being changed from "Array Candidate Disk" (default) to "AIX System Disk".

SSARAID -H -l (adapter name) -n (SSA physical disk name) -a (attribute) -d 
-k (system disk name)

examples :-
To change a pdisk to be an "AIX System Disk"
SSARAID -H -l ssa0 -n pdisk0 -a use=system -d -k hdisk1
000629D192FF00D changed 000629D192FF00D attached hdisk1 Available

To change a pdisk to be an "Array Candidate Disk"
SSARAID -H -l ssa0 -n pdisk0 -a use=free
000629D192FF00D detached

The example below has 3 SSA disks and the use of "hot spares" is turned off, and no "hot spare" disk is available. The array can either in the exposed state or in the degraded state depending if the array has been written to since the disk failure occurred.
---------------------------------------

Using Command Line
Add a new disk into the system (replace old and put new in)
Ensure that the new disk is an "Array Candidate Disk".
checking/changing this attribute.

SSARAID -Al (adapter name) -n (RAID Array name) -i exchange -a new_member=(new pdisk name)

example:
SSARAID -Al ssa0 -n hdisk1 -i exchange -a new_member=pdisk3 000629CA4A3900D Changed The SSA RAID Array will now start to rebuild.
Check the array status for rebuild.
---------------------------------------------------------------
Using SMIT
Add a new disk into the system
Adding an SSA Physical Disk"
Ensure that the new disk is an "Array Candidate Disk".
Enter the SMIT screen "Add a Disk to an SSA RAID Array" using 
the smit short cut smit addSSARAID

Note : You cannot enter this smit screen unless there is an SSA RAID Array in the "exposed" or "degraded" state.

This will display the status of all SSA RAID Arrays on the system.
Select the SSA RAID Array required and press Enter.
Move the cursor to the "Disk to Add" field and 
press F4 or ESC+4 (replaced or new disk)
Select the "Disk to Add" and press Enter.
Press Enter again at the "Add a Disk to an SSA RAID Array" screen
The SSA RAID Array will now start to rebuild.
See "SSA RAID MANAGER: RAID Array States" for checking 
the Array status.
(after rebuild)
Enter the SMIT screen "Remove a Disk from an SSA RAID Array" 
using the smit short cut smit redSSARAID
Select the SSA RAID Array required and press Enter.
Move the cursor to the "Disk to Remove" field and press F4 or ESC+4
Select the "Disk to Remove" and press Enter.
Press Enter again at the "Remove a Disk to an SSA RAID Array" screen