Learn Sun InfiniBand Switch

On Exadata X2-2, Oracle use "Sun InfiniBand Switch".
How to learn about it? We can find out Sun Datacenter InfiniBand Switch 36 Documentation and learn from them.
- On Operating System on it:

# cat /etc/redhat-release
CentOS release 5.2 (Final)

# uname -r
2.6.27.13-nm2

- Check the status of the management controller, power supplies, fans, and switch chip:

# showunhealthy
OK - No unhealthy sensors

# env_test
Environment test started:
Starting Voltage test:
Voltage ECB OK
Measured 3.3V Main = 3.30 V
Measured 3.3V Standby = 3.39 V
Measured 12V = 12.06 V
Measured 5V = 5.00 V
Measured VBAT = 3.11 V
Measured 2.5V = 2.50 V
Measured 1.8V = 1.80 V
Measured I4 1.2V = 1.22 V
Voltage test returned OK
Starting PSU test:
PSU 0 present
PSU 1 present
PSU test returned OK
Starting Temperature test:
Back temperature 32.5
Front temperature 33.1
SP temperature 37.1
Switch temperature 52, maxtemperature 58
Temperature test returned OK
Starting FAN test:
Fan 0 not present
Fan 1 running at rpm 11627
Fan 2 running at rpm 11627
Fan 3 running at rpm 11847
Fan 4 not present
FAN test returned OK
Starting Connector test:
Connector test returned OK
Starting onboard ibdevice test:
Switch OK
All Internal ibdevices OK
onboard ibdevice test returned OK
Environment test PASSED

and other samples:

# showtemps
Back temperature 32.5
Front temperature 33.1
SP temperature 36.1
Switch temperature 52, maxtemperature 58
All temperatures OK

# checkpower
PSU 0 present status: OK
PSU 1 present status: OK

# getfanspeed
Fan 0 not present
Fan 1 rpm 11847
Fan 2 rpm 11736
Fan 3 rpm 11847
Fan 4 not present

- Check the link status of the InfiniBand cables:

# listlinkup
Connector 0A Not present
Connector 1A Not present
Connector 2A Not present
Connector 3A Not present
Connector 4A Not present
Connector 5A Not present
Connector 6A Not present
Connector 7A Not present
Connector 8A Present <-> Switch Port 31 is up (Enabled)
Connector 9A Present <-> Switch Port 14 is up (Enabled)
Connector 10A Present <-> Switch Port 16 is up (Enabled)
Connector 11A Present <-> Switch Port 18 is up (Enabled)
Connector 12A Not present
Connector 13A Present <-> Switch Port 09 is up (Enabled)
Connector 14A Present <-> Switch Port 07 is up (Enabled)
Connector 15A Present <-> Switch Port 05 is up (Enabled)
Connector 16A Present <-> Switch Port 03 is up (Enabled)
Connector 17A Present <-> Switch Port 01 is up (Enabled)
Connector 0B Not present
Connector 1B Not present
Connector 2B Not present
Connector 3B Not present
Connector 4B Not present
Connector 5B Not present
Connector 6B Not present
Connector 7B Not present
Connector 8B Present <-> Switch Port 32 is up (Enabled)
Connector 9B Present <-> Switch Port 13 is up (Enabled)
Connector 10B Present <-> Switch Port 15 is up (Enabled)
Connector 11B Present <-> Switch Port 17 is up (Enabled)
Connector 12B Present <-> Switch Port 12 is up (Enabled)
Connector 13B Present <-> Switch Port 10 is up (Enabled)
Connector 14B Present <-> Switch Port 08 is up (Enabled)
Connector 15B Present <-> Switch Port 06 is up (Enabled)
Connector 16B Present <-> Switch Port 04 is up (Enabled)
Connector 17B Present <-> Switch Port 02 is up (Enabled)

*** "Not present" status is no cable attached or the link at either end of the cable is down.
and check status port

# getportstatus 31
Port status for connector 8A Switch Port 31
Adminstate:......................Enabled
LinkWidthEnabled:................1X or 4X
LinkWidthSupported:..............1X or 4X
LinkWidthActive:.................4X
LinkSpeedSupported:..............2.5 Gbps or 5.0 Gbps or 10.0 Gbps
LinkState:.......................Active
PhysLinkState:...................LinkUp
LinkSpeedActive:.................10.0 Gbps
LinkSpeedEnabled:................2.5 Gbps or 5.0 Gbps or 10.0 Gbps

- Verify the InfiniBand Fabric
ibdiagnet - Performs diagnostics upon the InfiniBand fabric and reports status.
ibnetdiscover - Discovers and displays the InfiniBand fabric topology and connections.
ibcheckerrors - Checks the entire InfiniBand fabric for errors.

# ibcheckerrors
## Summary: 14 nodes checked, 0 bad nodes found
## 62 ports checked, 0 ports have errors beyond threshold

Other Links:
Sun Datacenter InfiniBand Switch 72 Documentation