The results show that the HELIOS UB+ server solutions, in combination with the tested network and server, offer unparalleled performance that nobody has seen before. The total solution costs a fraction of competing file server storage solutions, which cannot offer this end-to-end performance.
Who would benefit from 10 Gigabit networks?
Customers working with large files on the server
-
Customers transferring video files to / from the server
-
Larger servers with many users connected
-
Multi server sites, all servers could be interconnected via 10 Gigabit
-
Customers using proprietary local SAN solutions can switch to standard file server TCP/IP protocols using AFP for Mac clients, and SMB/CIFS for Windows clients
Q & A
Will any 10 Gigabit server offer this great performance?
No. We figured out that depending on the server, the 10 Gigabit adapter, the TCP/IP stack, the operating system, and storage the performance is very different.
Is 10 Gigabit Ethernet faster than Fibre Channel?
10 Gigabit Ethernet offers a 5 times faster bandwidth than 2 Gigabit Fibre Channel and a 2.5 faster bandwidth compared to a 4 Gigabit Fibre Channel. Another question is if the maximum Fibre Channel bandwidth can be utilized by the RAID system and the server / workstations. The Sun X4500 system offers up to 2 GB (Gigabytes) disk I/O performance, which is 5 times better than a 4 Gigabit Fibre Channel. An Apple Xserve RAID offers about 130 MB/sec.
Is a total server throughput of 600 MB/s good?
Yes, this is excellent and represents already 60% of the maximum bandwidth available. This also requires a multi-CPU server, fast storage, and a fast TCP/IP stack.
Is every switch offering 10 Gigabit at this performance?
No. Many switches only offer one or a few 10 Gigabit ports but many 1 Gigabit ports. Usually these switches cannot handle full 10 Gigabit traffic.
Will 10 Gigabit fiber-optic crossover cables work?
Yes. A server can be directly connected with a workstation using a standard LC/LC cable. The LC/LC cable uses swapped transmit and receive wires by default.
Tested Configuration
Server:
-
Sun X4500 (with OpenSolaris 11 B85)
-
16 GB RAM
-
48 disks (total of 24 TB or 48 TB, ZFS striped)
-
10 Gigabit Sun card (driver: ixgb)
-
4x 1 Gigabit ports
-
Less than 1 € (Euro) per GB
Clients:
-
2x Mac Pro (2x 2.8 GHz Quad-Core)
-
10 Gigabit fiber-optic Ethernet adapters (Small Tree: PETG1F-XF-SR)
-
3x Mac (Intel) OS X 10.5
-
1 Gigabit onboard Ethernet
Network:
-
24x 10 Gigabit ports switch from Arista 7124S with SFP+ fiber-optic SR adapters
(Arista 7148S offers 48x 10 Gigabit ports)
-
2x 10 Gigabit Mac Pro client connected with the switch using fiber-optic cables
-
3x 1 Gigabit dedicated lines to 3 Mac Intel clients
-
10 Gigabit testing has been done with jumbo frames using an MTU of 9000. Using the default of 1500 will reduce the performance by about 30% on Mac OS X. This can be changed in the “Network” control panel. On Solaris, the MTU is specified for this “ixgb” driver in “/kernel/drv/ixgb.conf”, add the line “default_mtu = 9000;”
-
10 Gigabit connection has been realized with 10 Gigabit multimode fiber-optic links
Software:
1x HELIOS UB+ Server Solutions Suite
Benchmarking has been performed based on HELIOS UB+, with
HELIOS LanTest 4.5.0 which is capable of 10 Gigabit Ethernet.
Note: The company formerly known as Arastra has changed its name to Arista Networks.
Glossary
XFP
A 10 Gigabit small form factor pluggable module is called “XFP”. XFP cards and switches do not have a direct cable or fiber connection. Instead, an XFP transceiver is needed, e.g. an XFP optical transceiver with duplex LC fiber connector or an XFP copper-CX4 adapter. Some cards do not require an XFP and use an on-board fiber or copper connector.
SFP+
Same as XFP but with a very small form factor. The Arista switch uses the more advanced SFP+.
LC optical fiber-optic cable
A fiber-optic cable connector type that is used with SR Ethernet ports. Some cards already have SR connectors that allow using LC cables. Some other cards require an XFP-SR adapter to use LC fiber-optic cables. The LC cable has the benefit that both the receive and send cables are settled in a fixed connector. For the HELIOS 10 Gigabit testing we used LC/LC cables from Belkin.
10 Gigabit media types
Type
|
Description
|
10GBASE-CR
|
10 m over copper twinax
|
10GBASE-LR
|
10 Km over SMF
|
10GBASE-LRM
|
220 m over OM1 / 2 / 3 MMF
|
10GBASE-SR
|
300 m over OM3 MMF
|
1000BASE-LX
|
10 Km over SMF
|
1000BASE-SX
|
275 m over OM1 MMF
|
1000BASE-T
|
100 m over UTP
|