This design should be reasonably simple to implement and maintain, has isolation for storage and the DMZ and excellent throughput for Fault Tolerance traffic.
For this design the physical switch uplink ports should all be configured to use PortFast.
Both the storage uplink ports and the virtual switch used for storage should be set to use Jumbo Frames by specifying an an MTU of 9000.
In this design trunking should be configured on all uplinks. Trunking at the physical switch will enable the definition of multiple allowable VLANs at the virtual switch layer.
* 1 - Distributed virtual switches require Enterprise Plus licenses. If you do not have Ent+ then replace these switches with standard virtual switches.
* 2 - Route based on physical NIC load is a policy only available in a dvSwitch. If you do not have Ent+ then use the default policy instead.
* 3 - VLAN tagging for the storage network is not required as it is isolated but is still a good idea as it keeps networking configuration consistent and is not very hard to implement.
* 4 - Datastore path selection policy is typically best set at Round Robin, but always consult the storage vendor documentation to confirm the best type to use.
vSphere 5 - 10 NICs IsolatedStorage and IsolatedDMZ Design v1.0.jpg
Comments, feedback and suggestions welcome.