In our last episode we left our Security Administrator simmering in the fact that Virtual SAN is like many other Infrastructure-Consumer Solutions that rely on the Network Infrastructure to provide Security. Luckily she has seen this movie before. Matter of fact, it used to be the norm that the Physical Network would provide almost all of the Security for Applications. Thus there are well known ways to provide Network Security for Virtual SAN.
As a reference note, it is hard to separate the roles/services of Networking and Security when talking about the physical IP infrastructure. For shortness (and convenience) the physical IP infrastructure is referred as the Physical Network, and the Physical Network provides both roles/services to Infrastructure-Consumer Solutions. As a second reference note, anything that uses the Network's Data Plane is considered an Application. So Virtual SAN is an Application from the Physical Network perspective...and so is iSCSI, NFS, FCoE...
...and now back to the our regular programming...
Before you can provide any type of Network Security we need to enlist the help of a Network Administrator and you need to understand the Application's behavior.
In this instance our Security Administrator is already well-versed in the matters of Networking (yes, she is that talented and for that her job title shall henceforth be Network Security Administrator), so we are good there. As for our Application, Virtual SAN, we know the following:
1) It uses a combination of IP Unicast and IP Multicast to communicate.
2) The Application works across Layer 3 boundaries.
3) The Application end points (the Virtual SAN nodes) are known.
4) The Application uses known port numbers.
So using this information, below is the Network Security Administrator's recommendation for providing Network Security for Virtual SAN with the goal of restricting unauthorized access to the Virtual SAN Datastore.
There are two strategic concepts for executing Network Security over Layer 3: route obscurity and end-point access restrictions. Route obscurity is achieved by limiting the number of routers that know of an end-point's subnet. If a router has no path to a destination, the packets for that destination will be dropped. End-point access restriction is achieved with your old fashioned Access List (or its few derivates). Identify whom end-points need to chat with each other and on what ports, and only allow those conversations.
Elver's Opinion: If you can achieve 100% route obscurity then you may not need to employ end-point access restrictions. However, it is VERY challenging to obtain 100% route obscurity...unless you don't need to talk to anyone outside of your subnet.
Layer 2 Only Deployment
Virtual SAN was originally designed to only work in Layer 2 (meaning, you couldn't get a VMware supported deployment of Virtual SAN over Layer 3 boundaries). Thus in this deployment all Virtual SAN nodes will have the Virtual SAN VMkernel interface in the same subnet/VLAN (which should not be VLAN 1). There is absolutely no need for the Virtual SAN nodes to communicate over the Virtual SAN VMkernel interface with an entity not in the Virtual SAN VLAN. Thus the only security considerations you need to cover for are these:
1) Ensure that only ESXi hosts that are part of the Virtual SAN cluster have access to the Virtual SAN VLAN.
2) Limit the diameter of the Virtual SAN VLAN to those Access switches that have Virtual SAN ESXi hosts connected and any Distribution switches providing a Data Path between the Access switches.
Route Obscurity
The Virtual SAN subnet does not need to be advertised to the rest of the Network. A subnet that is not advertised to the rest of the Network is a called an isolated subnet. Isolated subnets are the easiest to protect. It is not possible to reach an end-point, via IP, in an isolated subnet if you are not already part of the isolated subnet.
End-Point Access Restriction
You should restrict connectivity to the Virtual SAN VLAN to only the ESXi hosts that are part of the Virtual SAN cluster. Since most Data Center switches will have their interfaces in a default VLAN different from the Virtual SAN VLAN, this becomes an Operations exercise with some regular audits.
If you want to get really protective about it, you could use MAC Access Lists as well. However, experience has demonstrated that managing MAC Access Lists is very cumbersome and is not worth the trade off when compared to the added level of security that you get in return.
You may also be tempted to create Community Private VLANS. Elver's Opinion: Don't do it. If you lock down connectivity to the Virtual VSAN VLAN to only those ESXi hosts in the Virtual SAN cluster, Private VLANs will buy you nothing. Absolutely nothing...You would get more value out of pounding sand. Plus, you would be adding unnecessary complexity to the deployment.
Layer 3 Deployment
With Virtual SAN 6.0, you can deploy multiple ESXi hosts in the same Virtual SAN Cluster in different Virtual SAN subnets. This is a bit more involved compared to a Layer 2 only deployment, however it is doable and manageable as long as some planing is put in place first.
Route Obscurity
Each Virtual VSAN subnet, per Virtual SAN cluster, must be identified (you don't want to be making Virtual SAN related Network and Security changes every 3 months). VMware only supports the Virtual SAN VMkernel port in the Default TCP/IP stack, which is used by the Management VMkernel interface.
To ensure egress traffic goes out the correct VMkernel interface, create static routes to the Virtual SAN subnets (in each Virtual SAN node) pointing out to the Virtual SAN subnet's default gateway. You should NOT place the Virtual SAN VMkernel port in the same subnet as the Management VMkernel port. Would it work if you place them in the same subnet? Of course it would work...as much as your New Year resolution. It'll be fun for a bit but then it would get old fast and the reality of ensuring Security is not breached between entities in the same subnet that shouldn't talk to each other will then sink in...it is not worth it.
If you only have a single router (with multiple interfaces) acting as the default gateway for all Virtual SAN subnets, then you have nothing to advertise as the Virtual SAN subnets would be unreachable to anyone without a direct connection to one of the router subnets (similar to a Layer 2 Only Deployment).
However if two or more routers are the default gateways (not counting the VRRP scenario), then you have to figure out how to announce to all those routers (and the routers in between on a need to know basis) how to reach the other Virtual SAN subnets. Depending on the Physical Network layout, you may need to advertise the Virtual SAN subnets via a routing protocol. In such a case, consider using a routing protocol (such as BGP) that supports route filtering and use those filters to advertise the Virtual SAN subnets to only those routers that need them.
If you have a small number of routers, consider using static routes in the routers to reach the Virtual SAN subnets. With static routes you can manually control which routers have a Data Path to the Virtual SAN subnets.
Alternatively, if the routers support VRFs, create a Virtual SAN VRF. Today almost all routers (and Layer 3 switches) that are deployed in the Data Center support VRFs. Make the Virtual SAN default gateways CEs/PEs, and restrict the Virtual SAN VRF to the smallest feasible diameter within the confines of the Data Center.
End-Point Access Restriction
Since we are aware of the Virtual SAN subnets, it is relatively straightforward to provide some end-point security for the Virtual SAN nodes. In the Virtual SAN default gateways, put Access Lists only permitting traffic sourced from the Virtual SAN subnets, and the Virtual SAN port numbers, destined other Virtual SAN subnets and the Virtual SAN Multicast Groups. Also if using a RP, consider restricting membership to only the Virtual SAN Multicast Groups.
If the Virtual SAN subnets must be advertised widely in the Network, create Access Lists in the entry points where the Data Center Firewalls are located. If you have the ability to do so, move these Firewalls as close as possible to the Virtual SAN subnets. No entity outside the Data Center should be allowed to reach the Virtual SAN subnets (on any port) nor be the source or listener of the Virtual SAN Multicast Groups.
Elver's Opinion: None of these Network Security suggestions would matter much if you don't have physical security for the Physical Network and the vSphere Infrastructure.
.elver
No comments:
Post a Comment