Quality of Service

Any sufficiently advanced incompetence is indistinguishable from malice.

Archive for the ‘Switching’ Category

Catalyst Campus Design

Posted by qualityofservice on September 2, 2009

Appears to be a work in progress here, but since it flatters my own approach to design where I can get away with it, I consider it worth sharing. ^_^

http://www.cisco.com/en/US/docs/solutions/Enterprise/Campus/campover.html

Among the highlights: “proper” use of the 3750 series Catalyst switches.  Apart from what may be ever-so-slightly higher backplane capacity that will never be used in a my existing $dayJob environment anyway, there is one architectural difference between the 3750 and the 3650, and one architectural difference only: the ability to do a proper switch stack via the StackWise feature.

If one does not plan to stack the switches, there is no reason to shell out the extra money for a 3750 when the 3650 line will do the job more than adequately.  If the money’s already been spent, however, you can do some great things to simplify and optimize your network.  These include reduced sevice requirements (no need to run STP* or HSRP), simplified configuration and manageability (done only once on the master switch in the stack as opposed to two physical switches) and increased availability and load-balancing.  Of course, this precludes physical separation of the switches by distances greater than the longest available StackWise cable; but if its physical separation one wanted, it’s cheaper to do so with the 3560.

By running etherchannels (which can span multiple switches in the stack) to your up/downstream switches, you can make use of multiple links to your “core” stack instead of having the redundant links blocked by STP.  By definition, loops cannot form, reducing complexity associated with STP (especially in multi-vendor environments, with HP defaulting to MSTP, Cisco defaulting to proprietary PVST+, Dell defaulting to god knows what…), and more easily facilitating the spanning of VLANs across access-layer switches – generally not desired for your user-access, but sometimes preferred for server-access where layer-2 adjacencies are preferred for convenience/clustering’s sake.  This is best illustrated on page 18 of the document, with the next page illustrating the benefit in terms of number of active, traffic-passing links vs. an STP design.

Not running HSRP elimates the risk of split-braining, which occurs with physically separated layer-3 switches when they lose connectivity to one another; both switches go active for the HSRP virtual IP address, but return traffic only goes towards one of the two switches, blackholing traffic destined for one of the switches.

And it’s certainly easy enough to do; assuming your existing separate 3xxx series each have links to the same upstream and downstream devices, just take one ‘of ‘em offline, configure the other with a nice high priority (“switch priority 15”), connect the “secondary” switch via the StackWise cables and power it up.  Recommend administratively disabling all the ports until the appropriate EtherChannels have been set up on all switches.

In summation, a quick general-usage guide for the current models of Catalyst switches, all of which come in PoE and non-PoE, and 12/24/48-port flavours of various speeds (10/100/1000); FYI, their QoS architectures and configuration are identical, differing only in buffer pool size and policer granularity (1Mbps policer intervals on the 2960, 8kbps intervals on the 3560/3750):

2960: Dumb layer-2 wiring closet switch.  Replaces the 2950.  No layer-3, no routing protocols, no VRF stuff, nothing.  Can be managed via IPv4/v6.

3560: Smarter layer-3 wiring-closet/distribution/core switch; successor to the 3550.  Can route IPv4 and v6 over all protocols.

3750: Smartest layer-3 1RU switch.  Exactly the same as the 3560, but with the StackWise feature to enable virtual switching — presenting multiple physical switches as a single switch.

3650-E and 3750-E: Same as their counterparts, but more backplane capacity (StackWise backplane doubles from 32 Gbps to 64 Gbps), dual AC power supplies, and support for 10GigE.

Save for the E-series, all of the above feature a single AC power supply, and a proprietary DC power input.  It can be fed redundant power to this DC input via the RPS 675 (EoS) and RPS 2300.  Each RPS can feed up to six different devices; however, the RPS 675 can only actively back up one device.  The 2300 can provide redundant power for up to two.  If you load up 6 switches on a 2300 and all of their internal supplies fail, four of the switches are out of luck.

As far as dual AC supplies go (again, save for the E-series), one has to take a large step up to the 4500 and 6500-series chassis.  Each features the potential for in-box power and supervisor redundancy.  4500 is relatively (of the two) low-cost choice for port-density and minimal features; 6500 fully loaded can hold most if not all of the internet routing table, all manner of line-cards for all manner of port types, and a variety of service modules (load-balancing and SSL-terminating ACE modules, firewall modules, network analysis modules, etc), and a ton of features.  Not likely to see deployment in $dayJob any time soon, but the datacenter aggregation switch of choice in large environments.

*Yah, yah, you should still “run” STP so as to prevent accidental loops from forming, I know, I know: http://www.cisco.com/web/strategy/docs/gov/turniton_stpt.pdf

Posted in Miscellany, Switching | Leave a Comment »

These days, I just hard-code for job security.

Posted by qualityofservice on August 12, 2008

Ultimately, it’s a lot easier to keep one’s job if you can convince your manager that it takes highly skilled, motivated, and incredibly rare individuals to log into a switch and change a port default or two.  I suspect this is also the reason why I read about cable runs taking nine-and-a-half weeks.  The whole stack of cards comes tumbling down the minute one of us admits that it should really only take 2-3 minutes, given the proper cable length and a path with minimal obstruction between distribution frames.

But I digress: http://etherealmind.com/2008/07/15/ethernet-autonegotiation-works-why-how-standard-should-be-set/

Some interesting notes there. I’ve actually been a fan of set-and-forget over the years, but never could explain why. It just seemed to fix a lot of errors, and ultimately that was what was important. Having had the chance to work with quite a few makes and models over the last two years or so, I’m getting more comfortable with the idea of auto-negotiation.

After all, auto-neg IS a standard, and the Gigabit Ethernet specification is pretty clear on the matter of its importance. If it doesn’t work between two devices, it’s ultimately not a design problem (as I preferred to think of it), but rather a driver incompatibility problem (one of your vendors is making garbage NIC’s). These days, my thinking is more along the lines of “if you have to disable a standard to make it work, you’ve got bigger problems than a few collisions.”

You can indeed force Gigabit Ethernet on a Cisco switch to “speed 1000,” but you will be unable to set the duplex; it will default to full and stay there. You’d think this was a good thing until you see the GigE spec excerpted in the article, where it states that GigE uses auto-neg to detect which end of the link will provide clocking. : )

Ex:

lab-sw-3(config)#int gi0/24

lab-sw-3(config-if)#speed 1000

lab-sw-3(config-if)#duplex full ! accepts this command without complaint; it defaulted to “full” in the first place

lab-sw-3(config-if)#duplex half

Gigabit port is restricted to full duplex ! gets upset if you try to set it to “half”

lab-sw-3(config-if)#duplex auto

Gigabit port is restricted to full duplex ! equally upset if you try to set it to “auto”

lab-sw-3(config-if)#

Little food for thought!

Posted in Switching | Tagged: , , | Leave a Comment »

Getting to the root of the problem.

Posted by qualityofservice on August 9, 2008

From the “Learn something new every day” department…this one could actually be relevent to my studies.*  It was certainly relevant to my job.

In building a three-switch lab to simulate one site’s access-layer and test out some link bundling, I was seeing that my access switch’s uplink had a cost to the root of 16.  This did not make sense.  The path went through a Gigabit uplink to the secondary root, which is connected to the primary root by way of a two-link EtherChannel (two 100Mbps links).  I was expecting a cost related to the sum of costs for one 1Gb and one 100Mb link (4 + 19 = 23).

In looking up the path costs to try and work backwards, I found the cost for 200 Mb is 12.  This combined with the cost of the Gig uplink would give me my 16 value, but where was the 200 Mb coming from?

The answer was that the EtherChannel’s virtual interface gets its own bandwidth, equal to the product of the individual link bandwidth and the number of links in the channel.  Doing a “show interface port-channel” would display a bandwidth of 200 Mbps, and STP uses this interface’s characteristics to derive its path costing:

Port-channel21 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 0019.5669.5382 (bia 0019.5669.5382)
Description: *** HQD_CORE_01 ***
MTU 1500 bytes, BW 200000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 2/255
Encapsulation ARPA, loopback not set
Full-duplex, 1000Mb/s, link type is auto, media type is unknown

This led me to wonder if this could be changed with the interface-level “bandwidth” command.  So for kicks, I changed the bandwidth of a random Gig link to 100 Mbps.  Here’s the before and after:

SW-1>sho span int gi2/0/9

Vlan             Role Sts Cost Prio.Nbr Type
—————- —- — ——— ——– ——————————–
VLAN0102         Desg FWD 4 128.61   P2p

Then the change…

SW-1(config)#int gi2/0/9
SW-1(config-if)#bandwidth 100000

Then the result:

SW-1>sho span int gi2/0/9

Vlan             Role Sts Cost Prio.Nbr Type
—————- —- — ——— ——– ——————————–
VLAN0102         Desg FWD 19 128.61   P2p

The cost to reach the root switch will be the primary determining factor in root-port selection, for any STP topology that involves uplinks to non-root switches. ** This can be influenced by the bandwidth of your links, which I knew already, but what I didn’t know was that the path-cost may inadvertently change with link-bundling or manual interface bandwidth configuration.

*Picture the following scenario: Given a topology consisting of three switches, make Fa0/1 on Switch-C the alternate port…but change nothing on its upstream switches; do not disable STP or employ FlexLinks; and do not manually change the port cost with any “spanning-tree” commands.

**If you’re merely running multiple uplinks from one switch to the root, given interfaces of equal bandwidth, your root bridge ID, path cost, and sender bridge ID will be equal.  The tie is broken based on port ID (2-tuple value consisting of Priority:Interface-Number).  Of course, if you’re doing that, you should really consider EtherChannel; additional path-redundancy and bandwidth, what’s not to love!

Posted in Switching | Tagged: , , , | Leave a Comment »