Light posting in the last little while due to a death in the family.

Will resume over the coming days with more than one could ever care to learn about QoS architecture of the Catalyst 2960/2970/3560/3750 series switches!

NetFlow

NetFlow is a Cisco proprietary standard, soon to become (if it’s not already) an international standard in the form of IPFIX (http://www.ietf.org/html.charters/ipfix-charter.html). It tracks flows ingress into an interface and does accounting based on source/dest IP/port, TOS, originating autonomous system, and all manner of other cool things.  This info can be exported to central collectors which can store the data in a DB and mangle it as they see fit.

NetFlow is supported on any and all recent IOS routers (read as: 1800/2800/3800 ISR series, 7200/7600, etc).  Alas, no support on Catalyst dumb Layer-2 and multilayer switches outside of the 4500/6500 line, and even then it requires special hardware in the form of proper line cards/Supervisor Engine(s).

However, you can do a poor-man’s NetFlow by building a “probe” that accepts mirrored traffic from a SPAN port on a switch, and crafts its own NetFlow data from the observed traffic (see also: nTop).  Your mileage may vary depending on your IOS version; this note’s test router uses 12.4(15)T7.

You don’t need a collector to get some use out of the feature, though; it maintains a local cache and that’s what this note’s going to be about.  Quite easy to turn on:

interface FastEthernet0/1
ip address x.x.x.x y.y.y.y
ip flow ingress

Verification:

TEST-VPN-Hub-01#sho ip flow interface
FastEthernet0/1
ip flow ingress

Then turn on the top-talkers feature:

TEST-VPN-Hub-01#conf t
TEST-VPN-Hub-01(config)#ip flow-top-talkers
TEST-VPN-Hub-01(config-flow-top-talkers)#top 100

Then we get the option of viewing un-aggregated cache data, or aggregated cache data:

TEST-VPN-Hub-01#sho ip flow top-talkers ?

Display aggregated top talkers:
<1-100>  Number of aggregated top talkers to show

Display unaggregated top flows:
verbose  Display extra information about unaggregated top flows
|        Output modifiers

Un-aggregated provides a very granular view of flows stored in cache; one flow per source/dest IP/port and IP Protocol number (with protocol number and src/dst ports reported in very obnoxious hex), and by default sorted by bytes ingress to the interface:

TEST-VPN-Hub-01#sho ip flow top-talkers

SrcIf         SrcIPaddress    DstIf         DstIPaddress    Pr SrcP DstP Bytes
Tu110232      10.0.30.63      Fa0/1         192.168.141.81  06 170C 88ED  2074K
Tu110232      10.0.30.40      Fa0/1         192.168.141.71  06 0FD9 F727  1519K
Tu110232      10.0.30.140     Fa0/1         192.168.141.144 06 0BFE DB22  1275K
Tu110232      10.0.30.140     Fa0/1         10.1.250.81     06 0BFE B70E  1243K
Tu110232      10.0.30.140     Fa0/1         10.1.250.81     06 0BFE B70F  1242K
Tu110232      10.0.30.62      Fa0/1         192.168.141.80  06 170C 83ED   532K
Tu110232      10.0.30.140     Fa0/1         10.1.250.81     06 0BFE A204   340K
Tu110232      10.0.30.140     Fa0/1         192.168.141.144 06 0BFE D3D8   251K
Fa0/1         192.168.141.81  Tu110232      10.0.30.63      06 88ED 170C    69K
Fa0/1         192.168.141.80  Tu110232      10.0.30.62      06 83ED 170C    60K
Fa0/1         192.168.141.144 Tu110232      10.0.30.140     06 D3D8 0BFE    38K

Useful if you have a source that’s just pounding away; you can easily see where it’s coming from (and the interface through which it enters) and where it’s going (and the interface through which it leaves).

Aggregated view allows you to aggregate the NetFlow data a whole bunch of different ways (I’ve cut a bunch of ways out for sake of brevity):

TEST-VPN-Hub-01#sho ip flow top-talkers 100 aggregate ?
bytes                  number of bytes
destination-address    Destination address
destination-interface  Destination interface
destination-port       Destination port
icmp                   ICMP type and code
ip-nexthop-address     IP nexthop address
max-packet-length      Maximum packet length
min-packet-length      Minimum packet length
packets                number of packets
source-address         Source address
source-interface       Source interface
source-port            Source port
tcp-flags              TCP flags

What follows are ways to find the hot destination ports from your router’s point of view:

TEST-VPN-Hub-01#sho ip flow top-talkers 100 aggregate destination-port sorted-by packets

There are 20 top talkers:

TRNS DST PORT       bytes        pkts       flows
=============  ==========  ==========  ==========
35053     1638362        8922           1
54232     1462512        4017           1
33773      861529        3757           1
63271     1161960        2904           1
56098      950000        2609           1
46862      916876        2518           1
46863      916472        2516           1
5900      110858        2226           2
0      688278        1030          13
2048      658800         549           1
3070       12480         312           5
4056        3492          70           1
4057        2680          67           1
57556        6804          67           1
41476       15288          42           1
3092        2860          35           3
161        2556          35           3

Note that “Port 0” shows up in the above; I believe this may be related to packet fragmentation.  Non-initial fragments will not contain a transport-layer header; rather, they’ll simply have more transport-layer payload.  NetFlow can relate such a packet to a particular transport-layer protocol on account of the IP Protocol field of the IP packet (6 = UDP, 17 = TCP), but that’s as good as it can do without reassembling the entire packet.

Mind you, the traffic could also be IPSEC, which uses IP Protocol 50 or 51 for AH or ESP, respectively, and does not have port numbers for NetFlow to count.  This test bed was also running EIGRP and GRE tunnels; this traffic may have also been counted as “Port 0” traffic.

And to see some equally hot source hosts:

TEST-VPN-Hub-01#sho ip flow top-talkers 100 aggregate source-add sorted-by packets

There are 25 top talkers:

IPV4 SRC ADDR         bytes        pkts       flows
===============  ==========  ==========  ==========
10.0.30.63          1758749        9609           1
10.0.30.140         3161180        8681           5
10.0.30.62           996875        4319           1
10.0.30.40          1266040        3226           5
192.168.141.80       121738        2444           1
10.1.250.81           35960         899           3
192.168.139.66       990000         825           1
192.168.139.129      988800         824           1
192.168.141.144       24640         616           2
192.168.141.81        22520         451           2
192.168.141.71        12372         309           2
192.168.191.234       19008         288           1
192.168.191.242        9900         150           1
192.168.141.70         3944          81           2
192.168.141.66         3360          56           1
192.168.141.65         3300          55           1
192.168.191.238        2508          38           1
192.168.141.70         1680          28           1
192.168.141.76         1680          28           1
192.168.141.75         1680          28           1
192.168.141.71         1620          27           1
192.168.141.72         1620          27           1
192.168.191.230        1650          25           1
10.1.40.169              72           1           1

The command “show ip cache flow” also produces interesting results, including timers associated with the flow cache.

TEST-VPN-Hub-01#sho ip cache flow

IP packet size distribution (26090 total packets):
1-32   64   96  128  160  192  224  256  288  320  352  384  416  448  480
.001 .500 .155 .007 .005 .005 .006 .007 .007 .007 .006 .204 .045 .004 .004

512  544  576 1024 1536 2048 2560 3072 3584 4096 4608
.003 .002 .002 .012 .008 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 278544 bytes
60 active, 4036 inactive, 675 added
29520 ager polls, 0 flow alloc failures
Active flows timeout in 30 minutes
Inactive flows timeout in 15 seconds
IP Sub Flow Cache, 25800 bytes
0 active, 1024 inactive, 0 added, 0 added to flow
0 alloc failures, 0 force free
1 chunk, 0 chunks added
last clearing of statistics 00:07:23

Protocol         Total    Flows   Packets Bytes  Packets Active(Sec) Idle(Sec)
——–         Flows     /Sec     /Flow  /Pkt     /Sec     /Flow     /Flow
TCP-other          338      0.7        29   155     22.6      18.6       9.6
UDP-NTP             37      0.0         1    76      0.0       0.0      15.4
UDP-other          184      0.4        13    75      5.8       6.2      15.5
ICMP               100      0.2         2   757      0.5       0.5      15.6
Total:             659      1.4        19   151     29.1      11.4      12.5

From the above output, you can see that flows will age out of the cache 15 seconds after data associated with the flow stops flowing.  You can test this by pinging something through the router (in my tests, locally-originated ICMP traffic was not counted by NetFlow, but there’s a chance I may have just been doing it wrong), and filtering the output of “show ip flow top-talkers” or “show ip cache flow”, until there’s been enough transferred data associated with the flow for it to work its way into the cache.

Then stop the ping.  15 seconds later, the flow won’t be there anymore; so by definition, flows that have accumulated a lot of traffic have been active for a very, very long time.  This technique is incredibly handy for tracking DoS activity; if you’re able to log into a terminal, you can work backwards to find the source address and input interface of potential DoS’ers, misbehaving hosts, etc.  Taken to its logical conclusion – assuming cooperation with a supportive and clueful ISP — you can even trace a spoofed IP address back to its real source. How this would be accomplished is left as an exercise for the reader.

There’s also a packet-size histogram; from the above, you can deduce that 50% of the packets transiting the router are between 32-64 bytes; 15.5% are between 64-96 bytes; and 20% are between 352-384 bytes.

Over at $dayJob, I use http://www.plixer.com/products/free-netflow.php to keep track of a day’s worth of NetFlow data; for a free tool, it’s incredible for providing point-in-time analysis of application use on my network.  As they say,  in network analysis, there is no substitute for knowing your network.  While longer-term analysis would be ideal, I don’t have long-term enterprise NetFlow collection in my budget, nor the time to build out my own; though after you’ve kept a watchful eye on links for a few weeks, you start to see patterns, and deviations from that pattern should be either easily explained or quickly investigated.

Monitoring/managing logins and config changes with IOS

For the purposes of this note, I’m going to pretend Telnet doesn’t exist.  Most of the stuff applies regardless of whether you use it or not, but I’m happier working under the assumption that all VTY configs look like this:

line vty 0 15
transport input ssh

I’m going to digress already and say that it’s a good idea to restrict access to certain networks:

line vty 0 15
access-class 101 in
transport input ssh

And that some go a step further and protect the last VTY line as a last resort in the event that the other 14 or so are occupied by someone with less-than-benevolent purposes; that way, the host(s) specified in ACL 102 can still manage the router:

line vty 0 14
access-class 101 in
transport input ssh

line vty 15
access-class 102 in
transport input ssh

But back to the point.  In the early days of IOS 12.3, they introduced the “login” command-set for login security enhancement (http://www.cisco.com/en/US/docs/ios/12_3t/12_3t4/feature/guide/gt_login.html).

login block-for 60 attempts 3 within 60
login delay 3
login on-failure log
login on-success log

This gives you three chances to pass the test before the router blocks all logins for 60 seconds (this period is called “quiet mode”).  There’s also a 3-second delay between attempts.  This mitigates someone throwing the kitchen sink at your device; it takes them 9 seconds just to try three times, and they can only do so once a minute without changing IP addresses.  A “quiet mode” list can be configured to allow certain hosts to get around these restrictions; this is a good idea, because someone spamming login attempts can lock you out, and it’s a race to log in when quiet time ends.  Luckily, the “on-failure log” will tell you which IP address is responsible for the attack. Info on configuring quiet-mode bypass is in the documentation linked at the end of this note.

Of course, the problem with this is that by default, IOS will let you attempt four SSH logins before terminating the session.  You can fix that, too. I use this:

ip ssh authentication-retries 2
ip ssh logging events
ip ssh version 2

“authentication-retries” is, literally, retries.  It lets you make two additional attempts after the first failed attempt; hence three in total, which matches up with the three attempts before you’re locked out for a minute, configured above in the “login” section. “Version 2” forces the use of SSHv2 by the client side; SSHv1/v1.5 considered insecure and deprecated for well over a decade.

Finally, the built-in config-change archiver/logger:

archive
log config
logging enable
logging size 200
notify syslog
hidekeys

This will take any change made in config mode, save a small local copy of said changes to a local buffer, and spit them out to syslog. “hidekeys” keeps sensitive info obscured (syslog packets being unencrypted and all).  How many times have you asked yourself “well, what’s changed?”  This lets you know in real-time.

All this and more over at the IOS Security Configuration Guide and Command Reference, which can be found here for IOS 12.4: http://www.cisco.com/en/US/docs/ios/security/configuration/guide/12_4/sec_12_4_book.html

Whole bunch of examples below the cut!

What is a “slow link”?

Cisco’s slightly-outdated QoS SRND (http://www.cisco.com/univercd/cc/td/doc/solution/esm/qossrnd.pdf) refers to slow, medium, and fast-speed links.

What’s the distinction? Why is anything less than 768kbps considered “slow”?

Well, one of the large ones is serialization delay.  This is somewhat covered in the SRND, but I wanted to highlight some of the key reasonings behind the distinction, as they’re applicable regardless of how you define “slow.”

Slow links suck, on that everyone can agree, but for real-time traffic –voice and interactive-video, for example–  they suck even more than only being able to torrent at 80 KB/s.

First, a few constants, some of which influence the “slow” characterization moreso than others, and are specified for informational purposes only:

Well-designed VoIP expects a one-way delay — mouth of speaker to ear of receiver — of approx 150ms.  This is largely going to be a function of processing and propagation delays along the link; as link speeds increase, the effects of serialization delay on the total delay budget are reduced.

Second, standard G.729 codec specifies 10ms of voice per frame.  Cisco IP phones pack two G.729 frames into each VoIP packet, for a total of 20ms of voice carried per packet; I’m focusing exclusively on the G.729 codec as it is the de-facto standard for VoIP over the WAN, due to its compression algorithm producing an 8kbps voice stream –before IP overhead is taken into account– compared to the 64kbps of G.711.

G.729 codecs can compensate for approx. 30ms of lost voice.  Given that a single VoIP packet contains 20ms, a loss of two consecutive VoIP packets (40ms worth of voice) will be noticed by the receiver.  Naturally you want to avoid losses, but moreso in a real-time voice environment than any other.  This is what Low-Latency Queuing is for, but it will not be the focus of this note.

Third, IP phones expect a relatively constant stream of voice packets.  Non-constant delays experienced along the path produce jitter; if one packet takes 160ms to get to the receiver, and the following packet takes 170ms, the stream has experienced 10ms of jitter.  DSPs in IP phones can compensate for approx 40ms of jitter by buffering received packets and replaying them to the receiver at a constant rate.  For this reason, you always want to have packets falling in the window of the jitter buffer (20ms to 50ms is a decent target for the IP phones I’ve read about).

So, given the above, what makes a 768kbps link slow?

Well, you want to stay within the jitter-buffer window, which means approx 30ms of jitter.  Cisco’s speed characterization is based on the assumption of 10ms of jitter per hop.

If we take one extreme, a link that carries a single voice call experiences very low jitter, as there is extremely low delay between voice packets :

———-[voice][voice][voice] ———–>

But what happens when you throw a large data, unfragmented data packet into the mix?

———-[voice][------Data Payload-----][voice] ———–>

If that data packet is 1500 bytes, and the line speed is 768kbbps, it takes 15ms to clock that packet onto the physical line (warning: math!):

(1500*8) = 12000 bits/data-packet

12kb / 768kbps = 15.625ms

Thus, the 2nd voice packet in the stream experiences over 15ms of jitter.  This is fine if you have a site-to-site WAN link, but less fine if your voice path transits multiple routers, as it significantly impacts your jitter and delay budget.

Remember, these numbers are based on Cisco’s assumptions.  If you can get within ~30ms jitter while going through a single slow link and multiple extremely fast links (with very little serialization delay), then the number of routers you transit becomes less of a concern; whereas Cisco’s numbers are based on a three-hop path experiencing 10ms of jitter per hop.

The important thing to take away is that you want to stay within the window, and that at low speeds, serialization delays profoundly impact your jitter budget.  See the following spreadsheet for reference:

	Packet Size
Bandwidth	20	40	80	128	256	512	1024	1200	1500
56	2.86	5.71	11.43	18.29	36.57	73.14	146.29	171.43	214.29
128	1.25	2.50	5.00	8.00	16.00	32.00	64.00	75.00	93.75
256	0.63	1.25	2.50	4.00	8.00	16.00	32.00	37.50	46.88
512	0.31	0.63	1.25	2.00	4.00	8.00	16.00	18.75	23.44
768	0.21	0.42	0.83	1.33	2.67	5.33	10.67	12.50	15.63
1024	0.16	0.31	0.63	1.00	2.00	4.00	8.00	9.38	11.72
1544	0.10	0.21	0.41	0.66	1.33	2.65	5.31	6.22	7.77
4632	0.03	0.07	0.14	0.22	0.44	0.88	1.77	2.07	2.59
10000	0.02	0.03	0.06	0.10	0.20	0.41	0.82	0.96	1.20
100000	0.00	0.00	0.01	0.01	0.02	0.04	0.08	0.10	0.12
1000000	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.01	0.01
10000000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00

Note that above 768kbps, the influence of serialization delays on the links can only get better than 15ms, regardless of packet size; at these speeds, serialization delays become less of a concern.

There are a few things you can do to overcome the serialization limitations imposed by slow-speed links, and there are good reason why [some of] these same techniques should NOT be employed on high-speed links; these will be covered in a future note.

Right now, I’ve got a buddy’s 30th birthday to attend and I plan on getting drunk enough to forget most of this.

Add a little flash (to your IOS router)

Can’t believe I’ve never played with these before, they’re brilliant.   12.4T Advanced IP Services images are over 32MB in size and it’s not possible to store two different images on the same stock flash drive, which introduces a risk when remote upgrades are required.  If an upgrade goes bad, there are some sites where I can count on remote hands capable of solid support; others, I’m not so fortunate.  So all the remote sites are getting USB keys, now, which will do more for my ability to keep my sites consistent and stable than any other measure implemented in my three years in this position.

 The ISR routers come with a USB port.  Insert USB stick, router recognizes it immediately. 

 Do a “format usbflash0” and it was ready to go.  TFTP’d an image, and set it to boot from the USB stick with “boot system usbflash0:[imagename]”, rebooted, and came back up on an upgraded image.  Removed the memory key, rebooted, and it ignored the “boot system” specification and booted back into the old image from flash.

 Copied the old image from flash onto the USB stick (“copy flash:[oldimage] usbflash0:”), deleted the old image from flash, copied the new image to flash, and done.  Known working image in flash, and both old and new images stored on the USB stick.  In my case, an 1841 recognized a 4GB USB key, which provides 16x more image storage capacity over the default 64MB of Flash that ships with the ISR bundles I order. 

 No need to worry about a reboot leaving you high-and-dry mid-upgrade after you’ve removed an old image to make room for the new one; which should remove any reticence to keeping IOS images current.  Just copy to USB and boot from the stick, first (caveat: takes about 220 seconds to load a 36MB image from USB into RAM on an 1841; takes about 120 seconds to load the same image from flash).  Worst case, you fall back to a known good image in flash.

 For the security conscious, yes, this opens up the ability to have someone stick their own file onto the USB key and somehow get your router to load it; but if they have the physical access to permit them to do this in the first place, it’s simpler for them to just reboot into password recovery mode and do whatever they like.

 Caveats: Cisco will sell you their own USB keys, but they’re about $300 after discount to add 256MB (part number: MEMUSB-64/128/256FT);   I’d rather pay $10 to add 4GB.  I’ve only tested this with a Kingston DataTraveller stick; YMMV.  I also move the “new” image to Flash once I’m ready to go into production with it; the risk being that if you find yourself having to work through a TAC case and they notice that you’re booting from a non-Cisco flash, they may tell you to suck rocks — which is a risk I’m willing to take in order to be able to test and upgrade on my own terms

Eponymous.

I’m currently studying/practicing for the Cisco 642-642 QoS exam, and I’ve gotta say, it’s opened up an entirely new toolset for me.  I’m but a mere enterprise admin, but I’ve seen a lot of routers and switches in my day, and it’s rare that I’ve seen anything (configuration-wise) that’s truly difficult.  We’ve all got our own little configuration quirks (given a device with a legacy configuration at $dayJob, I can likely tell you who configured it within a 95% confidence interval*) that we’ve either picked up from our peers — or on our own and thought “neat, I’m going to try that everywhere!” — but this is the first “feature” I’ve seen that requires some truly in-depth planning; I’ve no doubt this perceived difficulty contributes to my lack of having never seen it in production, so I’ll likely spend some time in an upcoming post ruminating over some of the reasons for or against QoS deployment in enterprise networks.** 

But first up, a quick note on what actually constitutes a “small link,” as far as Cisco documentation is concerned.  I say Cisco doc, but there is math involved and I believe the concepts to be vendor-independent.

But before that, there’s a hockey game on. ^_^

*We still have some routers that are explictly denying IP protocols 53, 55, 77, 103 ingress on all interfaces, for anyone with memories stretching back a few years: http://www.cisco.com/warp/public/707/cisco-sa-20030717-blocked.shtml

**I make the distinction because those with service provider backgrounds have the luxury of bandwidth; bandwidth can solve any QoS-related problem, but I think there’s still a home for the concepts in MPLS VPN networks.

1841 Modules.

I’m putting this here for my own reference.  I forget this all the time.  Stemmed from an argument with another admin who insisted that his Cisco SE told him that an 1841 supported FXS/FXO and E&M modules.

FAQ: http://www.cisco.com/en/US/prod/collateral/routers/ps5853/prod_qas0900aecd80181208.html

Module support: http://www.cisco.com/en/US/prod/collateral/routers/ps5853/product_data_sheet0900aecd8016a59b.html (bottom of page)

Long story short: supports wireless and every WAN under the sun (DSL/Cable/T1/E1/ISDN/Serial); no support for voice cards.

Voice can transit it like any other data packet, but the router itself cannot terminate voice circuits.

If the above is untrue, then Cisco’s documentation is woefully out of date.

IPv6

Y’know, I really wanted to throw myself into IPv6 this year; then I went and got distracted by a large-scale VMware deployment (hence the lack of posting over the last…three months).  We’re a three-person shop at $dayJob these days, supporting 700+ users across 20+ different countries, and that’s regrettably meant that the things that aren’t required RIGHT NOW get pushed off to the side.

Now that I’m finally finishing that up and comfortable enough with my giant NetApp storage array that I can go without looking at it for a few days, I’m starting to look back into IPv6 again.

I’ve some familiarity with the way the header looks and some basic deployment scenarios — but mostly just those acquired from my CCNP studies of old. Having gone through months of NANOG archives and found disagreement all over the ISP community with respect to the best way(s) to deploy IPv6, I’m even more intimidated.

(That said, I’ve done a paint-by-numbers deployment of IPv6 over MPLS VPN with some Cisco 3800-series routers we snagged from a decommissioned branch to bring some of my BGP/MPLS studies together; that was a ton of fun)

I’ve been prepping for it for a while, though, in terms of all my new hardware acquisitions. Anyone pushing something that wasn’t v6-aware right NOW has been shown the door since 2007, so I’m just about ready to go dual-stack across the enterprise (though few if any of my ISP’s are ready to support this deployment). Going to be one of those things where I’ll just have to take the documentation and start pushing it out and breaking it to see what works and what doesn’t.

But the most frightening thing of all is the sheer size of the address space. Jesus Christ, it’s big. Like, really big. Big enough that I completely forgot how subnetting worked in the first place. 32-bit dotted-decimal was easy to wrap one’s head around; hard to find anyone who’s been doing this for a while who doesn’t have a few hundred critical infrastructure/server addresses committed to memory — safe to say those days are gone.

Think of all the pages wasted on teaching those new to networking how to properly subnet in order to efficiently provision what was once a scarce resource, and how those practices are still being taught without a really big caveat: “Oh by the way, you don’t really have to know this anymore; the value of these pages is going to plummet in the next five years, and here’s why…”

For a lot of people, it’s going to be the first large technical revolution they’ve had to face.  IP hasn’t changed in over three decades; new features were merely layered on top of a fully functional protocol on demand.  But now everything that uses that fundamental protocol has to change; the magnitude of this project is enormous and IT departments who haven’t yet begun planning are years behind the curve (and this is a lot of IT departments, by my anecdotal measure).

I look around at the people who’ve been doing this stuff for years; they’d probably hoped to not have to face this before retirement, but that’s not going to be the case. How does one best go about convincing them that not only is a an IPv6 /64 a completely valid way to address a point-to-point link^[1], but a way that’s encouraged over the old practice of allocating an IPv4 /30 (or in the case of IPv6, a /127)?

There’s going to be a lot of money to be had in the IPv6-migration consulting business.

[1]: http://tools.ietf.org/html/draft-palet-v6ops-point2point-01

I’m going to get back to this, I swear; just took some time off to start working on the CCIP. 

Pleased to report that I passed the BGP exam yesterday, having recently completed the Implementing BGP on Cisco Routers course as delivered by Elan Beer — who, at #1837, is one of the first 1000 CCIE’s. 

For a full week, we had the opportunity to pick the brain of someone who has acted as a technical reviewer for Cisco Press products; impossible to come out of that and not know a little something about the stuff. : )

Anyway, happy new year, etcetcetc.  Lots of nerdly goodness to come, honest!

Non-routing router: bug or feature?

Trying to connect to some routers in Rio, couldn’t do it.  Had to access from a local device, instead.  No problem, just needs a default gateway, right? 

Log in, add default gateway.  Try again.  No luck.  ACL?  No.  “show ip route” completely blank.  What the.  Bug?  Reload.  Still nothing. 

“show run”?

“no ip routing”

 /groan