J Tech Notes

More bits more places

Inside the Meraki MR58

We were recently gifted a box of old Meraki MR58s from a friendly neighborhood consultant that had removed them from service. Since there is essentially no information available on using the Meraki MR58 outside of the Meraki environment I had little hope that we’d be able to put them to good use.

Anyway, here I am a few months later sitting at home looking for a puzzle to solve while trying to get over a cold…

Ubiquiti AirFiber 5X: The Perfect Medium-capacity Backhaul

Ubiquiti is going to announce their AirFiber 5X radio tomorrow at the Animal Farm User Group (AFMUG) meeting in Salt Lake City tomorrow. The quick start guide was posted early to their web site and gives us a sneak peak. From their teaser emails (that mention price and nothing else) it has a MSRP of $399 per radio (without dishes).

Essentially it’s a connectorized version of the AirFiber 5. It uses airFiber branded dishes or RocketDishes with a retrofit kit. Dishes are cross-pol instead of V/H. It supports channel sizes from 10Mhz to 50Mhz.

I’m pretty excited about this radio. It fits a niche between the Rocket & ePMP radios on the low end and licensed microwave on the high end. This is a market where Cambium’s PTP3/5/600 radios have reigned supreme but at a significantly higher price point. The ability to use large dishes means it has a fair chance to actually hit the marketing numbers of 500Mbps HDX or 200Km range.

Also, it doesn’t hurt that this is coming from their AirFiber group out of Chicago. They seem to have a better track record of delivering products with fewer teething problems relative to the AirMax M/Ti/AC line. I assume that’s because their people brought along some of that Motorola engineering discipline when they left the Canopy group. In any case, I expect this to be a radio that I can drop in to replace a couple wonky AirMax ac links and use instead of upgrading to PTP500 in some other cases and have it actually work without fuss.

Ubiquiti AirFiber24HD: Solid Improvement With Excessive Hype

Ubiquiti announced their AirFiber24HD today. It looks like a solid incremental improvement to the classic AirFiber24 with a roughly 250 Mbps (FDX) increase in throughput.

Key differences

Feature AF24 AF24HD
Maximum Modulation:   64QAM   256QAM
RX Gain: 38 dBi   40 dBi
Housing: Plastic Aluminum
Maximum Throughput (FDX):    774 Mbps   1Gbps
Link MSRP: $3,000 $6,000

Excessive hype

A unlicensed-frequency radio that can do bog standard 1Gbps FDX is actually a big deal. You can use these for short links between buildings and achieve just as much throughput as if you trenched fiber and used normal SFPs. No more caveats about actually only having 700Mbps of throughput.

Unfortunately with Ubiquiti a decent improvement isn’t good enough. Every product or service has to be a revolutionary disruptive breakthrough for the masses. This was on full display with the AF24HD launch where apparently line-rate 1G wasn’t good enough.

That’s right. The headline throughput number for a 1Gbps FDX (2Gbps HDX/aggregate) radio is shown as 20 Gbps because that’s the theoretical aggregate throughput of devices that may be connected behind the link. If that was a thing I’d be bragging about my network having over 3Tbps of throughput, except we don’t because that would be dishonest.

Cisco Site-to-site VPN With Outside VRF

TL;DR: Outside interface is a VRF, inside on global, must Specify VRF on crypto keychain.

I recently set up a IPSec VPN between two Cisco routers where the Internet-facing interface was in a VRF and the tunnel interface was in not in a VRF.

The VTI was configured in the usual way:

VTI Configuration
1
2
3
4
5
6
7
8
9
10
11
interface Tunnel0
 description VPN: site2 via provider A
 ip address 192.168.1.1 255.255.255.254
 ip mtu 1438
 keepalive 5 3
 tunnel source 192.0.2.1
 tunnel mode ipsec ipv4
 tunnel destination 192.0.2.2
 tunnel vrf Outside-A
 tunnel protection ipsec profile vpn-to-site2-via-provider-a
!

With the usual crypto configuration:

IPSec configuration
1
2
3
4
5
6
7
8
9
crypto isakmp policy 1
 encr aes
 authentication pre-share
 group 2
 lifetime 3600
crypto isakmp key SecretKey address 192.0.2.2
crypto ipsec transform-set vpn-peer esp-aes esp-sha-hmac
crypto ipsec profile vpn-to-site2-via-provider-a
 set transform-set vpn-peer

but the tunnel wouldn’t come up.

“debug crypto isakmp” showed a error to the effect that a preshared key wasn’t configured or couldn’t be found for the peer. This was puzzling because there was clearly a preshared key defined.

There weren’t any messages in the debug logs about incoming connections or policy mismatches so I assumed that the problem was occurring before the router tried to communicate with its peer or it wasn’t seeing traffic from the peer.

From the documentation I found it seemed like the “tunnel vrf Outside-A” statement in the VTI was all that was necessary for IOS to know that the outside was in a VRF (the tunnel endpoint is in a VRF so the protection profile must be on the VRF as well). This turned out to not be the case. The vrf statement on the VTI is just for the tunnel. You need to specify the VRF in the crypto keyring definition otherwise the router isn’t looking for VPN traffic in the VRF.

I changed the crypto configuration to use a keyring and specified the outside vrf on the keyring definition:

Working IPSec configuration (VRF keychain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
crypto keyring vpn-peer vrf Outside-A
  pre-shared-key address 192.0.2.2 key SecretKey
!
crypto isakmp policy 1
 encr aes
 authentication pre-share
 group 2
 lifetime 3600
crypto isakmp profile vpn-to-site2-via-provider-a
   keyring vpn-peer
   match identity address 192.0.2.2 255.255.255.255
   keepalive 10 retry 5
!
!
crypto ipsec transform-set vpn-peer esp-aes esp-sha-hmac
!
crypto ipsec profile vpn-to-site2-via-provider-a
 set transform-set vpn-peer

and now it all works. Of course now that it’s working I find a Cisco doc on New Version Site-to-Site Configuration that shows this.

Power & IO Connector for Cisco 2955

If you bought a used Cisco 2955 switch and it didn’t come with the power/IO terminal block connector, it’s a Phoenix Contact 1827761 (Mouser part number 651-1827761).

IPv6 via 6rd on Charter With Cisco IOS

Charter does not yet offer native IPv6 but they do have IPv6 via 6rd tunnels for testing. It mostly works but I’ve seen some weirdness with various v6 destinations. Thumbnail images breaking on Facebook periodically and things like that.

Cisco IOS Configuration

I have this working on a Cisco 881W running IOS 15.3(2)T with the advsecurity license.

Enable IPv6:

ipv6 unicast-routing
ipv6 cef

Set up a DHCPv6 pool:

ipv6 dhcp pool home-pool
 dns-server 2607:F428:1::5353:1
 dns-server 2607:F428:2::5353:1
 domain-name home.example.com

Tunnel interface:

interface Tunnel0
no ip address
no ip redirects
ipv6 enable
 tunnel source FastEthernet4
 tunnel mode ipv6ip 6rd
 tunnel 6rd prefix 2602:100::/32
 tunnel 6rd br 68.114.165.1

Routes & general-prefix:

ipv6 general-prefix DELEGATED_PREFIX 6rd Tunnel0
ipv6 route 2602:100::/32 Tunnel0
ipv6 route ::/0 Tunnel0 2602:100:4472:A501::1

LAN interface:

interface Vlan1 
    ipv6 address DELEGATED_PREFIX ::1/64
    ipv6 nd other-config-flag
    ipv6 dhcp server home-pool

Basic Configuration in Ciena SAOS

Ciena makes Ethernet & Optical switches for the service provider market. There’s essentially zero publicly available documentation available which is a shame because they make some interesting equipment and as carriers upgrade there’s a lot of used equipment available that smaller providers can put to good use. I’m not sure why it’s traditional for telco-oriented companies to be stingy with their documentation but it’s annoying.

I have some LE-311v, 3911, and a 3940 here for a project so here’s my notes. They are applicable to all devices as far as I know unless otherwise noted.

This also applies to devices running World Wide Packets LE-OS. Ciena acquired WWP and built their packet Ethernet offerings on tech from the acquisition.

A Zone Named Management on a Juniper SRX

TL;DR: Don’t name a zone “management” on a Juniper SRX (11.4R7.5).

One of my on again, off again projects involves moving a datacenter management network with devices on public IP space with ACLs for protection to private IP space with a zone-based firewall (Juniper SRX240).

When I last touched it I ran into a problem where one zone would not pass traffic even though it had identical rules to a different zone that worked. It happens that the zone that didn’t work was named “management”.

Earlier today I was browsing around and found a article that mentioned the management functional zone which got me wondering if there was something special with naming a zone “management”. I thought that didn’t make sense since the functionality comes from the functional-zone tag not the zone name. Cue more unsuccessful searching for any mention of reserved zone names. Eventually I decided to just rename the zone and see what happened. One quick rename statement and “management” became “dcn-mgmt” and everything started to work.

What?

Then I came across a post to J-NSP that mentions management being a reserved keyword for zones. Oh. That explains it.

Cisco CSR 1000V on SmartOS

In pursuit of some Friday Evening Fun I decided to try to get a Cisco CSR 1000V VM running on SmartOS. I was successful. It runs. It seems to run quite well.

First, a overview of the systems involved since the CSR1000V is fairly new and SmartOS is relatively uncommon.

The Cisco Cloud Services Router 1000V (CSR1000V) is Cisco’s entry in the virtualized router field. It runs IOS-XE and brings many of the handy features that you can find on the ASR1k platform into the cloud so you can do fun and exciting things with MPLS, VPLS, LISP, etc. This is a pleasant change from Vyatta which lacks any sort of MPLS support and Mikrotik which blackholes traffic for no good reason.

SmartOS is a hypervisor based on Illumos which is in turn a fork of OpenSolaris which was next-gen Solaris before Oracle borged Sun. SmartOS has all the wonderful features of OpenSolaris (ZFS, Zones, network virtualization) and adds support for KVM. There are some interesting design choices in SmartOS that turn out to be pretty cool once you get used to them, in particular: Netboot only (or USB if you are desperate) with all local disk used for VMs and VM Management by JSON config files.

I started by downloading the CSR1000V ISO (csr1000v-universalk9.03.10.00.S.153-3.S-ext.iso) from CCO and then uploading it to a server in the datacenter. I followed the instructions for creating a KVM VM on SmartOS except I copied the CSR1000V ISO over instead of a Debian ISO. Installation was smooth but after boot and initial configuration I wasn’t able to reach the VM. It turns out that SmartOS has some decent network security policies by default so I had to add "allow_ip_spoofing": true to the VM config before traffic started to flow.

After the CSR was installed and configured for basic connectivity I set it up as a route reflector and sent it the partial BGP feed that I run on our old Cisco 6500 series routers (~100,000 routes). sho proc cpu hist showed up to 40% CPU usage while it was loading. After that I added a table-map containing a deny statement to the BGP config to prevent it from wasting time trying to update the RIB. This reduced loading time to two seconds with no response lag on the terminal.

My feeling in general on the CSR is that it simply has too large of a footprint for general cloud routing & VPN duties. Vyatta gets the job done with a lot less than 4G of RAM and four CPU cores. If your needs go beyond simple routing & VPN then the CSR becomes more appealing. So far it seems well suited to the route reflector role but I have more testing to do before I commit to it.

If the CSR holds up to testing I hope to run a pair of them in place of the aging 7200s that are currently acting as my route reflectors. Cisco’s recommended route reflector platform is the ASR1k but as a small ISP I just can’t justify the cost on something that doesn’t generate revenue. If this works I can stay with Cisco instead of cobbling something together on a different platform.

Configs used are available as Github Gists: VM config json and CSR config.