Notes on Cisco IOS Interface Order of Operations

Part of being a good network engineer is having the ability to effectively troubleshoot problems and answer questions without ambiguity. For the entry level networking technician (CCENT/CCNA), the implementation and troubleshooting tasks may not be involved enough to require detailed knowledge of how a router handles packets from ingress to egress. For intermediate network engineers (those at the CCNP level), this knowledge is important and as an individual transitions from the intermediate level to the advanced level (CCIE), this type of knowledge is extremely important to quickly isolate, identify, and solve issues in an enterprise network.

There are a lot of different manipulations that can happen with a router, including manipulations like tunneling/IPSec encryption, network address translation (NAT), policy routing, and Quality of Service (QoS) manipulation. Understanding the order that these happen is important for designing and troubleshooting different flows through a network. Cisco has published a couple of Technology Notes on the order of operations involving NAT and QoS and these same steps are in the CCNP TSHOOT study guides.

Here are the order of operations for interfaces involved with NAT

Inside to Outside	Outside to Inside
Input Access Control List (ACL) for Encrypted Packet	same
IPSec Decryption	same
Input ACL for Decrypted Packet	same
Input QoS	same
Input Accounting	same
Web Cache Redirection	same
Policy Routing	NAT inside global to inside local translation
Routing	Policy Routing
NAT inside local to inside global translation	Routing
Crypto Map (mark for encryption)	same
Output ACL	same
IP Inspection (a.k.a Context Based Access Control [CBAC], Stateful IOS firewall)	same
TCP Intercept	same
IPSec Encryption	same
Output QoS and Queuing	same

The logic is fairly simple for NAT, it is impossible to make a forwarding decision without performing an inside global to inside local address translation. Without NAT, the NAT step above simply disappears and the order of operations are the same. Additionally, any features that are not configured drop out of the steps above. In the simplest case, the only action is routing.

I was working through a CCIE lab exam study guide and it posed an interesting question that got me thinking more about order of operations. The question asked to set different QoS markings (DSCP values) based on packet length. With regular class based marking this is not possible, but it is possible to match length using a route map. I was curious to see which would take precedence, DSCP set using a route map or a DSCP set using an egress policy map. I set up a really simple lab below in Dynamips/GNS3 using Cisco c3725 routers:



The idea is that I want to change the DSCP for ICMP traffic that is being NAT'd from 10.0.0.1 to 192.168.0.3. I configured a policy map on the NAT router that set the DSCP to AF21 on egress via fa0/0. I set up a route-map that set the appropriate IP precedence (3/immediate) and TOS (2/max-reliability) to support a combined DSCP of AF31. Important configuration pieces on the NAT router are below (MQC commands are in green and policy routing related commands are in blue):

!
class-map match-all match-echo
 match protocol icmp
!
!
policy-map set-icmp
 class match-echo
  set ip dscp af21
!
!
interface FastEthernet0/0
 ip address 192.168.0.2 255.255.255.0
 ip nat outside
 ip virtual-reassembly
 ip policy route-map set-icmp-dscp
 duplex auto
 speed auto
 service-policy output set-icmp
!
!
access-list 101 permit icmp any any
!
!
!
route-map set-icmp-dscp permit 10
 match ip address 101
 set ip precedence immediate
 set ip tos max-reliability
!

The final question that we really have to look at on the outside router is what DSCP values are received for the ICMP traffic (ping in this case)? This can be found by setting an extended access list that matches one or more DSCP values and the correct source/destination addresses:

access-list 101 permit ip any host 192.168.0.1 dscp af21

And then we watch for packets that match that ACL:

debup ip packet 101

Performing a ping to the outside router indicates that the traffic is received with DSCP AF21. This indicates that the egress marking takes precedence over marking that happens due to policy routing.

Outside(config)#do debug ip packet 101
IP packet debugging is on for access list 101
*Mar  1 00:25:06.879: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar  1 00:25:06.883: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar  1 00:25:06.959: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar  1 00:25:06.959: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar  1 00:25:07.003: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar  1 00:25:07.003: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar  1 00:25:07.043: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar  1 00:25:07.043: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar  1 00:25:07.087: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar  1 00:25:07.087: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3

If there was no marking occurring on egress, then the policy routing marking would take precedence.

See Also,
The Road to the CCIE

Any source