IP Encyclopedia > IOAM

What Is IOAM?

In-band Operations, Administration, and Maintenance (IOAM) is a network measurement and monitoring technology. It enables devices to sample service traffic in real time at a high speed, adds IOAM information (metadata, including the device ID, inbound and outbound interfaces, and timestamp) to the sampled data, and proactively sends the sampled data to an analyzer for analysis. In this way, the network running status is detected and monitored in real time.

Contents

Why Do We Need IOAM?

The continuous growth of users and Internet services is driving networks to become large-scale, high-speed, multi-access, and unpredictable. This raises the requirements on network management and control. Network measurement is the basic means and data source of network management and control. Common network measurement methods are classified into three types: proactive, passive, and hybrid.

Proactive measurement enables a sender to proactively send detection information to a receiver on the network to analyze network behaviors. Common proactive measurement technologies include ping. Although proactive measurement methods are flexible to use, they increase the bandwidth usage and processing overhead of the network.
Passive measurement enables measurement points to obtain service traffic to measure the network status, traffic characteristics, and performance parameters. Passive measurement technologies include sFlow. Passive measurement methods do not generate extra measurement load, but cannot obtain network-wide information such as the network status and packet loss rate.
Hybrid measurement enables intermediate switching nodes on a path to add metadata to data packets in sequence to collect information about the network status. IOAM technology is a typical hybrid measurement method. Hybrid measurement combines the advantages of proactive measurement and passive measurement. It can ensure the accuracy of the obtained service traffic status and occupy as little bandwidth as possible. Compared with the traditional network measurement solution, hybrid measurement can implement more fine-grained measurement. It has good programmability and various types of data information can be collected.

What Are the Components of the IOAM System?

The IOAM network system consists of the data collection module and data analysis module, as shown in the following figure.

The data collection module includes three nodes: the encapsulation, transit, and decapsulation nodes, which collect the running data of network nodes and forward packets.
The data analysis module mainly consists of an analyzer, which analyzes the collected data for network monitoring.

The three nodes in the data collection module implement the following functions:

Encapsulation node: samples service packets, adds IOAM information to packets, and forwards the packets.
Transit node: adds the IOAM information of the local node to the IOAM packets received from the encapsulation node and forwards the packets, or directly forwards the packets without adding IOAM information.
Decapsulation node: adds the IOAM information of the local node to the IOAM packets received from the transit node, copies the packet headers, encapsulates the copied packet headers into packets in NetStream V9 format, and sends the packets to the analyzer. Additionally, the decapsulation node removes IOAM information from the received IOAM packets, and then forwards the packets to the destination address.

One or more nodes may be responsible for sending the collected data to the analyzer. Therefore, the data collection module can work in different modes according to different data collection methods.

IOAM network system

How Does IOAM Work?

IOAM Packet Format

The following figure shows the format of sampled TCP and VXLAN packets after IOAM encapsulation.

Formats of IOAM-encapsulated TCP and VXLAN packets

IOAM Header Format

An IOAM header consists of the probermarker, IOAM SHIM header, and IOAM option header, as shown in the following figure.

The probermarker is a 64-bit IOAM header that identifies an IOAM packet.
The 32-bit IOAM SHIM header identifies the data collection mode of IOAM packets.
The 64-bit IOAM option header identifies the data encapsulation format of the metadata (MD) in different modes. The IOAM option header format varies according to the data collection modes.

IOAM header format

The MD information of a collection node varies according to the data collection modes. As shown in the following figure, the MD information of a collection node contains the inbound and outbound interfaces, forwarding delay, timestamp, and queue depth of the node in trace and direct-export modes, and contains the sequence number and the timestamp when a packet enters the device in edge-to-edge mode.

IOAM MD formats

IOAM Implementation

For the data collection module, three data collection modes are available: trace mode, edge-to-edge mode, and direct-export mode.

On the network shown in the following figure, the encapsulation, transit, and decapsulation nodes perform the following functions in trace mode.
- Encapsulation node: samples service packets sent by Host1, encapsulates the packets with IOAM headers and its own MD information, and then forwards the packets.
- Transit node: receives IOAM packets from the encapsulation node, adds its own MD information to the packets, and then forwards the packets. This node is mainly used in hop-by-hop analysis scenarios to obtain information of the inbound and outbound interfaces, forwarding delay, timestamp, queue depth, and time to live (TTL) of traffic. As the number of hops that packets traverse increases, so does the packet headers, requiring high device forwarding performance.
- Decapsulation node: adds its own MD information to received IOAM packets, copies the packet headers without the payload, encapsulates the copied packet headers into packets in NetStream V9 format, and sends the packets to the analyzer. Additionally, the decapsulation node removes IOAM information from the received IOAM packets, and then forwards the packets to Host2.
Packet forwarding on an IOAM network in trace mode (TCP packets are used as an example)
The edge-to-edge mode is similar to the trace mode. The difference is that the transit node does not add its MD information to packets and only functions as a packet forwarding device to directly forward the IOAM packets sent by the encapsulation node in edge-to-edge mode. As the transit node does not add its MD information to the packets, the length of packet headers does not increase significantly. Therefore, in this mode, there are no high requirements on the device forwarding performance. This mode applies to scenarios where only edge-to-edge delay information is required.
The direct-export mode differs greatly from the preceding two modes.
- The encapsulation node encapsulates service packets with IOAM headers, copies the packet headers, adds its own MD information to the copied headers, encapsulates the copied packet headers into packets in NetStream V9 format, and sends the packets to the analyzer. Additionally, the encapsulation node forwards the received packets with IOAM headers (excluding the MD information) to the transit node.
- After receiving the packets, the transit node copies packet headers with IOAM headers, adds its own MD information to the copied headers, encapsulates the copied packet headers into packets in NetStream V9 format, and sends the packets to the analyzer. Additionally, the transit node forwards the received packets with IOAM headers (excluding the MD information) to the decapsulation node.
- After receiving the packets, the decapsulation node processes the packets in a similar way as the transit node. The difference is that the decapsulation node removes IOAM headers from the received packets and then forwards the packets to the destination address.
In direct-export mode, each node sends packet headers carrying IOAM information to the analyzer. If packet loss occurs on an intermediate node and the node does not send the packet headers to the analyzer, the analyzer can receive packets from other nodes. This mode requires high network bandwidth because each node sends packets to the analyzer.

Application of IOAM

As shown in the following figure, IOAM can be used on a VXLAN network to measure service traffic transmitted through a VXLAN tunnel. In the actual application, the data collection mode and the position of each node should be determined according to the network monitoring requirements and network performance.