Home Search Center Intelligent Model Selection IP Encyclopedia

What Is NetStream?

NetStream is a technology that collects packet statistics on a per-flow basis. It classifies data flows based on key fields of packets such as the destination IP address, destination port number, protocol number, and source IP address, collects statistics on the data flows, and sends the statistics to a server for analysis. By analyzing these statistics, a network administrator can determine the source, destination, and occupied egress bandwidth of traffic, providing a data basis for applications such as accounting, network management, and network optimization.

Components in a NetStream System

A NetStream system consists of the NetStream Data Exporter (NDE), NetStream Collector (NSC), and NetStream Data Analyzer (NDA).

NetStream system
NetStream system
  • NDE: analyzes and processes service flows, extracts the flows that meet the conditions for statistics collection, and exports the statistics to the NSC. The NDE can process the statistics (such as aggregating statistics) before exporting them to the NSC.
  • NSC: stores the packets received from the NDE and saves statistics to its database for the NDA to analyze. The NSC can collect data exported from multiple NDEs for further processing.
  • NDA: a traffic analysis tool that extracts statistics from the NSC, processes the statistics, and generates a report. This report provides a basis for diverse applications such as accounting, network management, and network optimization. The NDA provides a graphical user interface (GUI) for users to easily obtain and analyze collected data.

In practice, the NDE is a network device enabled with NetStream, and the NSC and NDA are typically integrated into the same NetStream server.

How Does NetStream Work?

The working process of NetStream is as follows:
  1. An NDE periodically sends collected flow statistics to an NSC.
  2. The NSC processes the flow statistics and sends them to an NDA.
  3. The NDA analyzes the flow statistics, helping implement accounting, network management, and network optimization.

The following figure shows NetStream processing on a device acting as an NDE.

NetStream processing on an NDE
NetStream processing on an NDE
  1. The device performs NetStream sampling on service traffic in a specified sampling mode.
  2. The device creates NetStream flows for sampled packets based on values of key fields.
  3. The device ages out NetStream flows that meet certain aging conditions.
  4. The device exports NetStream flows in a specified mode.

Key Implementations of NetStream

NetStream Sampling

Only statistics on sampled packets are collected. Such statistics present a sufficiently accurate representation of the states of network-wide flows. Setting an appropriate sampling mode is critical to reducing the impact of NetStream on the performance of the device.

The following NetStream sampling modes are supported:

  • Packet-based random sampling: One packet out of a specified number of packets is randomly sampled.

    For example, if the number of packets is set to 100, one packet out of every 100 packets is randomly sampled.

  • Packet-based regular sampling: One packet is regularly sampled at a fixed packet interval.

    For example, if the packet interval is set to 100 and packet sampling starts from the 5th packet, the 105th packet is sampled and the rest packets are sampled in the same manner.

  • Time-based random sampling: One packet is randomly sampled at a specified time interval.

    For example, if the time interval is set to 100 ms, one packet is randomly sampled among packets that pass through the device within every 100 ms.

  • Time-based regular sampling: One packet is regularly sampled at a fixed time interval.

    For example, if the time interval is set to 100 ms and packet sampling starts at the 5th ms, one packet is sampled at the 105th ms and the rest packets are sampled in the same manner.

NetStream Flow Creation

You can set a proper sampling mode on an interface to sample service traffic. After obtaining sampled packets, the NetStream module creates NetStream flows based on values of key fields in the packets. Packets with the same key field values belong to the same NetStream flow.

The key fields based on which NetStream flows are created vary according to packet types.

NetStream Flow Aging

After the NetStream function is enabled, the device stores NetStream flows in the cache. When the flows in the cache meet aging conditions, the device ages out the flows and exports them to the NSC.

NetStream flows can be aged out in the following modes:

  • Active flow aging

    A flow is sampled within a specified period, starting from the time that its first packet is sampled. When this specified period elapses, statistics about this flow are exported. This type of aging is known as active flow aging.

    It is suitable for analyzing flows that last for a long period and for periodically exporting flow statistics.

  • Inactive flow aging

    If a flow is not sampled within a specified period, counting back from the time that its last packet is sampled, the device exports statistics about this flow to the server. This type of aging is known as inactive flow aging.

    This aging mode clears unnecessary entries in the cache in a timely manner to save memory space. It is suitable for analyzing flows that span a relatively short period of time.

  • FIN- or RST-based aging

    A flow is aged out when the FIN or RST flag is detected in the packets of the flow.

  • Oversized NetStream flow table aging

    The device limits the size of the NetStream flow table, and automatically ages out excess flows when that limit is exceeded, thereby ensuring accurate statistics.

  • Byte threshold-crossing aging

    When the number of bytes in NetStream flows in the cache exceeds the upper limit, the recording of new statistics will cause cache overflow and inaccurate statistics. To prevent this, the device ages out a flow as soon as it detects that the number of bytes in the flow exceeds the limit.

  • Forced aging

    You can run a command to forcibly age out all NetStream flows in the cache.

    Forced aging is used when the aging conditions are not met but the latest flow statistics are required; or when flows in the cache cannot be aged out due to an exception of the NetStream service.

NetStream Flow Export

The device exports statistics about aged flows from the NetStream cache to a specified NSC for further analysis.

  • Original flow statistics export

    In this mode, statistics about each flow are exported to an NSC when the flow aging time expires.

  • Aggregated flow statistics export

    The device aggregates the original flow statistics with the same aggregation key items, and then exports the aggregated statistics to an NSC. This mode can significantly reduce the occupied network bandwidth.

    Currently, the device supports the following aggregation modes:

  • Flexible flow statistics export

    You can define the conditions for creating NetStream flows as required. The device collects statistics on the user-defined flows and sends the statistics to an NSC. Compared with original flow statistics export, flexible flow statistics export consumes fewer bandwidth resources.

  • Layer 2 flow statistics export

    The device collects Layer 2 information of traffic on the network and sends the information to an NSC.

NetStream packets support three versions: Version 5, Version 8, and Version 9. Packets of all versions are transmitted using UDP.

  • Version 5: exports original flow statistics based on 7-tuple information. The packet format is fixed and cannot be extended.
  • Version 8: supports aggregation data export. The packet format is fixed and cannot be extended.
  • Version 9: exports flow statistics flexibly through combinations of various data formats that are defined in a template.

Implementation of NetStream flow export depends on the selected flow export mode and NetStream packet version.

Application Scenarios of NetStream

NetStream collects statistics on and analyzes service traffic on the network. It can be deployed at the access, aggregation, and core layers.

NetStream is typically applied for the following purposes:

  • Accounting

    NetStream provides fine-grained data for time-, bandwidth-, service-, and application-based accounting. For instance, enterprise customers can calculate department expenses or allocate budget to effectively use resources.

  • Network optimization

    NetStream provides key information required for network management, such as network traffic statistics, to optimize network planning and achieve optimal network performance and reliability with the minimum operating expense (OPEX).

  • Network monitoring

    NetStream can be deployed at the network egress to monitor network traffic in real time and obtain the egress bandwidth occupied by various services. Based on such bandwidth information, network administrators can identify inappropriate network planning and adjust network resource allocation in a timely manner to ensure stable and reliable running of network services.

About This Topic
  • Author: Liu Jieyuan
  • Updated on: 2021-12-03
  • Views: 10710
  • Average rating:
Share link to