What Is gRPC?
Google Remote Procedure Call (gRPC) is a high-performance open-source RPC software framework based on the HTTP/2 transport layer protocol. It provides an application programming interface (API) design method for managing and configuring network devices. gRPC provides multiple programming languages, such as C, Java, Golang, and Python.
gRPC can be used together with telemetry as a data transmission protocol to accurately monitor the running status of network devices in real time and at a high speed. In addition, gRPC provides the configuration, query, and capability obtaining methods, which are used by devices connected to a collector for reporting device data.
gRPC VS REST
Representational State Transfer (REST) is a software architecture style used to guide the design and development of the web architecture. REST also provides an API design method for managing and configuring network devices. The main differences between gRPC and REST are as follows:
- REST complies with the HTTP/1.1-based request-response communication model, and gRPC complies with the HTTP/2-based client-response communication model.
Compared with HTTP/1.1, HTTP/2 has absolute advantages in speed. Although REST can also transmit data based on HTTP/2, it does not make full use of the advantages of HTTP/2 in order to be compatible with HTTP/1.1.
- Almost all browsers support REST, whereas only a limited number of browsers support gRPC. This is the main advantage of REST over gRPC.
- REST uses the JSON or XML encoding format to carry data, while gRPC uses the Protocol Buffers (ProtoBuf) encoding format to carry data by default.
ProtoBuf is binary and data in ProtoBuf encoding format is transmitted as binary data, while data in JSON or XML encoding format is transmitted in text format. Therefore, gRPC is more advantageous in the transmission rate.
- REST does not provide the built-in code generation function. You need to use tools such as Swagger to generate API request code. gRPC has a protoc compiler that provides the code generation function, and the protoc compiler is compatible with multiple programming languages.
How Does gRPC Work?
gRPC Protocol Architecture
gRPC is a technology used to implement RPC APIs. Because gRPC is an open-source framework, both communication parties perform secondary development based on the framework, so that they focus on services without the need to pay attention to underlying communication implemented by the gRPC software framework. As shown in the following figure, the DATA layer carries service module data, and information at the other layers is encapsulated using gRPC.
gRPC protocol architecture
Operations Supported by gRPC
A device can connect to a collector in the dial-in and dial-out modes.
- Dial-in mode: The device functions as a gRPC server, and the collector functions as a gRPC client, which initiates a gRPC connection request to the device to obtain data or deliver configurations. This mode applies to small-scale networks.
The following operations can be performed in dial-in mode:
- Subscribe: collects interface traffic statistics, CPU usage, and memory usage of the device at a high speed. Currently, this operation can only be performed based on telemetry.
- Get: obtains the running status and configuration of the device. Currently, this operation can only be performed based on the gRPC Network Management Interface (gNMI) protocol.
- Capabilities: obtains capabilities of the device. Currently, this operation can only be performed based on gNMI.
- Set: delivers configurations to the device. Currently, this operation can only be performed based on gNMI.
- Dial-out mode: The device functions as a gRPC client, and the collector functions as a gRPC server. The device proactively establishes a gRPC connection with the collector to push the subscribed data configured on the device to the collector. This mode supports only the telemetry-based subscribe operation and applies to large-scale networks.
gRPC Interaction Process
As shown in the following figure, gRPC uses the client/server model. A network device functions as a gRPC client and a collector functions as a gRPC server. The following describes the gRPC interaction process.
- A gRPC-enabled device functions as a gRPC client, and a collector functions as a gRPC server.
- The device constructs the data format (GPB/JSON) based on the application service (such as the subscribed event) and compiles a .proto file through ProtoBuf. The device then establishes a gRPC channel with the collector and sends a request message to the collector through gRPC.
- After receiving the request message, the collector uses ProtoBuf to parse the .proto file and restores the predefined data structure for service processing.
- After processing the data, the collector needs to use ProtoBuf to recompile the response data and sends a response message to the device through gRPC.
- After receiving the response message, the device ends the gRPC interaction.
Simply speaking, the device proactively establishes a gRPC connection with the collector to push the subscribed data configured on the device to the collector. In the entire gRPC interaction process, the device and collector need to use ProtoBuf to define the .proto file.
gRPC interaction process
Application of gRPC
gRPC supports the subscription function (Subscribe operation) through the telemetry technology. Telemetry is a technology that remotely collects data from physical or virtual devices at a high speed. Devices periodically send interface traffic statistics, CPU usage, and memory usage to collectors in push mode.
As shown in the following figure, a gRPC connection is established between the NMS and the network device so that the NMS can subscribe to data of a specified module on the device. Telemetry supports dynamic subscription and static subscription, which are used in dial-in and dial-out modes, respectively.
gRPC-based telemetry technology
Telemetry implementation process:
- A user defines a static or dynamic telemetry subscription.
- Static telemetry subscription: defined in the huawei-grpc-dialout.proto file.
- Dynamic telemetry subscription: defined in the huawei-grpc-dialin.proto file.
- The user encodes collected information in GPB or JSON format and defines key information including the sampling path and timestamp in the huawei-telemetry.proto file.
- If the GPB format is used, the value of the encoding field in the huawei-telemetry.proto file is 0 (GPB encoding), the data_gpb field carries sampled data in GPB format, and the data_str field is empty.
- If the JSON format is used, the value of the encoding field in the huawei-telemetry.proto file is 1 (JSON encoding), the data_str field carries sampled data in JSON format, and the data_gpb field is empty.
- The device transmits data to the collector, decodes the data, and analyzes the decoded data.
- The data_gpb field in the huawei-telemetry.proto file needs to be decoded by the .proto file of the corresponding service, which is determined by the sensor_path field in the huawei-telemetry.proto file. For example, if the value of the sensor_path field is huawei-ifm:ifm/interfaces/interface, the .proto file is huawei-ifm.proto.
- If the pure JSON encoding format (both the telemetry layer and data model layer use the JSON encoding format) is used, the user only needs to decode the huawei-grpc-dialout.proto or huawei-grpc-dialin.proto file. If the hybrid JSON encoding format (the telemetry layer and data model layer use the GPB and JSON encoding formats, respectively) is used, the user only needs to decode the huawei-grpc-dialout.proto or huawei-grpc-dialin.proto file and the huawei-telemetry.proto file. The .proto file of a service is not required.
What Is gRPC ProtoBuf?
gRPC ProtoBuf is an interface description language of the gRPC protocol. It is a language-neutral, platform-neutral, and extensible mechanism for serializing structured data of communications protocols and data storage. The gRPC ProtoBuf encoding format is also called the Google Protocol Buffers (GPB) encoding format. GPB provides a flexible, efficient, and automatic mechanism for serializing structured data. Different from XML and JSON, GPB is a binary encoding format with high performance and efficiency.
Two GPB versions are available: v2 and v3. Currently, the device supports GPB v3.
GPB provides the following functions in the gRPC framework:
- Define the data structure.
xxx.proto
syntax = "proto3"; //The .proto file version is defined as v3.
message serviceArgs { //Message format description.
int64 ReqId = 1; //Request ID.
oneof MessageData {
bytes data = 2; //Sampled data in GPB encoding format is carried.
string data_json = 4; //Sampled data in JSON encoding format is carried.
}
string errors = 3; //Error description.
}
- Define a service interface.
xxx.proto
syntax = "proto3"; //The .proto file version is defined as v3.
package huawei_dialout; //The package name is huawei_dialout.
service gRPCDataservice { //The service name is gRPCDataservice.
rpc dataPublish(stream serviceArgs) returns(stream serviceArgs) {}; //The dataPublish method is used to push data. This method uses the bidirectional streaming mode. The input parameter is serviceArgs.
}
- Improve the transmission efficiency through serialization and deserialization.
The content in GPB encoding format is provided only for the operator to read. Actually, the content is transmitted in the form of serialized binary data instead of text. The content in JSON encoding format is displayed and transmitted both in the form of data text. Therefore, the transmission efficiency of the GPB encoding format is inherently higher than that of the JSON and XML encoding formats.
GPB Encoding
JSON Encoding
{
1:"HUAWEI"
2:"s4"
3:"huawei-ifm:ifm/interfaces/interface"
4:46
}
{
"node_id_str":"HUAWEI",
"subscription_id_str":"s4",
"sensor_path":"huawei-ifm:ifm/interfaces/interface",
"collection_id":46,
}
What Is a .proto File?
gRPC uses the GPB encoding format to carry data. The file name extension of a file in GPB encoding format is .proto, indicating that the file is a .proto file.
GPB uses a .proto file to describe a dictionary for encoding, which is the data structure description. The collector can use a tool such as Protoc to automatically generate code (for example, Java code) based on .proto files. Users then can perform secondary development based on the automatically generated code to parse the obtained data, implementing data interconnection with devices.
Proto files include general-purpose .proto files and service data .proto files.
General-Purpose .proto Files
Telemetry provides three general-purpose .proto files to support data sending and subscription functions.
- huawei-grpc-dialout.proto file: defines the device as a gRPC client to push data.
- huawei-grpc-dialin.proto file: defines the device as a gRPC server to push data.
- huawei-telemetry.proto file: defines the header for the sampled telemetry data to be reported, including key information such as the sampling path and sampling timestamp.
Service Data .proto Files
The device provides multiple service data .proto files, which are used to define GPB encoding formats of specific service data. The collector needs to select a .proto file based on the service to be monitored.
Example of a Simple .proto File
Static telemetry subscription: A device functioning as a gRPC client initiates a connection to a collector functioning as a gRPC server to send collected data.
huawei-grpc-dialout.proto |
|---|
syntax = "proto3"; //The .proto file version is defined as v3. package huawei_dialout; //The package name is huawei_dialout. service gRPCDataservice { //The service name is gRPCDataservice. rpc dataPublish(stream serviceArgs) returns(stream serviceArgs) {}; //The dataPublish method is used to push data. This method uses the bidirectional streaming mode. The input parameter is serviceArgs. } message serviceArgs { //Message format description. int64 ReqId = 1; //Request ID. oneof MessageData { bytes data = 2; //Sampled data in GPB encoding format is carried. string data_json = 4; //Sampled data in JSON encoding format is carried. } string errors = 3; //Error description. } |
How to Obtain .proto Files
You can log in to the Huawei technical support website to obtain .proto files.
- For carrier users, log in to https://support.huawei.com and browse or search for the product name. On the tab page, select the required version and download the .proto file. Generally, the .proto file is named in the format of ProductName VersionNumber Proto.
- For enterprise users, log in to https://support.huawei.com/enterprise/ and browse or search for the product name. On the tab page, select the required version and download the .proto file. Generally, the .proto file is named in the format of ProductName VersionNumber Proto.
- Author： Tang Jie
- Updated on： 2024-02-27
- Views： 8218
- Average rating：
