RTP (Real-time Transport Protocol) is primarily used to provide end-to-end real-time multimedia data transmission services. It is responsible for delivering multimedia data such as audio and video from the sender to the receiver. RTP itself does not handle the reliability or sequencing of data transmission; instead, it focuses on real-time performance.

 

1.1 How does RTP works？

Here I’m using a typical Audio and Video Call as an example:

① Media Capture
Microphones and cameras capture raw audio and video data.

② Encoding and Compression
The audio (e.g., G.711, AAC) and video (e.g., H.264, VP8) data are compressed through encoding to reduce the amount of data.

③ Encapsulation into RTP Packets
The encoded data is divided into multiple fragments, each encapsulated into an RTP packet, with the following fields:
Sequence Number: Used for detecting and reordering out-of-order packets
Timestamp: Used for playback synchronization
SSRC: Identifies the sender
Payload Type: Indicates the encoding format (audio or video)

④ Sending via UDP over the Network
RTP packets are sent to the receiver using UDP. UDP is a connectionless protocol that is fast but does not guarantee reliability.

⑤ RTP Packet Processing on the Receiving Side
The receiver performs the following after receiving RTP packets:
a.Reordering based on sequence number
b.Playback synchronization based on timestamp
c.Decoding to restore the original audio or video data

⑥ Playback Output
Finally, the restored audio and video data are passed to the player for playback
 

Advantages:

• Strong real-time performance with low latency
• Supports audio and video synchronization
• Scalable, supports various media codecs
• Widely used in systems like SIP and WebRTC

Limitations:

• Does not provide reliability by itself (packet loss and out-of-order delivery must be handled
separately)
• Requires RTCP or other mechanisms for quality control
• Sensitive to network conditions

The following packet capture example is the data of initiating an intercom and making a voice call on a SIP phone.
 

The data illustrates the SIP three-way handshake during the call setup process. After the connection is established, audio data transmission begins. 

The RTP packets contain key fields such as :
Codec Format, Synchronization Source Identifier (SSRC), Timestamp, and Sequence Number.

RTCP (Real-time Transport Control Protocol) is the commander behind real-time transmission, providing control information for RTP, such as network status feedback, synchronization of multiple media streams, and identification of sender and receiver information.

RTCP is not the main carrier of real-time media data, but it periodically sends several types of packets to "monitor" the situation.

① RTCP Runs in Parallel with RTP
• RTP sends media data (e.g., audio/video).
• RTCP runs on a separate UDP port (usually RTP port + 1) and sends control packets 
at regular intervals (usually every 5 seconds).

② Each Participant Sends RTCP Packets
All participants (senders and receivers) in a session send RTCP packets. These packets contain 
feedback or information about:
• The quality of the RTP stream.
• Identification of who is sending or receiving.
• Timing information to help synchronize multiple streams (like audio and video).

③ Common RTCP Packet Types:

 

④ RTCP Helps with Synchronization
• Audio and video often come in separate RTP streams.
• RTCP's Sender Reports include NTP and RTP timestamps, which allow receivers to sync the streams.

⑤ RTCP Provides Feedback for Quality Control
• RTCP lets the receiver tell the sender how the stream is performing.
• For example: "I received 100 packets but lost 5" or "Your stream is arriving late".
• This helps the sender adjust bitrate or quality dynamically.

⑥ RTCP is Bandwidth-Efficient
• RTCP is designed to use no more than 5% of the session bandwidth.
• This ensures it doesn't interfere with the actual media transmission.

Advantages:

• Provides Quality Feedback
RTCP reports include statistics such as packet loss rate, latency, and jitter, helping the sender understand network conditions.
• Supports Synchronization Control
RTCP provides absolute time information (e.g., NTP timestamps), which assists in synchronizing audio and video streams.
• Identifies Participant Information
Can carry identifiers like CNAME (Canonical Name), useful for managing multi-source sessions.
• Lightweight Design
Consumes minimal bandwidth—typically limited to 5% of the RTP session bandwidth—posing little impact on the network.
 

Limitations:

• Delayed Feedback
RTCP packets are sent infrequently (usually every few seconds), making it unsuitable for immediate responses to rapid network changes.
• No Media Transport
RTCP is only responsible for control and reporting—it does not handle error correction or retransmission and cannot directly improve media quality.
• Bandwidth Allocation Dependent
In low-bandwidth environments, RTCP reports may be compressed or delayed, reducing their effectiveness.
• Not Secure by Default
Standard RTCP lacks encryption; it must be used with SRTP to provide security features.

The data shows a standard RTCP packet. The Sender Report includes information such as timestamps and the number of RTP packets sent.
 

The packet is an RTCP (Real-time Transport Control Protocol) Sender Report, used to transmit control information in an RTP (Real-time Transport Protocol) session. It includes sender statistics and network status feedback, such as timestamps, sender data, and quality-of-service (QoS) metrics.

 

Here's a clear comparison to help understand their differences:

 

To summarize the above, RTP is responsible for "transmission" and RTCP is responsible for "management".

★ During Session Setup:
RTP ports are negotiated (e.g., via SDP in SIP/WebRTC).
RTCP typically uses RTP port + 1 (e.g., RTP=5004 → RTCP=5005).

★ During Data Flow:
RTP streams media continuously.
RTCP sends periodic reports (e.g., every 5 sec) with:
Sync data: Aligns audio/video streams.QoS feedback: Lets senders adapt (e.g., reduce bitrate if loss is high).

★ Key Benefits of the Pair
RTP focuses on low-latency delivery (no retransmissions).
RTCP adds monitoring/adaptation without bloating RTP headers.
Together, they enable adaptive streaming (e.g., WebRTC adjusts codecs based on RTCP feedback).
 

5.1 Surveillance and Security System Scenario

Typical Applications: IP cameras (e.g., Hikvision, Dahua), video surveillance platforms
RTP Function: 
● Used to transmit real-time video streams, typically encoded with H.264 or H.265.
RTCP Functions: 
● Detects network interruptions, such as unstable links between cameras and NVRs.
● Provides QoS (Quality of Service) statistics, including average latency and packet loss, helping maintenance teams diagnose network issues.
Example: When a surveillance video freezes or lags, administrators can check the RTCP reports to determine whether the issue is caused by network instability or a device malfunction, allowing for accurate troubleshooting.


5.2 Voice and Video Communication System Scenario

Typical Applications: Zoom, Microsoft Teams, WebRTC, SIP phones (e.g., Asterisk)
RTP Function: 
● Used to transmit real-time audio (e.g., Opus, G.711) and video (e.g., H.264, VP8) data packets.
RTCP Functions:
● Provides network quality feedback (e.g., packet loss rate, jitter), enabling the system to perform adaptive bitrate adjustment.
● Ensures audio-video synchronization by aligning streams using NTP timestamps, preventing lip-sync issues.
Example: In a Zoom meeting, when the network condition worsens and high packet loss is detected, RTCP sends feedback to the sender. This may trigger a resolution downgrade or switch to a more robust codec to maintain smooth communication.

RTP and RTCP are core protocols that enable the reliable delivery and management of real-time audio and video over IP networks.Together, RTP and RTCP form a powerful duo: RTP carries the media, while RTCP monitors and optimizes the media delivery process. They are widely used in applications like VoIP calls, video conferencing, and IP surveillance systems, making them essential protocols in today’s real-time communication landscape.

Understanding how these protocols work—individually and in tandem—enables developers, engineers, and IT professionals to build more robust, responsive, and scalable real-time systems.