Add Messaging, Voice, and Authentication to your apps with Vonage Communications APIs
Visit the Vonage API Developer PortalAbout the Vonage Video APIs that enable real-time monitoring of call quality
Building a robust, performant application can be challenging. In order to achieve this, several aspects must be accounted for. The Vonage Video web client SDK provides APIs that allows developers to monitor changes in call quality. By detecting these changes, developers can tune their applications in real time and continuously give users the most optimal call experience.
This topic includes the following sections:
Applications may run on various mobile and desktop devices on different platforms. Additionally, the hardware specification for the devices aren't homogeneous. For example, some mobile devices may have better CPU performance than many desktop devices and vice-versa. The large number of possible hardware configurations — CPU(s), GPU, RAM, hardware encoders/decoders, etc. — means that some tuning may be required. Less capable devices can be configured to disable more CPU intensive features, while more capable devices can default to a more immersive experience.
You can detect changes in the device's CPU load by monitoring the Session cpuPerformanceChanged
event. The event contains a cpuPerformanceState property, which is set to one of the following values:
'nominal' — The device can take on additional work.'fair' — The device can still take on additional work, however battery-life may get reduced; additionally, for devices with fans, the fans may become active and audible.'serious' — The device is stressed, so throttling of resources (e.g., CPU) may occur.'critical' — The device is extremely stressed; if not alleviated, issues may occur.For more details, see this W3C specification.
In response to this event, an application can notify users about resource consumption or disable computationally expensive processes, such as video transformers See the Session cpuPerformanceChanged.
The following code disables video capture when the CPU enters a 'critical' performance state and
reenables it once the state returns to 'fair' or better:
let isVideoDisabledByCPU = false;
session.on('cpuPerformanceChanged', (event) => {
if (event.cpuPerformanceState === 'critical') {
// The application should alert the user why their video is being disabled
publisher.publishVideo(false);
isVideoDisabledByCPU = true;
} else if (event.cpuPerformanceState === 'nominal' || event.cpuPerformanceState === 'fair') {
if (isVideoDisabledByCPU) {
publisher.publishVideo(true);
isVideoDisabledByCPU = false;
}
}
})
The quality of experience that a user perceives from a service can be rated using Mean Opinion Score (MOS).
MOS is expressed as a positive number. The score can range between 1 (the worst quality) to 5 (the best quality):
The MOS rating takes into account several factors, all of which impact (and can degrade) the user experience. These factors include (but are not limited to):
For example:
Use the Subscriber qualityScoreChanged event to monitor changes in audio and video quality. Observing changes in media quality is insufficient, though. Given the real-time nature of call applications, responding to the observed changes by tuning your application to continually deliver the best user experience is necessary as well.
A simple heuristic is shown below. An application is dynamically optimized, based on resource constraints.
// We want to know if the CPU is overloaded. If it is, then we
// can disable certain features so that the best call possible
// can still take place.
let isCpuOverloaded = false;
session.on('cpuPerformanceChanged', (event) => {
isCpuOverloaded = event.cpuPerformanceState === 'critical';
});
// We monitor for changes in call quality. This allows us to
// tune our application, taking into account multiple factors
// (inlined below)
subscriber.on('qualityScoreChanged', (event) => {
const { qualityScore } = event;
const isVideoQualityBad = qualityScore.video < 2;
if (!isVideoQualityBad && !isCpuOverloaded) {
// Subscribe to the highest quality video since the CPU isn't taxed
// and the quality received is good
subscriber.setPreferredResolution('1280x720');
subscriber.setPreferredFrameRate(30);
}
else if (isVideoQualityBad && !isCpuOverloaded) {
// Even though the CPU isn't taxed, the video quality received is
// bad. This might be due to (hopefully) intermittent network issues, so
// we subscribe to lower quality video.
subscriber.setPreferredResolution('320x180');
subscriber.setPreferredFrameRate(7);
}
else if (isVideoQualityBad && isCpuOverloaded) {
// The video quality received is bad and the CPU not being overloaded.
// Let's disable video for now.
// We can enable video once conditions improve. See statement below.
subscriber.subscribeToVideo(false);
}
else {
// Enable video
subscriber.subscribeToVideo(true);
}
});
You can use the Vonage Video Network test library for Web see if the client will support publishing audio and video and report the estimated audio and video MOS scores for a client's published stream. This library uses the Publisher.getRtcStatsReport() and Subscriber.subscriber.getStats() methods to calculate the MOS score.
