In the live video production industry, I think it's safe to say, although I have not researched it thoroughly, that the two fastest growing live video production switching products are Open Broadcaster Software (OBS Studio) and VMix. These two products share two interesting characteristics:
1: Neither came from the live production or broadcast industry.
2: Both are software products.
The idea that “software is eating the world,” as Marc Andreessen has written, has become the generally accepted conventional wisdom, and yet, as is often the case with disruptive events, the incumbent solutions providers are not the primary source of change.
In the broader professional audiovisual world, we can see this play out amongst the proprietary AV-over-IP solution options, with the notable exception of NDI. Each one of the AV-over-IP solutions that were primarily designed for perfect desktop and KVM performance (SDVoE, Dante AV, ASPEED) are, at bottom, hardware solutions.
For the SMPTE ST 2110 open standard, it's a bit murkier. There are software-based SMPTE ST 2110 solutions, but handling uncompressed video over an IP network is processor-intensive work. SmartNICs and hardware accelerators are often used to lessen the impact on CPU and GPU usage. Compressed flows are much easier to work with and SMPTE ST 2110-22, along with the lightweight and visually lossless JPEG-XS, make these streams much easier to handle for broadcast production software. That said, not everyone has a grandmaster PTP clock lying around or IT engineers who know how to design and scale PTP on a real-world network. The bottom line is that while multiple successful software implementations of SMPTE ST 2110 exist, for laypeople, technicians, and other mere mortals, a more accessible solution would be welcome.
Defining the Requirements for IPMX
"By carefully defining limitations such as a default codec, PTP profile, and settings for a receiver’s buffer, we’ve made IPMX software-friendly by default."
When the Alliance for IP Media Solutions (AIMS) set out to define the requirements for IPMX, deference to software implementations and everyday IT environments was established as a core principle. We understood the lesson that NDI has demonstrated so well: AV professionals want an accessible transport technology for use in systems made up of a mix of hardware and software. That said, there are many times they need to turn the dial on quality, latency, and/or reliability to 11, just as they can with SMPTE ST 2110. There are a few important ways that IPMX enables this balancing act.
First, IPMX requires that the base codec be lightweight for software, GPU, and ASIC/SoC implementations. There are only a handful of codecs in this category, and even fewer with the efficiency of JPEG-XS, which is currently the leading candidate codec for IPMX. JPEG-XS has already proved itself in broadcast, and the latest advancements in the codec include support for P-frames, which is information from Previous frames used to increase quality while maintaining subframe latency. This is critical for IPMX, where the extra efficiency bump is required to achieve visually lossless 4K60 4:4:4 desktop video on a 1GbE connection.
Another way that IPMX supports software is through its use of PTP. In IPMX, PTP is optional. Each source can use its own media clock, while the receiver is given a constant stream of timing information (through RTCP Sender Reports) so that it can quickly recover timing. For software implementations and applications that do not require subframe latency, this is often ideal as memory for buffering is abundant, but CPU/GPU cycles are rather reserved for other tasks and low-power operation. When PTP is present, IPMX makes it simpler to handle by backing off the default frequency of PTP delay request packets from eight times per second in SMPTE ST 2110 to once every four seconds in IPMX.
Read the first two articles in this series on IPMX
IPMX's Strength is its Interoperability
IPMX: Media Experience Nirvana?
As a final example of how IPMX supports software implementations, let's consider the Virtual Receiver Buffer, or VRX Buffer. In SMPTE ST 2110, the VRX Buffer model is used as a mechanism for reasoning about the receiver's buffer, and therefore the required behavior and transmission pattern of the sending device. Since the goal is to reduce latency, you'd ideally want to start displaying video as soon as you received the first packet. Given the jittery nature of IP networks, that is impossible, and so the VRX Buffer represents a "leaky bucket," where packets come in the top and leak out at a constant rate from the bottom, starting at a very short time after the first packet is received. The VRX buffer model describes the size of the buffer and when it will start draining.
The trouble is, software implementations can introduce their own jitter, overwhelming or starving some SMPTE ST 2110 receivers, causing a loss of video or degraded quality for the user. While an IPMX receiver may support SMPTE ST 2110’s VRX Buffer model as described by the standard, a new default profile is defined to help overcome this situation. In short, IPMX source devices can depend on the fact that the receiver will not start draining its “leaky bucket” until it is half full, significantly reducing the burden on software-implemented senders and receivers.
By carefully defining limitations such as a default codec, PTP profile, and settings for a receiver’s buffer, we’ve made IPMX software-friendly by default. Because it’s all based on SMPTE ST 2110, it’s simple to interoperate with broadcast systems, or make devices that can seamlessly operate in both worlds. In the end, this gets us to a world where video is just video and we pick the devices that suit our needs best, no matter what they’re made of.
