The road to HFR was not an unexpected journey for the industry, but harnessing the magic of a new platform is turning out to be a as tricky as Bilbo Baggins trying to decipher old maps in the quest to reclaim the Lonely Mountain and its vast treasure from the dragon Smaug. (Lord of the Rings/Hobbit director Peter Jackson in inset photo.)
Like many important technologies, HFR started with Hollywood. As did the modern loudspeaker and P.A. industry when Hollywood converted from silent films to talkies and the outfitting of movie theaters for sound gave birth to the top audio manufacturing companies that are still huge today. Is HFR another such milestone? The buzz that started last year over the HFR showings of Peter Jackson’s The Hobbit: An Unexpected Journey that opened in theaters in December 2012, raised the question. The Hobbit: An Unexpected Journey (shot on RED EPIC 4K Digital Cameras in 3D at 96fps– 48fps for each eye) premier was much anticipated in the cinema projection/engineering world, because after an April 24th, 2012 HFR demo and taped address in Las Vegas by Jackson at CinemaCon (a demo at a movie industry tech trade event of basically raw footage from a movie that was not even edited, timed or color corrected yet) a variety of bloggers and some industry critics (most of whom just discovered with The Hobbit that movie theaters had converted to digital projection), commented that the HFR images in the demo were not pleasing– too sharp, too detailed, too much like TV and less like film. Some critics were quite vocal in their dislike of the imagery. Given the speed at which the mass media and social media spread this hysteria and due to longer than expected delays in fully preparing enough theaters for HFR projection, Warner Bros chose not to really market HFR to the public with their full PR arsenal as the big studios had done, for example, with the first big 3D launches. So the The Hobbit opened in theaters in December 2012, in a combination of 2D/24 fps in some theaters and 3D/HFR (48fps at 2K/each eye, twice the standard 24fps rate that's been in place for more than eighty years) in most theaters, with not much hype around HFR. But the launch was huge, for the industry.
When the movie premiered, many movie critics felt vaguely compelled, after the spring 2012 media mini-frenzy after CinemaCon, to address the HFR issue– and most confused their opinions of the other aspects of movie– the plot, the lighting, the makeup, the computer generated graphics (CGI)– with HFR issues. Better said, they judged HFR based on whether or not they liked the movie as a whole including the other technical aspects of the movie. Understandable, but unfortunate.
I thought the scenes in the The Hobbit: An Unexpected Journey with the more natural interiors and involving non-CGI backgrounds (like the opening of the movie in the Bilbo’s hobbit hole) acquitted HFR extremely well. It was a lush, detailed high contrast image– an improvement in image quality all around in my book, over a 3D 24fps image. It was more high res. The increase was of course not an X/Y increase of pixels, but an increase of sequential resolution– more resolution nonetheless. And I think those more normally-shot, less CGI scenes are the best ones with which to judge HFR, despite all the discussion of HFR helping 3D action scenes that have lots of camera panning.
Post-Hobbit release research shows a demographic split: under 30’s liked HFR; older demographic were basically neutral– and there was not significant negative reaction from any demographic, despite some vocal small minorities. But whether you loved or hated The Hobbit in HFR, Peter Jackson’s Hobbit was something of a canary in the coal mine test for the video projection industry– for reasons well beyond just HFR.
The release of The Hobbit was a “limited release” of HFR. Both in scope (promotion of HFR to the public, and the number of theaters involved), and kind (the specs of the DCP). Warner made a fairly heroic effort to help get theaters ready for a full HFR event– but it was a race against the clock. After the less than ideal and premature showing of HFR at CinemaCon in April 2102, and facing the December 2012 Hobbit release deadline, they backed off their initial hope to make The Hobbit/HFR release a milestone not just for HFR, but a way to push the entire industry toward a faster adoption of full SMPTE DCP standards (DCP stands for the digital cinema package).
As the December 2012 Hobbit release date loomed large in the summer/fall of 2012, tensions mounted. 2011-2012 had seen the trend of placing the media block directly into the projectors of the big DLP Cinema projector manufacturers–without this new technology, HFR wouldn't have been possible. But there was a question that moving from HFR back to normal frame rate might be problematic for projectors– because unlike 24fps 3D content, 48fps 3D content needs to have the projector set for double flashing instead of triple flashing, done by creating a channel setup in the projector that enables 3D playout with the frame rate multiplication set to 4:2 (double flash), instead of 6:2 (triple flash). And there was question as to the readiness of the new technology to move to the new SMPTE DCP. How would the new equipment in which theater owners had invested millions of dollars respond to SMPTE DCP? No movies had been released in SMPTE DCP yet. Nail-biting at Warner ensued.
So when it came down to the wire in late 2012, two things happened. The good news we all know: Warner made The Hobbit release deadline, with HFR content. But many in the industry considered it bad news that The Hobbit DCP went out only in “Interop”. The Interop DCP (digital cinema package– or the package of files/code that makes up a full digitized movie file ready for projection in the theater) was created at the onset of digital cinema and is a kind of SMPTE DCP “light”, based on early standards drafts to promote interoperability at the beginning/middle of the digital cinema rollout. Interop DCP is not fully SMPTE DCP compliant. But when it came down to it last year in the thick of The Hobbit race, Warner was not going to throw any more money into pushing more theaters, and gear manufacturers, toward supporting a fully SMPTE-compliant DCP– one that would do HFR– in a gamble on one movie’s debut. Ironically, when it came down to the finish line, the problems with theaters attempting to play the full-fledged SMPTE DCP version that Warner provided some test theaters were about correctly routing SMPTE audio labeled channels in the theater. Not a video or HFR problem but an audio one– but enough to derail a fully SMPTE-compliant rollout. It’s important to note that image and sound quality, in today’s Interop, are identical to SMPTE DCP. Security is identical. So that is not an issue in the HFR debate. But the SMPTE DCP/Interop issue is highly important. Interop was introduced when there were only a few hundred digital screens, and it was envisioned to last for only a short time until standards were finished. Today there are nearly 100,000 digital screens. Some percentage of those will never be able to run SMPTE DCP without further replacement of expensive equipment. Some percentage won't be able to support all of the features in SMPTE DCP– which is why Hobbit had to be released on Interop. All of this speaks directly to the issue of whether HFR or any other new technology rollout in commercial movie theaters can or should be a catalyst to take either that technology or more importantly the very health of ticket sales and consumer experience to newer, higher levels.
It is unfortunate that so much was riding on one HFR movie release– in terms of the critics/viewers reception of the new technology, and the movie theaters being pushed to comply for one tent-pole release. Yet most movie theaters were ready for even a SMPTE DCP HFR rollout last year, had it happened. And it’s remarkable that the HFR premiere went as smoothly as it did.
Drew Kaza, EVP Digital Development Odeon UCI Cinemas, Europe’s largest chain of theaters, with 2200 screens in seven countries, said at the High Frame Rate session at NAB this past April, “with our NEC series II projectors, with Doremi servers, we were in a good position to roll it out.”
In fact it is inspiring to see how quickly and nimbly almost all the big cinema chains and smaller ones were able to get behind a new technology push. Pacific Theaters/Arclight Cinemas converted twelve screens to HFR in six weeks in late 2012.
“We didn’t use ‘silver screens’, we used white screens,” said Joe Miraglia, Director of Design, Construction and Facilities at Decurion Corporation, that owns Pacific Theaters/Arclight Cinemas. “We partnered with a dealer for installation and calibration. We did not upcharge moviegoers for 3D, or HFR. We ran the projectors at 100% of capacity– you have to really keep on top of bulbs. Guest comments were positive. Bottom line, we had a tent pole picture opening up, with new technology, no projection incidents. Warner worked hard, so we call it a success.
Drew Kaza summed up the feeling of most of the industry. “We were ready for 48fps. We went into it believing that Peter Jackson was right. If he wanted it, there had to be a good reason. And we are firm believers in new technologies. There are lessons learned from The Hobbit release. Start sooner. Go wider. Commit fully and market it passionately. But if someone came along next year, and said, let’s go 60 fps, we’d have to see some reason. But if that someone was Cameron, we’d probably do it.”
After The Hobbit/48fps drama, is the industry ready for the brash-talking James Cameron to pull them into 60fps? Indeed Cameron has been on a soapbox for two years, saying that 48fps isn’t enough– and all but promising that the next Avatar would be shot and presented in 60fps. Avatar II shooting has not begun, and the question of whether that will be 48 fps or 60 fps still hasn’t been answered publicly. Cameron likes to keep the industry guessing. And Cameron is not worried about the limited but vocal “the magic looks too real” backlash against HFR from some Hobbit reviewers and audience. Mainly because almost all the effects created on an Avatar production are CGI and not as human actor/make-up¬ and physical prop-reliant as the Lord of the Rings/Hobbit productions. Peter Jackson is more of an old-school director who uses technology to enhance shoots involving lots of human actors in makeup and real costumes running around (real) mountains in New Zealand. The fantasy starts out literary, and becomes theatrical– and involves lots of compositing of real scenery with CGI. Cameron creates virtual sets from the get go (and uses virtual lighting on both CGI and real sets)– the “magic” is all magic, and almost all CGI.
Cameron and Avatar producer John Landau are probably more worried this summer about how to deal with the “avalanche of data” that the Avatar 2 HFR shoot is going to release– as Phil Oatley, head of technology at Park Road Post, who did post on The Hobbit production last year, described the 6-12 TB (terabytes) of original shoot data from the cameras each day of the The Hobbit: An Unexpected Journey shoot. Between that challenge, and the fact that they’ll be shooting a lot of the new Avatar movie under water (Cameron’s favorite place to play but a darned hard place to shoot), the few HFR skeptics in the press are the least of the Cameron’s worries. Could Cameron’s water obsession put the production into Jackson territory– too much reliance on real scenery capture to mesh seamlessly with the CGI under the warts-and-all bright lights of HFR– the even higher data capture rate 60fps would provide? (Interestingly, the special effects loving Cameron has one of the most subtle touches in the industry with 3D– not putting it in your face but using 3D to move the viewer’s eye within the frame the same way directors have used depth of focus for decades for that purpose. He needs the smoother 3D panning/tilting that HFR provides less than most 3D directors, despite his outspoken stance on the benefits of HFR for 3D action and 3D panning/tilting specifically.)
Some of the word behind the scenes in the SMPTE world this summer is that 60fps could be too much of a gamble and Avatar will go out in 48fps. No one will say this publicly, and Cameron and team are still hyping 60fps in general while not committing to anything. But there are fears that if shot at 60fps, there's no clean way to play the movie at 24fps in some theaters– an easier transition for 48fps but even untested in 48 given that that issue helped derail the move to SMPTE DCP for The Hobbit release. And fear that current playback gear can’t do 450Mb/s at 60fps. Gear manufacturers say they can do 500Mb/s, but the studios in reality have to settle for less, due to the bit-rate bottlenecks in various systems. It’s hard to underestimate the power of Cameron and the Avatar franchise (AvatarI was the highest grossing movie in all of film history and his Titanic, the second) to push the industry, but I’d bet on the next Avatar (release date December 2014) being 48fps. A step back, or sideways? No– it would be a huge shot in the arm for HFR– learning from past mistakes and fixing them, not hoping an even higher HFR will somehow avoid them.
Even as we acknowledge the importance of the cinema market as the harbinger of new technology that will no doubt extend out to all of pro video, what HFR issues– beyond the tent pole Hollywood productions like Peter Jackson’s and James Cameron’s latest– are on the immediate horizon?
Wendy Aylsworth, Senior VP of Technology at Warner Bros Technical Operations, and President of SMPTE, managed both sides of the rollout– in HFR– of the Hobbit that released in theaters December 14th, 2012. Aylsworth is shown here addressing the High Frame Rate session at NAB in April 2013. This question is related to 4K issues, because at one level 4K HFR is coming, and on another level both 4K and HFR present image capture, bandwidth, storage and distribution issues due to the larger data pipes and more intensive compression regimes required. And on the broadcast television side, television and commercial AV/digital signage manufacturers–LG, Planar, Sharp, Viewsonic, Sony, Seiki– are starting to release new 4K flat panel displays. We saw them at InfoComm in Orlando. SMPTE for its part has already created a set of technical standards for a new 'Ultra-HDTV' format– and it could be a 4K or even 8K format. Wendy Aylsworth at SMPTE is this year looking at the new issues of moving big data through broadcast channels. “That is the next big step—how do you get that into the home?”
I believe 4K will come to the broadcast TV world before it comes fully to the cinema market, for three reasons: 1) In the broadcast world you can compress the heck out of the image and squeeze down the pipe and no one will complain– and you don’t have SMPTE trying to create bit-rate standards. In broadcast it’s a wild west of bandwidth and bit-rate, where HD or Ultra HD is whatever you say it is. 2) The Hollywood studios went “all in” with the 3D gamble for movie theaters. 3D tent pole releases for the Lord of the Rings movies, and Avatar, to differentiate the commercial movie theater experience from the home theater experience (3D never really took off for home movie watching), means that emphasis on 4K in cinemas by Hollywood is delayed (even though of course all Sony and now many DLP cinema projectors do 4K). Because you can’t simultaneously do 4K and 3D, at HFR, in a movie theater. Showing a 48 fps 3D movie where both the left and right eye images are at 4K is not possible today, due to bandwidth limitations in either the internal media block, the projector, or both. 4K/HFR projection would require up to 4X the input bandwidth of our current cinema projectors. 4K clips placed in an SPL (Show Playlist) configured to play HFR, will play in 2K, with any current system. And, 3) big 4K “tent pole” events for broadcast– the World Cup, and the Olympics– are around the corner. Both 2014 events will be broadcast in 4K, and, I predict, in HFR (60fps)– why?– circle back to 1).
What about HFR in the pro AV, or AV staging world? First, it comes round full circle from the large projector manufacturers that hold the DLP Cinema license from Texas Instruments (Christie, Barco, and NEC). And to the other big player in the cinema market, Sony. What theses companies do, in the cinema market, bleeds over rapidly to us. The first 4K resolution projectors from the DLP Cinema projector manufacturers were released on the cinema market about a year and a half ago.
But HFR, outside the cinema market, is new. And it’s spectacular. It will get into installed AV systems that need the highest res. And it will impress live event producers and stagers who are a bit tired of edge-blending or edge-butting 2K to get the resolution they need.
The first place we’ll see the new generation 4K/HFR projectors is in post production (The Christie projector mentioned above has a quad 3G-SDI connect, in place of the internal media block the Digital Cinema version in a movie theater would have). How long before we see 4K and or HFR on a live event? The 4K part of the question is kind of a silly question on one hand. 2K? 4K? 8K? You can do any of it– have been able to for years– in the staging word. And for that matter, in the world of installed huge screens as in stadiums– either by edge-blending or tiling multiple display devices, or with a lot of LED wall real estate. But single-display, small display resolution milestones are important. The HFR part of the equation is all about bandwidth, a separate issue but an important one in many video arenas. We’ll look at both higher res platforms, and higher bandwidth initiatives, in coming issues.
Judging by the 4K buzz at the InfoComm show last month in Orlando, these high resolution issues are real– not in fact just buzz but a testimony to the increasing cross-over from cinema and broadcast, where high resolution video is not a luxury but a part of a constant race to keep one step ahead of an audience that just keeps on coming back for more– more pixels, and faster pixels.
