I decided that after the last test, The Matrix just didn't cut it anymore as a test source. Codecs no longer had the problems with this particular source as they had when I started making codec tests 3.5 years ago. Being a fan of the Matrix series, I looked at the sequels, and finally settled for Revolutions, which I found had the most difficult scenes.
I picked a bunch of shorter scenes that I had a look at: The first 1.5 minutes with the famous Matrix characters scrolling off the screen and a computer representation of an entire city, then Neo's waking up scene in the subway (3:56 - 4:45), the fight in the club's atrium up until the point where the Frenchman has the music stopped in the club (15:45 - 18:52), a scene where we have a ship racing through underground tunnels followed by a swarm of squids (1:01:59 - 1:02:40), the beginning of the squid attack on Zion (1:03:40 - 1:04:24), Neo in the machine city (1:38:40 - 1:40:00) and finally the beginning of the fight between Neo and Agent Smith (1:41:54 - 1:43:50).
It is now possible to switch between the screenshots for a certain scene using the links below the image. The comment concerning the screenshot that is currently shown is displayed below the screenshot. By default, the source image from the DVD is shown. Many thanks to Zolaerla for the image switching and zooming script.
Then my usual disclaimer: Screenshots are not necessarily representative of the video quality. It might be that a screenshot of codec A just happens to fall onto an I-frame which is the least compressed (and thus looks pretty good), whereas quality quickly degrades after the frame shown in the screenshot. And a codec B might have a heavily compressed B-frame at the same position, so comparing the two would be inherently unfair. Bottom line: Take screenshots with a grain of salt, make sure you read all my comments and perform your own tests when in doubt. Every human being perceives visual quality differently and you might come to completely different conclusions watching the same clips as I did. Also, this time I made a point of looking at the source throughout the comparison so I'll have some comments on quality compared to the source as well.
Scene 1: Intro
I've always found that even in the first Matrix, codecs had a lot of trouble with the intro, even starting with the Warner Brothers logo, and I was proven right this time. Codecs hardly like black backgrounds (and many had few nasty block in scenes that should be pitch black), and scrolling fonts also seem quite difficult. I made a point going a bit longer than just the intro to see the first few actors to see how a transition between purely computer generated content and live content can be handled.
Overall, it seems to be rather difficult for ASP codecs to keep black black, but even proprietary encoders have this. The AVC encoders show how it's done, and NeroDigital clearly dominates in this scene, followed by XviD and VP6.
Scene 2: The subway
As Neo wakes in an almost white subway station, the codecs are hard taxed to keep the tile structure of the walls.
There is a definite possibility for improvement in this scene for all codecs. I'd especially like to see how the High Profile AVC encoder handles once it adds film grain (which is part of the High Profile specs).. I'd expect that with the grain efficiently stored, there'd be more room left to draw all those tiles the way they're supposed to be.
Scene 3: The Club
What would Matrix be without a nice shoot-out? Fortunately, they reintroduced the lobby shoot-out in Matrix3:
Immediately after the fight our protagonists enter the club:
Scene 4: The tunnel
Hold your food, we're going for a ride..
The tunnel is definitely a challenge for all participants. There's no representation that I'd consider good, but we have okay, so so and outright bad results.
Scene 5: Attack on Zion
As thousands of squids are met with hundreds of thousands of bullets, there's bound to be work for our contestants.
It feels almost as if the squids were attacking our contestants, doesn't it?
Scene 6: Machine City
As Neo enters the heart of the machine city, our contestants have to deal with dark scenes, lots of blue, and a lot of small things flying around Neo's face.
Scene 7: Neo faces Agent Smith
If you think you saw fighting in Matrix 1 & 2, here's the ultimate battle: Neo and Agent Smith face off in pouring rain, with thousands of Smith copies watching.
Clearly, rain is something that'll follow us in future comparisons as contestants are not very adept at representing it.
Conclusion
This was the last movie I reviewed, and just when I was getting a little frustrated thinking my sources weren't tough enough, Matrix3 came to my rescue. While shorter than the first Matrix, clearly this is yet unmastered territory. If anybody claims any codec can deliver DVD quality at 700MB for a 2h movie, have him encode Matrix3 and compare it to the DVD. And if he still thinks so, send him to an eye doctor.
I found that ateme's offering performed best in this test. ND HP has a slight edge over ND MP, but pays it with encoding time. And, the differences are very hard to spot. In fact, I expected a bit more from a High Profile codec, but then again it's very early, and generally early for AVC codecs. But the result of the first comparison where I've included AVC clearly show where we're heading in the future. The ND encoders are followed by XviD and then VP6.
Just recently, somebody in my forum dug up an old 3ivX interview (from March 04), and apparently has managed what they've announced: quality like DivX, speed like NeroDigital. Okay, NeroDigital ASP delivered 50.99 fps on my old box, so it should be considerably faster on my new machine (plus offer improved quality), but I think they reached their goals. The results of HDX4 and RV10 are often similar. Perhaps that's because they appear to conserve grain. But, whereas details are smoothed out in RV10, they appear a bit blocky in HDX4. I'd probably have tended to give RV10 a better rating because of the almost complete lack of blocks, but taking into account how the source looks, I'm afraid a visually pleasing impression is just not good enough if the source is stripped of a lot of details.
Last but not least, we have WMV9 and VSS. VSS imho has a problem with rate control, a problem which clearly can fixed (and I can remember other codecs having this issue, for instance early 3ivX releases) but currently the encoder needs a lot of work. The discoloration in WMV9 also left unsatisfied, though below the discoloration, things were really blocky, and sometimes reminded me of DivX3 (we'll get to that topic again later on).
Now let's move on to Saving Private Ryan. If you think you've seen enough you can proceed directly to the conclusion.
This document was last updated on December 28, 2004