Samsung accused of altering TV behavior to look better in benchmarks

Samsung has been accused of benchmark manipulation by two TV testers. The tech megacorp has found itself in hot water regarding benchmarking, and not for the first time, after some of its TVs have been seen to alter brightness when certain criteria are met.

According to testing carried out by YouTuber HDTVTest on the Samsung S95B QD OLED, and by website FlatpanelsHD on the QN95B QLED, these two 4K HDR TVs run differently under certain test conditions compared to how they perform in everyday use.

HDTV explains it well in reference to the S95B: it comes down to the TV accurately matching luminance, EOTF (Electro-Optical Transfer Function), and colour when using certain test window sizes (5%, 10%, 20%) most often used for benchmarking and certification. But when using non-standard window sizes to test, or when using the screen for normal day-to-day viewing, it doesn't offer as accurate a picture as it seems. This can lead to an over-brightened picture.

This accuracy is important for matching the reference HDR image on the TV, rather than boosting colours or brightness where they shouldn't be.

These aren't cheapo TVs, either. These models range from $2,000−$6,000.

FlatpanelsHD experienced similar behaviour on the QN95B, and reached out to Samsung Korea for comment.

"Samsung remains committed to relentless innovation to provide the best picture quality to our consumers," it says. "To provide a more dynamic viewing experience for the consumers, Samsung will provide a software update that ensures consistent brightness of HDR contents across a wider range of window size beyond the industry standard."

Samsung is addressing what it sees as a confirmed issue with HDR content across various window sizes on the affected screens, though it's not clear if the claimed issue is present on other screens.

The Register has since reached out to Samsung regarding the test results, and it responded that "Samsung Electronics does not use any algorithm for the purpose of yielding specific test results."

It also claims its own tests "show that HDR content is accurately displayed on various window sizes, and not just at 10 percent."

So the jury's still out on this one, but it does appear that these TVs are acting weird when they shouldn't be, and even Samsung has admitted to that. It says it has rolled out an update for the S95B to adjust its behavior, but the QN95B may still be awaiting a patch.

Samsung's track record is far from flawless when it comes to these matters. It once settled a case regarding speeding up benchmarking applications on its Galaxy S4 phone, and similar cases were alleged against it for the Galaxy Note 3. More recently, benchmarking software Geekbench removed support for many modern Galaxy phones, including the latest S22, for alleged manipulation.