Chipset Aftermath: Real World UX, Strengths & Weaknesses

The one debate that we often find discussed on Android forums and websites at every new superphone release is “do benchmarks matter?” The response is usually something that implies that benchmarks don’t have a correlation with the resulting user-experience. Believing any of both extremist notions – that is, that benchmarks are an absolute indicator or that they do not correlate to real world UX – is a myopic practice that often leads to disinformation in whichever discussion is taking place. With both the Galaxy S6 and the One M9, the debate took place once more.

First, let’s remember the context: Qualcomm’s Snapdragon 810 was reported to have overheating issues way before its prime time, and these were followed by rumors that, for this very reason, Samsung would opt for their inhouse Exynos 7420 SoC instead. This is what ended up happening, and the Snapdragon 810 suffered from intense throttling that resulted in some issues for consumers. We originally took a look at early benchmarks that indicated that the Snapdragon 810 was behind the Exynos 7420 by a landslide in a variety of ways. But, like we predicted, the Snapdragon 810 actually featured very good real-world performance.

So, what is the aftermath? Which device won? How does the Snapdragon 810 really compare to the Exynos 7420, and its predecessors too, in the real world and in that of synthetic benchmarks? Are there differences when gaming, multi-tasking or simply launching apps? And why? In this feature we will begin exploring these questions, and hopefully come up with some answers.

Res Cogitans – Synthetic Look

CPU Performance

The Snapdragon 810 was first analyzed in the MDP/T model where it showed extremely promising CPU performance. In many ways, it matched the astoundingly powerful Tegra K1 processor found in the Nexus 9, but at the time it was expected that this chipset would circumvent heat and battery constraints (something Nvidia couldn’t do, and thus the K1 is strictly a tablet chipset) to deliver that kind of performance. Sadly, the Snapdragon 810’s overheating reports were true, and in both the G Flex 2 and M9 we see a significant drop in CPU performance due to system throttling. Now that a thorough analysis of the M9 has been conducted, we can piece together what is truly happening to HTC’s flagship and its overheating Snapdragon.

According to AnandTech, the CPU of the SoC has clear throttle constraints, and the processor’s listed frequency of 2.0GHz is actually limited to a maximum of around 1.6GHz for regular use. It is only when HTC’s systems detect a benchmark that this cap is lifted, to allow for enhanced scores that, due to
the nature of the downgrade, can not represent real-world performance. This is something that some apologists called “a non-issue” – and claimed it was so as long as the real-world performance didn’t suffer. The SD810 here shows remarkably worse performance in virtually every metric than what the reference MDP/T model featured, but we don’t know the exact reason for this. Looking at the benchmarks is enough to tell that the device under-performed in this regard.

 

When it comes to the Exynos, its CPU and system performance is simply excellent. If you head over to this benchmark breakdown and analysis, you will see that it consistently makes it to the top spots, in most cases outclassing all other smartphones. The only room for improvement is in GPU system performance, and that’s something we’ll touch on below.

GPU Performance

The Adreno 430 in the Snapdragon 810 was expected to be a “beast” of a graphics processor, and it indeed was at release. Unlike the CPU, the GPU still features good performance and is more or less in-line with expectations. As a result, the graphics output of the M9 is actually rather excellent, and unlike many thought, things like video games or GPU-intensive tasks perform excellently on the M9. In fact, when you compare the graphics-intensive benchmark scores to those of the S6, you see that the S6 falls in behind in many tests. The reason should be obvious to many, but we’ll touch on it shortly nonetheless.

The fact that the GPU is powerful and doesn’t suffer from the throttling constraints of the CPU is a great benefit for the M9, but the Exynos S6 also features very good graphics performance. In fact, in off-screen tests the device typically surpasses the M9, but not on on-screen tests. The Mali T760 MP8 inside the S6 is more powerful than the Adreno 430, which is quite a surprising outcome considering the history of both GPU lines, but the M9 is powering a 1080p panel, and not a 1440p one. This is a very important difference to take into account when looking at benchmarks – particularly on-screen results – as the S6 has to push 78% more pixels than the M9. When you take this into account, the S6 is doing a remarkable job. But what does this all mean?

Throttling

ArsTechnica ran tests to get a better look at the throttling situation of these newer chipsets, and see just how bad the throttling constraints of the Snapdragon 810 were. Their results are very enlightening, and show that the Exynos 7420 features a much tamer throttling than the Snapdragon 810. In fact, the Snapdragon 810 throttles itself nearly instantly in the G Flex 2. The Exynos in the S6 can actually sustain its maximum frequency for quite a while, and once the throttling kicks in it still runs at higher frequencies than the G Flex 2 (keep in mind, the S6 only has a 100MHz advantage when it comes to maximum frequency). While the G Flex gets throttled before the minute mark, the S6 lasts close to 2 minutes at high frequencies and then drops to a clockspeed that still remains higher than the G Flex (~1700MHz to ~1450MHz). The SD810 lowers its frequency even lower while the Exynos sustains it at 1.7GHz for quite some time. The story is in the graphs, and they quantify just how bad of an issue it can be. After just a few minutes of continuous use, the probability of encountering slowdowns increases dramatically.

 

Res Extensa – Real-world Look

Real-world Performance

The devices are out and we’ve gotten plenty of comparison articles and videos, as well as deeper look at the benchmarks of these devices. It turns out that the M9 has very good performance for real world use, particularly due to its light software. The Sense UI in the M8 made for one of the smoothest and speediest experiences on any Android flagship of 2014, and the trend remains in the M9. Reviewers have reported some stutters in certain areas of the UI – some more consistent than others – but when it comes to getting things done, the M9 does the trick just fine. There are many text and video comparisons that tell the story, but we’ll limit this article to two that we think are worth dissecting:

This video shows the strengths and weaknesses of each device flawlessly. As it turns out, however, the weaknesses of the S6 have little to do with its hardware and more with its software. In the first “lap” of the video, we see that the S6 greatly outspeeds the competition at opening apps and games The faster memory solutions for storage and RAM as well as the processor show off in this segment. On the second lap, the S6’s weakness kicks in: its memory management is atrocious, and despite having the same amount of RAM, many of its apps had to be reloaded. This allowed the M9 to overtake the S6 and “win the race”. On a superficial level, it would appear that the S6 is indeed faster, but the M9 has better/more reliable multitasking. Both devices are still very fast and speedy enough to satisfy any user.

There is something that I want to note here, and that is that I too have noticed that Samsung devices have terrible memory management, specially on Lollipop. This is apparent when leaving apps unused for long – even without opening many more – as they have to be reloaded anyway. On a personal note, when the Note 3 launched with 3GB of RAM I was enticed to get it to enjoy better multi-tasking, but we all know how Android’s memory manager is. When I upgraded to Lollipop, the problem became much worse and apps would disappear simply by idling for too long. The Note 4 on Lollipop suffers from a similar problem when I directly compare it to AOSP devices, and shows similar behavior to my Note 3. In fact, this problem is something that I frequently ran into when doing a side-by-side comparison of my Note 3 and my Nexus 5 for the Note 3’s Lollipop performance analysis… 1GB differential and all. Sammobile reported on this problem in the S6, but it is not limited to Samsung’s latest hardware, just the software (Lollipop) as I do not have this problem on 5.1 AOSP ROMs. We do not know if it has to do with Lollipop’s memory leaks, but nevertheless it is quite annoying.

In this test we see the S6 put against the M9 using Gamebench, an application that records metrics during videogames such as frames-per-second, a processor frequency log and battery drain. Here you can see that the S6 actually performs worse than the M9 in games like Asphalt 8. In this particular game, the S6 shows around 29 frames per second while the M9 surpasses it with 41 frames per second. Further testing might reveal variations in these numbers, but is this really surprising? I would argue that, in many ways, it is not. For all the flak that the Snapdragon 810 received, its Adreno 430 GPU remains excellent for graphics-intensive tasks such as games. But the biggest difference is the resolution, which is the last thing I want to touch on.

Resolution

Resolution is by far one of the determining factors in this comparison. As we’ve seen earlier, the off-screen benchmarks show that the S6 actually surpasses the M9 in raw GPU output, but when it needs to put graphics on the 1440p screen, the performance is lowered dramatically. When you consider that the M9 is powering up a 1080p screen, you also run into the surprise that the GPU is not substantially more powerful than what was found in the Snapdragon 805. I want to demonstrate this with my own device:

My Note 4 has a default resolution of 1440p, but I used a terminal emulator on it to reduce the resolution to 1080p. As far as UI fluidity goes, the changes were minimal if not placebo. But I can attest that GPU-intensive tasks such as gaming and synthetic benchmarks showed quite the increase in performance and stability. While Gamebench is seemingly not compatible with my build of TouchWiz, the framerate on Asphalt 8 has improved from 28 FPS (on KitKat) to what I can only describe as ultra smooth. I will do a full feature where I will quantify the difference that the lower resolution made, but for now I think these benchmarks will give you a general idea of the kind of improvements I’ve seen.

Conclusion

Both chipsets can offer good performance, and the S6, G Flex 2 and M9 all flex a good amount of muscle. When it comes to the hardware winner, the S6 is undoubtedly superior. But the software of the S6 holds it back from achieving the real-world UX that we all wanted from Samsung – that is, one that is as good as the hardware can muster. Sadly, there are still clear issues with the memory management, even despite Samsung’s efforts to trim and debloat their TouchWiz ROM. The device is fast, but as the video showed, having to reload an application from scratch can ultimately cost you the race.

When it comes to the M9, however, we see something that is even more worrying: lack of solid progress. As seen in the comparison of the 1080p Snapdragon 805 benchmark, the jump to the Adreno 430 didn’t amount to nearly as much as one would expect from a generational jump like that. In many ways, the M9 would have been better off by simply going for a Snapdragon 805. Not only is the CPU more stable, but the GPU jump in off-screen tests is also quite close in terms of performance, and the chipset suffers from very little throttling in comparison… because of this, in the real world, the Adreno 430 is not quite what the 30% increase (and 100% increase for GPGPU) in performance that Qualcomm suggested once implied.

When it comes to future-proofing, the S6 has a severe edge (heh): the memory management can be improved through software updates to truly exploit the faster RAM, and you can manually lower the resolution of the display for better graphics performance (I really don’t think anyone needs 1440p in a 5.1 inch display anyway). The M9’s chipset will eventually show its age, but luckily its GPU is strong enough to handle current games with such a surplus that you might find yourself gaming on HTC’s latest for quite a while. Both feature a great real-world user experience as far as performance goes, but it would still have been nice to see the Snapdragon 810 live to its full potential. The upcoming Samsung custom cores and their new factory (which Qualcomm might end up fabbing in) coupled against the Snapdragon 820’s Kyro custom cores might make for quite the chipset battle. Manufacturers are now going for the Snapdragon 808, which might not even fare as well as the 805 either. This semiconductor war might not determine which approach is right, but it could certainly determine which competitor is left. May the best chip win!

The post Chipset Aftermath: Real World UX, Strengths & Weaknesses appeared first on xda-developers.

from xda-developers http://ift.tt/1GFBRU2
via IFTTT