On November 13th, 2012, Silicon Studio released MOBILE GPUMARK, which is a 3D benchmarking software available at no cost for the Android and iOS platforms. MOBILE GPUMARK is currently available on the App Store and Google Play.

• Click here for the Android version of the MOBILE GPUMARK software (Google Play)
• Click here for the iOS version of the MOBILE GPUMARK software (App Store)

Evident by the supported platforms, MOBILE GPUMARK is a benchmark program for smartphones and tablet devices. Moreover, the use of the term "GPU" reflects its focus on measuring graphics rendering performance for games, rather than general application performance.

The 3DMark series by Futuremark has established itself as the industry standard among PC gamers, and the company has announced that the newest version of 3DMark will be compatible with the Android and iOS platforms. Additionally, while GLBenchmark (developed by Hungary's Kishonti Informatics) is becoming the industry standard for smartphones and tablets, much attention is focused on Japanese developers as they take their place in this competitive marketplace.

I was able to talk to one of the creators of this software, and I will use this interview to explain more about MOBILE GPUMARK.

The Birth of MOBILE GPUMARK

Silicon Studio is a comprehensive middleware development company that has already made its mark in the gaming industry by developing software such as the all-in-one game engine OROCHI, the post-effects middleware YEBIS, the graphics library and renderer DAIKOKU - offering high dynamic range rendering support -, and the particle effects middleware BISHAMON.

Any attempt to list the games developed using Silicon Studio’s software would be endless, but some recent examples include "Gunslinger Stratos" using OROCHI, the "Dynasty Warriors" series using YEBIS, and "Child of Eden" which was made with BISHAMON. Silicon Studio has also developed their own game "3D Dot Game Heroes," which is distributed by From Software.

Silicon Studio is already respected as a studio with extensive technical capabilities, and the creation of this benchmark software aimed at smartphones and tablets is certainly a new venture for them.

Osamu Goto (Software Engineer, Silicon Studio)

MOBILE GPUMARK's lead developer, Mr. Osamu Goto, recalls that ”MOBILE GPUMARK started as a small in-house project launched in late fall of 2011 as a part of research to make YEBIS 2 compatible with smartphones and tablets.”

The first application of this in-house project was to port the "NATURAL BONE" demo – a program originally for DirectX 9 GPUs in which a young tribal man is attacked by mysterious spheres in a jungle – to the Android and iOS. As a result, instead of simply creating a YEBIS 2 Demo for Android and iOS, the overwhelming response to the success of this ported version led Silicon Studio to switch gears and begin work on a benchmark application aimed at a wider audience.

This change in direction generated comments from inside the company indicating that NATURAL BONE as a scene composition demo looked a bit lonely on its own. In response, Silicon Studio began development on "DEAD PARKING," an entirely new test sequence depicting a horde of zombies closing in on a young boy. Upon completion of this test sequence, development turned their attention into creating a fully-fledged product in April, which formed the origin of GPU BENCHMARK. In May, the creation of RIGID GEMS, portraying sparkling, beautiful gems, brought the software closer to its final form. 　Incidentally, RIGID GEMS was originally a free software GPU demo andafter its creation, the developer of RIGID GEMS joined Silicon Studio, leading to its addition to the project.

In July, Silicon Studio introduced a system to aggregate calculated benchmark scores in the form of an online ranking. The company's BENTEN social game engine technology was used in the implementation of this system. In the end, development took a year to complete, turning this into a project of considerable scale.

MOBILE GPUMARK Availability and System Requirements

As mentioned above, the Android version of MOBILE GPUMARK is available from Google Play, whereas the iOS version can be found in the App Store. Both can be downloaded for free, and there is currently no paid version available.

The system requirements for MOBILE GPUMARK specify that Android 2.3.3 (Gingerbread) or later andor iOS 5.1 or later must be used, but they do not define any specific CPU or GPU core, or main memory capacity requirements. Unlike iOS devices, Android devices do not have consolidated hardware specifications; and moreover, driver software updates cannot be performed by users. Mr. Goto says, "It's just chaos (he laughs). We have to test each model individually." In practice, there are even some smartphones or tablets running Android 2.3.3 that will encounter problems when running MOBILE GPUMARK.
　According to Mr. Goto, even the same program code conforming to the standard OpenGL ES 2.0 API works on some devices but not on others. The programming required to make the application compatible with these devices has become the object of superstition among developers.

Because this is a free application, these compatibility problems will have to be tolerated in the meantime. If you have a device that is incompatible with MOBILE GPUMARK, please contribute to its improvement by contacting Silicon Studio through Google Play's "send an e-mail to the developer" at (mobilegpumark@gmail.com), stating your device name and OS version.

Basic Information about MOBILE GPUMARK

MOBILE GPUMARK's Main Menu

The renderer used by MOBILE GPUMARK's graphics engine is basically the smartphone version of the DAIKOKU graphics middleware that gave the application its name. It also uses a smartphone version of the YEBIS 2 post-effects middleware for post-processing allowing for such effects as depth of field representation, glare, bloom, etc.
DIAKOKU and YEBIS 2 have been given virtually the same functionality as the PC and the home console version, rather than paling in comparison.

However, the word "basically" has been used above due to its proprietary specifications, with the main renderer being based on OPEN GL ES, and the sole addition of RIGID GEMS to MOBILE GPUMARK being somewhat special. That said, since RIGID GEMS uses YEBIS 2 for post effects, and DAIKOKU to display the scores and render the GUI. RIGID GEMS cannot be considered the only feature that turns it into a completely different implementation.

　The application exhibits some differences depending on the OS. On the Android and iOS versions, the rendering around the platform- is different, but the main program code is basically the same. In other words, there has been no special optimization applied to either platform.

The maximum frame rate is 60fps (60Hz). A maximum frame rate of 60fps is in accordance with the specifications of Android and iOS.
Its anti-aliasing functionality is based on FXAA (Fast Approximate Anti-Aliasing), provided by YEBIS 2, instead of MSAA (Multi Sampling Anti-Aliasing), which is provided by the GPU. FXAA is a post-effects anti-aliasing method that searches for changes in vertical and horizontal brightness in the rendering results, and applies a gradient to the areas within an object that have been detected as edges. Separate from the company’s mission to attract users to the features of YEBIS 2, this implementation was also a necessary element in the development of MOBILE GPUMARK.

Of the four test modes included with MOBILE GPUMARK (RIGID GEMS, DEAD PARKING, NATURAL BONE, and GPU BENCHMARK), RIGID GEMS, DEAD PARKING, and NATURAL BONE are technical demonstration test systems, whereas GPU BENCHMARK tests the GPU load.
Each test has a demo mode available where you can apply or remove effects, and increase or decrease the number of objects and characters in the demo. This mode also allows you to shift viewpoints.

In the technical demo test systems, the rendering resolution of the demo sequence is set to the display resolution of the device running the demo. So if you are using a smartphone with an LCD panel set to 720 x 1280 pixels, the rendering resolution will be set (in landscape mode) to 1280 x 720 pixels.

From the "Quality" option on the bottom left of the main menu, you can select "High," "Normal," or "Low." These are the quality settings (from YEBIS 2) for depth of field representation, glare, and bloom. In particular, the respective resolution settings of the primary rendering target (the original object of the post-effects) are subject to post processing-effects processing by YEBIS 2 as follows:

• High: set to the same resolution as the display resolution of the device (rendering resolution).
• Normal: set to 0.75 x the display resolution of the device. Post-effects generated by YEBIS 2 are enlarged to the display resolution of the device and then overlaid.
• Low: set to 0.50 x the display resolution of the device. Post-effects generated by YEBIS 2 are enlarged to the display resolution of the device and then overlaid.

This article will go into more detail later, but the official benchmark score ranking site operated and managed by Silicon Studio only aggregates demos set to the highest setting. The Normal and Low settings are available for reference only. Because the fill rate (≒ memory load) is reduced in the Normal and Low settings, they can be used to determine the degree that the frame rate increases when compared to the High setting.

Based on the above, let's move on to explain each test mode.

RIGID GEMS - Beautiful graphics created by unique rendering and physics simulation.

RIGID GEMS is a test mode showing multiple rings rotating on a marble table as multicolored gems fall from above.

The marble table shows reflections generated by a cube environment map of the background scenery. If you look closely, you can see the refracted images across the falling gems.

There are several types of gems, where each gem is made of 70 to 214 polygons, with 30 gems appearing during the benchmark. There are 832 polygons per ring, with 24 rings appearing in the benchmark. There are 180 polygons for the background and 1,682 polygons for the marble table, with approximately 26,000 polygons rendered per frame (with 142 polygons per gem).

The total size of the textures is 6.2Mb, with the largest being the background cube map (512 x 512 texels x 6 sides). Only this scene uses uncompressed textures. The marble patterns on the table are also made with textures.
　The gems and rings are subject to shading, but individual material textures have not been used. The background cube map has been applied only to the rings, much like an environment map. The reflected images seen in the gems are the refracted marble table patterns.

The refraction process uses a method whereby the viewing vector on the surface of the gems is bent once, and then the already rendered textures in the scene are sampled. In the original, full version of RIGID GEMS for DirectX 11, you could use ray marching (a method that calculates how light flowing through a medium disperses and attenuates) to see how the line of sight entering into the gems was reflected and refracted before exiting the gems. However, in the MOBILE GPUMARK version of RIGID GEMS, images are refracted only once off of the gem’s surface.

In the RIGID GEMS demo mode option screen, the post processing effects can be turned on or off through YEBIS 2.

The impressive light overflowing from the high-brightness areas of the scene, as well as the blurred depth of field representation of the background are both created by YEBIS 2. In the demo mode, these effects are available through the GLARE, DOF, and LUMINANCE configuration buttons. Tapping the buttons in succession changes the effects as seen below:

• GLARE Button: changes the amount of "overflowing light"
  STANDARD → CHEAPLENS → CROSS → SPECTRAL → HORIZONTAL → VERTICAL → DISABLE → BLOOM →...
• DOF Button: turns the depth of field representation on or off
  ON → OFF...
• LUMINANCE Button: intensifies the glare effect
  FAINT → MEDIUM → STRONG →...

In addition, the demo mode also has an ADD button that allows you to drop up to 100 additional new gems. It also allows you to drag and move the rings and gems.
　According to Mr. Goto, "RIGID GEMS does not put too much load on the vertex shader. If anything, most of the load is on the pixel shader."

This is not really noticeable because the movements are incredibly natural. The collisional motion of the rings and gems is not an animation that is prepared in advance, but is actually calculated on the CPU by a real-time physics simulation.
The physics simulation feature, where the collision and motion measurement units are converted into small particles, is an original implementation of RIGID GEMS. This functionality is not provided by Silicon Studio's middleware.

The simulation method is very basic, with rigid body physics. The hoop-shaped rings use collision models filled with microspheres as operational units. The gem shapes also use similar collision models filled with microspheres.
Each ring uses approximately 90 microspheres, and 16 to 47 microspheres for each gem. The toppling rings land properly on the marble table, and the falling gems collide precisely without overlapping. Demo modecan be fun as you can cause a domino effect by carefully toppling a ring, and seeing how each one affects the next one, creating a chain reaction as they fall over.

This type of rigid body physics simulation in itself puts a high load on the CPU, which to some extent turns RIGID GEMS into a test for CPU performance.

DEAD PARKING – A large army of zombies lurch forward in this dramatic test scene

A young boy flees into a dark, underground parking garage as a large horde of zombies close in on him. The boy fights back with his pistol but cannot withstand their overwhelming numbers. DEAD PARKING is the most dramatic of the test sequences in MOBILE GPUMARK.

Each of the 24 zombies that appear in this test are composed of approximately 4,000 polygons. According to Mr. Goto, "Although we originally used 2,100 polygons when modeling the zombies, the number of degenerate polygons increased when we converted them to the actual program, resulting in approximately 4,000 polygons."
　The boy, who is the main character in the demo, is made up of approximately 3,300 polygons. The background, depicting an underground parking garage, contains approximately 13,000 polygons. A simple calculation yields a total of approximately 640,000 polygons rendered in this scene.

The dynamic light source is a collimated light source that lights all characters in the scene. The background is filled with pale lighting has been burnt in with a pre-computed lightmap texture.
The fluorescent lighting on the ceiling features a glare, making it seem like an actual light source, but this is actually the effect of YEBIS 2’s post-effects. The high luminance texel painted on the background lightmap is caused by YEBIS 2's glare functionality. YEBIS 2 can be used in this way to enrich the look of an environment.

The shadows of the dynamic characters in the scene, the boy and the zombies, are dynamically generated, but there is no self-shadowing. These shadows are silhouette textures dynamically generated using DAIKOKU’s functionality. Because these shadows are designed only to be rendered on the floor, there is no self-shadowing or projection of shadows on other objects.

The total size of the textures is 57.6Mb, which have been compressed using the texture compression techniques available on each GPU.
Well known texture compression techniques include PVRTC, from the PowerVR family of GPU cores by Imagination Technologies; ATITC, from the Adreno family of GPU cores by Qualcomm; DXTC, from the Ultra Low Power GeForce family of GPU cores by NVIDIA; and ETC (Ericsson Texture Compression) from other GPU cores. MOBILE GPUMARK's archive stores texture packpatterns and they are compressed using the texture compression technique.
　The compression ratio is different for each compression technique. For example, PVRTC can compress approximately 57.6Mb to 1/8 of the original size to approximately 7.2Mb. This is included in the archive.

Textures of the zombie’s clothing.

The animation of the zombies attacking, and the boy as he runs and shoots his gun, are all implemented using previously stored motion data. The 24 zombies use a shared animation loop which is run with a time offset for each individual zombie, giving the impression that all 24 of them are moving separately. Where in actuality, all of the zombie’s movements are the same. There are several variations in the texture of their clothing, which enhances the sense of uniqueness for each zombie.
Additionally, three of the zombies that attack the boy in the middle of the test, transition to a special animation that showcases unique attacking actions.

There are no simulations in this test mode, instead, all movements are generated in advance. The increasing number of zombies close in on the boy and is simply done through animation.
Since the animation control process is done by the CPU, as more and more characters appear in DEAD PARKING, the load on the CPU becomes correspondingly higher. Naturally, the higher the number of characters in the simulation, the higher the load put on the vertex shader.

In the demo mode, you can increase the number of zombies that appear in the scene, up to a maximum of 100. This scene uses a simplified technique for collision among zombies. As the number of zombie increases, they will exhibit unnatural behavior such as overlapping or getting caught on each other. Depending upon the goal for the scene, this could be seen as a charming feature.
　In demo mode, you can turn shadows and YEBIS 2 post-effects on and off. DEAD PARKING has the same options that YEBIS 2 offers (GLARE, DOF, and LUMINANCE) as seen in RIGID GEMS.

You can change the viewpoint by tapping the CAMERA button. With DOF enabled, you can touch anywhere on the screen to set the camera’s focus on that area.

NATURAL BONE – The decisive battle between a young tribal warrior and small UFOs in the jungle

In front of ancient ruins, deep in the jungle, stands an expressionless, young tribal man standing in a fighting pose, ready for battle. Charging towards him are spherical flying objects that seem completely out of place in this scene. Are they part of an advanced security system left to protect the ancient ruins? The young man readies his dagger, leaving the viewer to wonderwhat he will do next.

NATURAL BONE's combination of characters seems too novel for a simple test, leaving us to wonder if there is not a deeper story lurking somewhere in the background.

The young tribal man, the scene's main character, is composed of 12,000 polygons, while the background with the jungle and ancient ruins come to a total of 150,000 polygons.
　As touched upon in the beginning of this article, the original NATURAL BONE is a Windows program based on DirectX 9, and MOBILE GPUMARK began as a project to port this version to mobile devices. Therefore, early in development, Silicon Studio used geometry reduction to cut the number of polygons in the background down to approximately 50,000, which takes into account the GPU performance on mobile devices. When the direction of the project moved towards the development of a benchmark application, the decision was made that a heavy load should be put on the device being tested to achieve proper benchmarking. The number of polygons in NATURAL BONE was then returned to the 150,000 of the original version.

It cannot be said that this version is quite the same as the DirectX 9 version. The animation for the surrounding vegetation swaying in the wind has been omitted from the MOBILE GPUMARK version.

The total texture size is compressed with PVRTC to approximately 8.2 MB (actual size before compression is 65.6 MB, or 8-times).
The light source in the scene is only single parallel light from the sun. Dynamic lighting is only used for the native man. As for the background, pre-computed lightmaps are applied, so there is no need for additional lighting.

Shadow generation is also only used for the native man at runtime. The shadow generation technique applied is the same as that for DEAD PARKING, where it uses a silhouette texture that is projected only onto the ground. The shadows made by plants in the background are pre-burned lightmaps.

The surrounding background is reflected onto spheres

Bump mapping using a normal map is applied to the muscles and hair of the native man, and lighting from both diffuse reflection and specular reflection occurs on the skin. A pre-made texture was also used to make the surroundings appear on the spheres that fly around.

Although NATURAL BONE does not require such a heavy CPU load, it is the test mode with the heaviest load due to the high polygon count and also the sophisticated post-effects. "Actually, the heavy pixel shader load is not only due to the YEBIS 2’s sophisticated post-effects, but is also partly due to the rendering of plant leaves in the background," said Mr. Goto.

The reason that this occurs is that on the rectangular polygons that represent the leaves, a "leaf-shaped texture with transparent parts" is applied. An alpha test is performed during rendering, and transparent parts must be removed prior to this. Given that the GPU cores for mobile devices do not have a wide memory bus bandwidth, the alpha test is not that effective. This scene, where the alpha test is performed many times, ends up putting a heavy load on GPU cores for mobile devices.

In that sense, if you get a high score using NATURAL BONE, it is probably safe to say that the smartphone or tablet in question has solid 3D graphics performance.

The control and setting options in the demo mode are the same for RIGID GEMS and DEAD PARKING.

The Secret to GPU BENCHMARK: "Your test is already done even before you know it"

At the beginning of this article, I first introduced the GPU BENCHMARK as not just a tech-demo test. Ten short tests run in this option, and it includes the 3D movement of geometric patterns, a dragon flapping its wings, and the waving and brandishing of a dagger by the native man in NATURAL BONE.

This test is equivalent to the Feature Test previously used in Futuremark's 3DMark series, so it can be said to be a "standard” benchmark method.

What you need to know is that while the tech-demo test mode explained above was run at the display resolution of an operating environment, only the GPU BENCHMARK is run at a locked rendering resolution of 850x450 pixels (aspect ratio of 16:9). Rather than checking the smartphone or tablet performance, it means that it is checking the performance of the installed GPU core units, so it is intentionally designed to do so. For this reason, the GPU BENCHMARK is not affected by the "Quality" setting that can be selected from the main menu.

Once executed, a fairly short test video appears, but actually the test itself is running when "please wait" is displayed before the video appears.

In GPU BENCHMARK, rendering to an offscreen buffer (a frame buffer that is not displayed) can be done without "the need for 16.67-ms intervals, since it displays 60 frames per second," and the test takes place there. The video that appears after the "please wait" message is essentially a dummy.

"During video playback, the score at the bottom of the screen is rising, but it is also designed to show the rising count of the score obtained at the time of the "please wait" message," Mr. Goto said with a chuckle. The fps figure displayed during video playback is just the frame rate for the video -- although real-time rendering, which is on par with the test, is taking place -- so it has no direct connection to the score.
　Personally, I felt that because in some instances you need to run on many different types of devices, it would be great if you could also select "calculate without video playback" which would likely reduce the calculation time.

So, let's take a look below at the test categories in order.

GPU BENCHMARK (1) Fillrate test

What runs first is the Fill Test (fill-rate test), which renders the numerous polygons applied with a translucent texture. As mentioned earlier, an alpha test with translucent rendering becomes a very heavy test case for GPU cores for embedded devices with a narrow memory bus bandwidth.

Since this is a fill-rate test, the measurements appear in pixels per second (MPixels/s), but the number that is displayed at the end is corrected to a "score-like number," said Mr. Gotoand it becomes part of the GPU BENCHMARK score. While the raw fill-rate number is displayed on the lower-left portion of the screen during video playback, it will not appear in the final score, so take note of it if needed.

GPU BENCHMARK (2) High polygon rendering test

Next, we have the High Polygon Model test. In this test, after rendering the dragon, which is modeled using 411,021 polygons, a single parallel light is aimed at the dragon as dynamic lighting. Lighting from diffuse reflection and specular reflection is measured in pixels.
The total texture capacity without compression is approximately 54 MB. Textures include diffuse reflection (decal) and diffuse map, normal map, and environment map with no background in place.

In addition to geometry load, the test is one with a heavier pixel shader load due to pixel-based diffuse and specular lighting calculations.

GPU BENCHMARK (3) Peak load test with zombies and dreams galore

‘Many Models’ is a peak load test. It is a test in which 100 zombies that emerged in DEAD PARKING (approximately 4,000 polygons per zombie) enter the scene.
Calculated simply, this means about 400,000 polygons are rendered. On top of that, , because the animation of 100 zombies is CPU-controlled, the test raises the CPU load percentage..

With approximately 400,000 polygons, the polygon count itself is nearly the same as that of the High Polygon Model test, but because there are 100 zombies as opposed to one dragon, the number of rendering calls is overwhelmingly greater in this test.

The dynamic light source is also only a single parallel light source in ‘Many Models.’ Lighting occurs only by per-vertex diffuse reflection(Specular lighting is omitted). Therefore, even though this is a load test of approximately 400,000 polygons like the High Polygon Model test, the pixel shader load is much lighter in the ‘Many Models’ test. The texture is just decal texture, so the uncompressed size is about 1.3 MB.

Geometry instancing (also known as instanced rendering in OpenGL) is a technique for rendering multiple copies of one common model while changing parameters. The MOBILE GPUMARK is OpenGL ES 2.0 based, so this function is not used.

GPU BENCHMARK (4) The actual handsome native man "stripped" from NATURAL BONE

The footage of "the alluring native man, who appeared in NATURAL BONE, brandishing a dagger in a scene with no background" is in four tests: Per Vertex Lighting, Per Vertex Lighting & Specular, Per Pixel Lighting & Specular, and Per Pixel Lighting & Specular & Normalmap.

Since the native man is taken from NATURAL BONE and used here, the polygon number remains unchanged at 12,000. The uncompressed total texture capacity is approximately 31 MB. Texture types include decal texture, normal map, and specular map (gloss map).

The dynamic light source is just one parallel light source. It means that reflected specular lighting and reflected diffuse lighting occur, but as rendering precision (≒ quality) is measured in four patterns, the purpose of this set of tests is to measure the performance in each respective pattern.

Per Vertex Lighting only calculates reflected diffuse lighting per vertex, and pixel rendering is a test case which is performed using colors obtained by linear interpolation based on the results.

Per Vertex Lighting & Specular calculates both reflected diffuse lighting and reflected specular lighting per vertex. Pixel rendering is performed using colors obtained by linear interpolation based on the results.

Per Pixel Lighting & Specular calculates the lighting from both diffuse reflection and specular reflection by using a pixel shader. Even here the normal vector per pixel is given as "the linearly interpolated normal vector at the peak level," but lighting calculations are individually performed per pixel. As such, computational effort in the pixel shader greatly increases.

Per Pixel Lighting & Specular & Normalmap is also performed using a pixel shader to calculate the lighting from both diffuse reflection and specular reflection. However, the normal vector which is used to calculate the lighting per pixel is not only the linearly interpolated normal vector at the peak level, but it is obtained after considering the content of the normal map texture. Since computational effort as well as the texture reading increase, the load is further increased.

GPU BENCHMARK (5) Showdown between the handsome native man and the mysterious sphere

The test uses a scene that seems to be taken as is from NATURAL BONE, and looks at the performance when YEBIS 2’s post-effect processing quality is changed. The test category is YEBIS Resolution.
Since the character that appears is taken from NATURAL BONE, the polygon count of the native man is approximately 12,000. The dynamic light source is only a single parallel light source. For the lighting model, calculation of the reflected diffuse lighting per pixel and the reflected specular lighting per pixel is applied.
The total texture capacity is about 31 MB + 1 MB. The background of ruins and jungle is not 3D object but a kakiwari (background) texture, and is the "+ 1 MB" in referenced above and is the part increased from GPU BENCHMARK (4).

According to Mr. Goto, the target for measurement of this test is only the post-effect portion, so the rendering load is not the test target. In other words, the test is more for testing the pixel shader load and memory bus bandwidth.

In this test, there is no change in lighting or shading, and the measurement of three patterns are performed by switching the resolutions of the primary render target. The the target for YEBIS 2 post-effect processing is 420×234 pixels, 800×450 pixels, and 1280×720 pixels respectively. The name of this test category is derived from this.

Earlier it was mentioned that the Quality setting in the main menu is the setting of the resolution that is made the primary render target with YEBIS 2.and the GPU BENCHMARK is not affected by the Quality setting. There is no concern for competing settings or settings being overwritten.
　In addition, because GPU BENCHMARK is fixed at a rendering resolution of 800×450 pixels, you might think that if YEBIS Resolution is 1280×720, then the resolution that is made to the primary render target with YEBIS 2 is probably higher than the resolution of the final frame. Mr. Goto says, "That's right." When YEBIS Resolution is 1280×720 pixels, after the processed post-effect results are reduced to 800×450, they are composed onto the final frame. This means that the post-effect is processed with slightly better-than-intended specifications.

Score mechanism and ranking system

Display of MOBILE GPUMARK results

Scores in MOBILE GPUMARK are measured using two different calculation methods.

One method is calculating the scores based on frame rate, and this is done for RIGID GEMS, DEAD PARKING, and NATURAL BONE. 　Since it is not "accumulative addition of the average frame rate per second," and the total frame count and playback time for each demo are set, it is a scoring system that "adds points when rendering is completed and results can be displayed." In other words, a perfect score is equivalent to "the total frame count of a scene," but when calculating the actual score, appropriate coefficients are given to each demo to make it "look like a score number," saidMr. Goto.

Ranking site screen

The other method is for GPU BENCHMARK, and this is a time-based measurement. Since GPU BENCHMARK measures under equal conditions, independent of the device type, it simply provides a high score if you quickly finish rendering. However, if nothing extra is done to the calculation, then the lower the number, the better the score becomes. When calculating total scores, it also calculates score numbers by using appropriate complements given the lack of consistency with the demo test's cumulative scoring.

The total score is the combined number of the demo test score and GPU BENCHMARK score. This is displayed together with each test score on the RESULTS screen. Despite this being benchmark software, the technique for score calculation is not revealed. There are many pieces of general benchmark software that disclose the score calculation technique, so we hope to eventually reveal it to the public.

You can upload the total score from the RESULTS screen to the score summary server administered by Silicon Studio. At the server, the "device ranking" is updated based on the scores that have been sent. Scores for each device are transmitted from all around the world, so to ensure fairness, the server averages the score numbers that are sent per device, and this is reflected in the ranking.

The ranking is disclosed on the ranking display page that is also administered by Silicon Studio. This page can also be accessed from the RESULTS screen.

As mentioned earlier, scores are transmitted to the score summary server only when the QUALITY setting is on "High". The ranking is also calculated only using the results on the “High” setting.

Moreover, when sending the score numbers from the device, the device's hardware profile information is also transmitted. The part corresponding to user information is not sent; only the device specs are transmitted. According to Mr. Goto, the CPU and GPU model numbers, panel resolution, amount of memory installed, and the expanded API information of utilizable OpenGL are transmitted.
　People who worry about what kind of information is actually sent out can check the DEVICE PROFILE after starting MOBILE GPUMARK. Only the information contained here will be sent along with the benchmark test scores.

Moreover, when sending the score numbers from the device, the device's hardware profile information is also transmitted. The part corresponding to user information is not sent; only the device specs are transmitted. According to Mr. Goto, the CPU and GPU model numbers, panel resolution, amount of memory installed, and the expanded API information of utilizable OpenGL are transmitted.
　People who worry about what kind of information is actually sent out can check the DEVICE PROFILE after starting MOBILE GPUMARK. Only the information contained here will be sent along with the benchmark test scores.

Also, according to Mr. Goto, devices that were released in 2012 do not greatly exceed 30 fps using the demo test when the Quality is set to High. This means that for a benchmark application for smartphones and tablets with a maximum of 60 fps, scores do not immediately become saturated.

In the case of the current MOBILE GPUMARK load, Mr Goto predicted, "I think it will be at least another year or two optimistically, before the score approaches the 60 fps plateau."

Since the ranking site has the score ranking for each device, it means that you can only get information about "which devices are fast." If you sort results by scores from GPU BENCHMARK which measures performance under equal conditions, you can find out which GPU cores are fast.
　That's right: after sorting, you can find detailed information by clicking the device names in the displayed results. According to this, you can say that as of February 2013, PowerVR Series5 SGX540, Adreno 320, and Tegra 3 are the top three.

Conclusion

MOBILE GPUMARK is certainly useful for knowing how your own smartphone or tablet's performance compares with your friends' as well as the latest devices found in catalogues. (It is currently necessary, however, to include the proviso "if it is compatible with that model.")

Also, the graphics in MOBILE GPUMARK are of a considerably richer quality than the average game graphics in current smartphones and tablets. As such, it is also probably useful for confirming that "this type of graphic moves at this kind of frame rate."

If they move at a decent frame rate, you can even boast to your friends, "Hey, these graphics are real-time rendering on my smartphone."

In COMING SOON, there are plans to add new features

According to Mr. Goto, in the future they are working on adding a new ranking feature such as "What is your device's position in the world," and a new function that captures specific frame(s). On top of improving compatibility with Android devices, Silicon Studio has the will to continue and develop this software and “is not finished after this release."

Silicon Studio plans to show off MOBILE GPUMARK at the Game Developers Conference 2013 (GDC 2013) which will be held in March 2013. I look forward to Japan's MOBILE GPUMARK becoming the standard 3D benchmark in the smartphone and tablet industries.

Official website of Silicon Studio

©Copyright. Silicon Studio all right reserved.