• 1.0 Benchmarks and 3D bottlenecks
• 1.1 Benchmarking an Illusion...Accuracy=Realism=Quality
• 2.0 3D Background and Terminology
• 3.0 The need for 3D Standards
• 3.1 Lack of Standards and Native Ports
• 3.2 Direct3D...an ill-birth 3D interface
• 3.2.1 Compatability Bits....the bane of Direct3D
• 3.3 OpenGL...a better 3D interface
• 4.0 Fourth Generation 3D
• 4.1 Trilinear Filtering and the Rage 128
• 5.0 DX8 mini-FAQ
• 5.1 DX6...DX8  History Lessons

1.0 3D Bottlenecks and Benchmarks

The task of creating an illusion of a 3-D world on a computer monitor is a very compute intensive task that inevetably involves either the CPU bogging down and/or the Video Graphics Chipset bogging down. Bogging down needs to be quantified and benchmarks are an invention to help gauge the amount of or lack of performance of advance graphics cards designed to remove this bogging down process. For computers, the benhcmarking task was typically a well defined task like calculating a number but how do you benchmark the task of creating an illusion ? Graphics cards use different algorithms to arrive at a screen image....the quality and time/speed taken must both be consider together, equally.

I was browsing for 3D benchmarks and came across Michael's 3D Benchmark Links . He stated that 3DBench was not a very good benchmark becuase of its dependence on the rating of the CPU used. To the extent that we would all like to be able to judge the merits of one subsystem...say CPU...independent of another....say Graphics card...I agree. One of the things that Steve Jobs hyped on when he was building his NeXT computer was that the performance is always limited (and thereby) sensitive to the slowest subsystem. You could say that sensitivity implies the existence of a bottleneck. The two most influential bottlenecks in our quest for creating an illusion are the CPU/motherboard and the Graphics card.

1.1 Benchmarking an Illusion

3D hardware gives you better performance in terms of the traditional computerspeak of better response and less time lag while addressing a issue which most of you can judge on your own but have not had to in the area of computers: visual realism and accuracy. Any hardware such as graphics cards or board architectures(such as AGP...combined software/operating system/board solution) that aid in this task of increase realism are on the right track.. PC World wrote one of the few good reviews recently on 3D but inadvertently described the advantages of 3D for computer gaming by saying:

So no change in speed--just a change in scene quality....
In fact, AGP promises to rid the world of ugly, oversimplified 3D scenes in PC
software. Running the same game--one AGP enhanced, the other not--we saw some
impressive improvements in visual detail. In hands-on tests using an AGP-enhanced
version of G-Police, we noticed that explosions lost the distracting blockiness around
the edges that we saw in the PCI version, and static billboards turned into flashy signs
with full-motion video.

2.0 3D Background and Terminology

With this as a backdrop, I like to give you some links which cover and allow you to explore the above ground on your own.

• 3D glossary-Wayback...fogging, textures, bilinear filtering in a nutshell...

1. object position
2. relative object position
3. transparency
4. surface properties
5. environment

3.0 The need for 3D Standards

Microsoft Windows performance used to be hampered by the CPU because it had a lot of information and work to do. With the advent of the "accelerated" chip, the CPU no longer became the bottlenck because its task was simplified to only issuing and implementing higher level and less data intensive calls. Prior to Windows, each and every DOS application had to be patched to take advantage of the hardware features. Software development was a little less hectic then and companies such as ATI hired people to do these sorts of things called native ports. With games, this is not so easy and manufacturers do these things co-operatively with game developers. The times they are a changin...

As the market matured and applications started writing to standard application interfaces (ie. Windows), Graphics manufacturers only had to map the Windows interface onto their hardware...the software stayed the same (regardless of the hardware) and a level playing field was established. Video graphics cards distinguised themselves through a combination of "marrying driver expertise to hardware features". Winstone and Winbench tests this marriage. One piece of hardware may actually be superior to another but be hampered by poor drivers...this begs the question "Should a benchmark be "truly" driver independent ?" ...It all depends on your audience. If what you want are practical, low cost, "green earth", solutions then it is to everbody's advantage to have a level playing field that starts with hardware and low level drivers and demands the existence of defacto high level application 3D interfaces. Else all this 3D activity is targeted towards niche techno-geeks like myself who post benchmark results for native ports of applications such as VQuake for other techno-geeks to have some semblance of community. AcK!

3.1 Lack of standards and Native Ports

Popular games such as MechWarrior and Quake are ported to specific video cards until the playing field is made even by standard interfaces such as Direct3D and OpenGL. At present, the defacto DOS interface for games is DOS as gamers are a different breed from the business types that embraced Microsoft Windows. Gamers are sophisticated and opt for immersion over neat icons.

It is unfair to compare a special native port of a game to one piece of hardware to gauge "intrinsic" hardware performance....however it does allow the industry to compete in more ways for the attention of gamers and those who succeed at it should be applauded for upping the software ante. Believe me, if the market wants the fragmentation created by "special ports", other hardware vendors will come on side (as they have with MechWarrior) and the beginnings of new emerging non-defacto standards begin to emerge. That is the standard will be to have standard ports. Confused ?

The defacto standard practice in the 3D industry amongst the big guns is to port MechWarrior over to specific 3D hardware. Maybe we should use MechWarrior (circa 1996) as a benchmark :) I know of some well respected Hardware reviewers/webmasters arguing for native ports of Quake/GLQuake (circa 1997) and in 1998 someone big is urging us to use Turok as a Benchmark....whatever happened to MS MonterTruchMadness. This is a reflection of the continued growth of games performance being tied to the identity of a PC that has led to 3D to garner the "heart and mind" of the graphics card industry. So if the person coding the port had a "bad" day while doing the "Creative" port for the nth time....guess who does and who doesn't benefit... Just don't expect to see any useful benchmarks until API are figured out.

3.2 Direct3D...an ill birthed Standard Interface

3D standard interfaces are designed minimize the amount of software required to support a the 3D features of a given piece of 3D hardware. As in any design, bad decisions in the design are soon flushed out as these designs are flushed out. Direct3D was a badly documented complicated design whose flaws in portability were masked. The concept of hardware shortcomings being adaptively addressed by software gave control to the game developer but also the also the responsibility of providing this capability. In Microsoft's haste to have hardware vendors on their side, the lack of a baseline 3D hardware platform resulted in the task of developing an application, such as a game, was a moving target. Software could always, given enough time, be tested on enough platforms to ensure that a game ran on most hardware.

3.2.1 Compatability Bits

Things are not entirely "rosy" in Direct3D Land in terms of application developers being able to write their application once due to Microsoft's ill conceived concept of compatabiity bits as pointed out in Game Developer March 97 as quoted below: 

• The primary problem we encountered while developing D3DBench is Direct3D's lack of a reference implementation or specification. Direct3D endorses the concept of capability bits, or the ability for a particular driver to tell an application exactly what 3D acceleration capabilities it supports. It is the application's responsibility to compensate for missing capabilities , a burdensome and error-prone requirement to say the least. The very nature of capability bits means that an application can be bug free when written for a specific piece of hardware, yet breaks down the moment a different piece of hardware is inserted

• One of the most important aspects of testing a 3-D accelerator is accurately determining which 3-D features the accelerator provides. Just because an accelerator promises to support fog, for example, it does not necessarily mean that it renders fog well. The 3D WinBench 97 attempts to determine which rendering features an accelerator and its driver support in the same way games do--by querying Direct3D. From this query, 3D WinBench 97 gets information on which 3-D features the accelerator claims to provide in hardware, via the HAL (Hardware Abstraction Layer), and which require the software HEL (Hardware Emulation Layer). This knowledge is vital and has huge performance implications, because the HEL, by virtue of being an entirely software-based system, renders 3-D features a great deal more slowly than the hardware can.

 

 
 
 
 
 

The way an application has to deal with this rendering information is both tricky and profound in its performance implications. Because of the design of Direct3D, if an accelerator does not support hardware acceleration for all the 3-D rendering options a scene employs, 3D WinBench 97--or any application--must display the entire scene using Direct3D's HEL. No application can choose to implement some of a scene's rendering options with the HAL and others with the HEL; it's all or nothing.

The story gets worse if an application wants to use MIP mapping. Because Direct3D's HEL cannot display MIP-mapped images, if the application desires MIP mapping and an accelerator (or its device driver) does not support that feature, neither 3D WinBench nor any application can display that scene with that accelerator.

There's more. As we've learned during our benchmark development and testing, some Direct3D drivers available today may "claim" to provide a 3-D rendering feature, but they either don't provide it correctly or don't do it at all. For example, we've seen some drivers that claimed they could provide specular highlighting, but did not do so on our tests, where a scene involving that feature appeared no different with or without specular highlighting enabled. In all cases, implementing these scenes using the HEL instead of the HAL resulted in the correct behavior.

3.3 OpenGL...a better 3D interface

John Carmack has stirred the pot and should make things interesting for 3D by providing his support of a "real visualization " vs "gaming texturing" API in the form of OpenGL. The very thing that made the likes of Terminator2 possible.

OpenGLhas been the workhorse standard in the Workstation world for years. It is robust and and yes...it runs on Linux (more recent link here). John Carmack's (of DOOM fame) offering of a OpenGL version of Quake and his view of a 3D programming has caused a clarification about an API's ease of use versus speed within the industry with regards to APIs. Card vendors I think will need to support both Direct3D and OpenGL. Please see the PeAK OpenGL pages located here

Currently, most chip manufacturers, including ATI, keep the 3D aspects of their chips under close wraps for proprietary reasons. It is not until open and defacto standards emerge for application interfaces exists, such as Direct3D or OpenGL, that a level playing field will exists for chip and and card vendors to distinguish their products to the guy on the street via marrying drivers for these common interfaces to hardware.

4.0 Fourth Generation 3D

Multimedda/Convergence of DVD, HDTV, TV-out, TV-in, and Broadcast TV support are important issues to consider for a video graphics card but I will stick to 3D for now. The quest for more realism/immersion in 3D graphics and visuals without incurring a significant response lag has demanded the following features:

1. Higher resolutions:24 or 32 bit Z=buffer, larger local memories(16-32 MB), double/triple buffering
2. Better/More filtering: trilinear filtering
3. Better throughput: pixel caches, 1.6GByte/second memory bandwidth, 128 wide internal buses, 128 wide external buses.
4. Workstation class OpenGL Graphics: Mature OpenGL support for workstation class applications.

4.1 Trilinear Filtering on the Rage 128
A A good synopsis of all these 3D terms can be fournd 3Dimensional 128 site. There have been many threads going around about "trinlinear filtering" and the Rage 128 and the throughput penalty for enabling it and "kluges" by other companies to get enhanced filtering using a technique LOD Dithering to approximate this filtering technique...the problem is that this technique is passed off as "trlinear filtering". Here are the two best responses that I have seen. The are by the "G" brothers Geoffrey and Dan who can be found in many of the 3D forums cutting to the chase on 3D issues.

Date: Thu Dec 17'98 - 1:02pm
     Author: Geoffrey G. (gbgitch@southwind.net)
     Subject: Nope.

        in response to Re: Urban Legend?, posted by Dave Steele on Thu  
        Dec 17'98 - 7:21am

     Nope.

They both have exactly the same hit. The issue is that Zanshin does not realize the Riva TNT is NOT TRILINEAR FITLERING. Trilinear is free as long as multitexture is not used (since they both use similar processes) since Quake uses multitexture you cannot use trilinear with it without a hit. The TNT's older drivers had a 30% hit in performance (about equal to the Rage 128's hit) the new (TNT) drivers DO NOT TRILINEAR FILTER, they just say they do. In fact the newer drivers do a thing called LOD Dithering, it has a similar effect to trilinear filtering only instead of blending the two mipmaps it stipples them. This is the trick used by the Banshee and the i740 to fake trilinear. It breakes up the 'line' artifacts from mipmapping, but it causes pixel crawl.

Date: Thu Dec 17'98 - 3:21pm
     Author: Dan G. (dangitch@southwind.net)
     Subject: Re: Nope.

     , in response to Nope., posted by Geoffrey G. on Thu Dec 17'98 - 1:02pm

     Re: Nope.

I think that theoretically trilinear filtering should never be completely free on either the TNT or Rage 128. Unlike, say, the Voodoo2 or Permedia3, their extra texture units should never be sitting idle. If they're rendering something simple, like single textured, bilinear filtered, un-bumpmaped pixels, then the second unit should be busy helping get the job finished in half the time. This is why a 90MHz TNT has a theoretical 180 Mpixel/sec single textured bilinear fillrate, and 90 Mpixel/sec single textured trilinear fillrate (or bilinear filtered, dual textured, etc). As such, switching to trilinear filtering should result in a 50% performance hit. In reality, the TNT doesn't get anywhere near its theoretical peak of 180 Mpixel/sec fillrate, so the drop-off isn't that sharp, but there should still be some slow down. To add to the confusion, however, the TNT's current drivers seem to avoid dual pixel rendering in DX6 titles, meaning it actually is effectively free there right now, until the drivers are updated. It's no wonder the "free trilinear on TNT" theory is so hard to disprove!

5.0 DX8 mini-FAQ

When DX8 was finally released, long discussion threads, indicated dissapointment in lower than DX7 benchmarks in the performance of some games. Although, there is some relationship between a driver and DX8, users seem to more frames/performance with DX8. A situation somewhat like expecting Win2K to support more hardware and more compatability. Right?

This situation with DX8 is different but at the same time reminiscent of the infancy of 3D when "Mech Warrior" was the first RPG to take advantage of 3D features in hardware. The result was a vast increase in image quality accompanied by a slightly "lower" framerates (than the software only version). This was due to the new 3D features exposed in the hardware engine. No DirectX existed at that time. The different layers of software that you see today were done by either the game developer/engine developer or the chip vendor.

Fast-forward to today. In order to see a feature in hardware, the chipset vendor must work with three different groups consisting of the game application/benchmark, graphics chip driver and Microsoft DX8 need to get their collective acts together. The plus and minuses of joint ownership of the various levels of abstraction..
 

1. Hardware Support by graphic chip vendor: lowest level owned by chip vendor (ATI)
2. driver Support by graphic chip vendor: 2nd lowest level owned by chip vendor (ATI)
3. Exposed by MS DirectX or OpenGL: 3rd lowest level owned by MS (MS usually)
4. Exposed by Game application such as Quake: Highest level owned by game developer (Game vendor usually))


In the case of the Radeon Ark Demo, the highest level can be controlled by ATI. The next lowest level can be extended by ATI (in the case of OpenGL) but I'm not to sure in case of DirectX. Perhaps writing right to the metal (bypassing DirectX) may be possible.

Below is a summary of DirectX8 in FAQ form. For the most definitive word on DirectX8, got to most comprehensive Windows95/98/ME/2000/NT/DOS site in the world for his view on DirectX8.

Two extremely good articles on DirectX8 can be found at Sharky and at Reactor Critical

1) DX8 Expectations/Myths:

• DX8 could mean more stability or speed
• DX8 could mean faster speeds over DX7
• DX8 could mean less bugs than DX7
2)DX8 Truths:
• DX8 has more features
• DX8 will expose more of Radeons intrinsic advanced graphical features
• DX8 will offer more realism and smoothness to future games by exploiting new hardware features.
• DX8 supports and exposes vertex blending, vertex skinning, and keyframe interpolation
• DX8 seems to be tuned for stability at the expense of speed based upon comparisons with DX7
• New benchmarks are needed to gauge "both" exposed hardware and exposing DX8 layer
• Some bugs attributed to drivers and hardware are fixed under DX8
5)DX8 in simple language
• Less "woody" and smoother movement with less framerate variation using keyframe interpolation
• Better belly buttons using vertex skinning
• More realistic jaw movements using vertex blending
• In more complex language
4)DX8 Remaining questions:
• How was Radeon Ark Demo implemented with respect to the 4 levels of abstraction.
• Will or does Radeon Ark Demo need to be re-written to take advantage of DX8 ?
• Was DX8 tuned for stability to overcome DX7 stability problems ?
• Should DX8 be accompanied by an updated benchmark to evaluate graphics chips supporting the "new" features of vertex blending, keyframe interpolation, and vertex skinning.
5)3D Benchmarks:
• Need deep "maze-like" worlds to benchmark overdraw/hidden surface management aspects of rendering
• Axiom: 3D games programmers write wth the advantages of hardware "costantly" in mind.
• Axiom: Programmers will workaround the bottlenecks/disadvantages of hardware.
• Do game developers wish to create James's bond types of chase scene envionmnets consisting of complex layers walls, bricks and mortar called Villages ?

5.1 DirectX History Lesson

There used to be and still is a tight binding force in the 3D community known as Winbench
Through a freak sequence of events, it came to be that hardware reviews were reduced to time demo execution runs of the latest hot game and the lustre of WinBench tarnished.

My prediction is that WinBench will make a return as the "benchmark" to gauge 3D hardware features fairly in terms of performance and quality.  They have  earned the respect of Vendors due to an experienced core group of graphics savvy programmers who are the "defacto self appointed guardian/consumer watchdogs"  of 3D maturation. i
One of the few entities capable of creating metrics to help gauge the validity of 3D and separate out the hype in the  hardware/driver/application pipeline.  Another unique aspect of them is the capability to debug/uncover/find "tricks" that are employed by drivers to workaround (sometimes illegally) deficiencies in the hardware. Unlike most test sites that stand on the coat tails of  frame rate counters (found in nearly all games today), ZDNet is capable of distilling and writing sophisticated code to uncover dissected focus views on hardware.  These   synthetic benchmarks dissect the goods on both  hardware features and the ability of the driver to expose it.  WinBench has concentrated on bug fixes for 2001.

•  Benchmarking  questionable implementations

• DXn...is a MS specification, drivers are a Hardware Vendors sample implementation of said specification, and applications are pudding using those ingredients
• Winbench ...the first application for DXn and a snapshot of what it does

"Bench Capades"...cheats,

1. You mean my name itself makes a difference in my report card
2. Approximating quality and performance...
• skipped frames: I'll deliver every other newspaper and be done in 1/2 the time
• queued frames: 80 yard times in a 100 yard dash
• mip mapping it ain't: drawing a road fading into the horizon is a useful tool  for the perspectively impaired
• blending it ain't: My shiny new egg beater is worse than my spatula
• Texture compression it ain't:...but I get a 30% increase in framerate...can I have my 32 bit "cake" and eat it too?
3. Thesis: games are a better indicator of readworld performance than synthetic benchmarks
4.  Blame Canada!!!
1. ATI 2 consecutive Vsync flipping driver (Rage Pro Turbo) serves as Tom's fodder for the demise of Winbench and Tom's hardware claim to fame. It was Monster Truck Madness, Turok, and now Mercedes Benz Trucking...WinBench has its place in this world but when you are far ahead of the pack, you often get overlooked.
That is it for the history lesson. Better benchmarks can be arrived at by bypassing either bypassing "some" of the rendering or a good chunk of it (stuttering, anyone?). Today, many have commented on lower benchmarks with the Radeon but better game play with smoother framerates. I hope some of the cheats in synthetic benchmarks and real world games give you an understanding of why this can be so.

Email: peakrchau[at]yahoo.ca