Realtime Graphics and the Importance of Fish

By Daniel Jackson on Siggraph 95, 16 October 2008

Fish are leading the way in cutting edge algorithms for realtime graphics. Application developers should take heed of these revolutionary new trends.

This year's Siggraph in the Los Angeles convention center impressed upon me some of the aspects of the relationship that industry and academia have with each other. The conference is an important event with regard to this as it presents a good overview of that which is going on in these different areas of cutting-edge computer graphics. Siggraph succeeds in doing this by hosting one of the most important trade shows in this rapidly developing and exciting industry; companies will often make their most important announcements here. Silicon Graphics presented project Maya (their new software which will combine both Alias and Wavefront which they recently purchased), and SoftImage, purchased by Microsoft, announced that they are porting their software to the Windows NT platform). In conjunction with this are exhibits by artists, games developers, community groups and academic institutions of their latest applications of computer technology. The raison d'etre of Siggraph and traditionally the focus of the conference is the presentation of papers, panel discussions and workshop sessions in the latest techniques and discoveries of the field. In these sessions the leading lights from the games industry, the film and special effects industry, academia and so forth, presented papers in the technologies, procedures and new discoveries used in the development of their specialities.

What made the event memorable for me was the combination of a technical demonstration by Silicon Graphics and a paper delivered by Demetri Terzopoulos from Toronto University. I believe taken individually they both had their weaknesses but that considered together the significance for realtime graphics is mind blowing.

The Importance of Fish - Daniel Jackson

Silicon Graphics, known for their technical brilliance in creating some of the world's most powerful and expensive graphics computers, presented a mind blowing demonstration of the latest incarnation of their triple pipeline Onyx. The Onyx is designed specifically for realtime 3D graphics and was fully loaded with lots of dedicated hardware. The most impressive of their demonstrations was the realtime museum. It impressed through the high resolution of the displays; the quality of the texture maps; the realtime reflection maps in the marble floor; the realtime spotlights; the high polygon count of the models whereby when you flew up very close to an edge of a cylindrical column intersecting the ceiling you could not see any starring lines; the paging of video from disk onto TV screens in the museum; the quality of the central exhibit in its reflection of the entire space; the refresh rate of 60Hz which means that the visual perception of lateral movement is hyper-real (film uses only 25 frames per second and fast panning movements are unpleasant). The reason why I found this so exciting was that for the first time one is beginning to see realtime actually exceeding the quality of pre-rendered animation. The high resolution output and high frame rates mean that because of the legacy of broadcast and film mediums realtime is starting to and will increasingly offer something which is a new experience and undoubtedly create a completely new medium for entertainment and for artistic expression.

One criticism that could be levelled at Silicon Graphics is that there isn't really much going on in these worlds. Maybe Demetri Terzopoulos could help them out there. He delivered a paper and presented a spoof Jacques Cousteau video of his Artificial Fish. This video was also shown at the Electronic Theatre (the presentation of the best computer graphics over the last year, including sneak previews of forthcoming movies such as Pixar's outstanding "Toy Story"), but was not appreciated out of context. His fish were truly intelligent, in as much as fish can be: there was no scripted animation. Their movement was determined by a rule based system which involved the fish learning to swim and which other fish were predators or prey. Their swimming action was determined by their muscle structures and experience. The system involves other aspects such as mating behaviour, a physics based environment, in this case a predator shark, schools of fish and stereoscopic retinal imaging. I will not attempt to go into the details of this system but it suffices to say that the fish "ecosystem" were in no way predetermined apart from the rules of the system and the physical characteristics of the environments. 

Okay you may say that fish are pretty easy; they have one of the shortest memory spans, their behaviour can be modelled by relatively simple rules and they don't seem to express any. What we really need are intelligent human characters in our realtime environments and these techniques are suggesting ways in which we can achieve this. In terms of realtime we are in the early days still, but if we can marry the technology of the likes of Silicon Graphics and the algorithms developed by academics and programmers in industry we are going to witness the birth of an incredible new medium over the next 10 years or so. It is already happening in the games industry with the recent launch of the Sony PlayStation and a catalogue of new 3D games, but as always we need more computing power and equally importantly we need the techniques to develop these applications. These techniques must involve character intelligence and expression which due to the nature of realtime cannot be predetermined if we are to create convincing and engaging experiences. As games magazine Edge says "The future is almost here" but we still need to do some work on our character animats if we are to realise the potential of machines such as the Onyx. 

For further information on the fish, visit Silicon Graphics at: