The Making of Koi 3D
Koi 3D was creating using advanced techniques at the cutting edge of both art and science.
Each koi fish in Koi 3D is actually a digital model comprised of a set of points in 3D space that map the contours of the surface of the fish.
This digital model was carefully animated, frame by frame, to create numerous motion sequences that the fish could then choose to follow when swimming (for example, turning left, accelerating, swishing the tail).
Each fish in Koi 3D was carefully painted using 3D painting software that allowed the artist to render her vision onto a 3D canvas - almost like painting a sculpture in the real world, except that all the work is done inside a computer program.
Koi 3D contains a real-time simulation of water, complete with real world properties of water such as reflection and refraction. As the koi fish swim beneath this virtual liquid surface, they appear bent and distorted as waves move over them.
As they swim, each koi uses a little artificial intelligence "brain" to allow it to navigate the pond alongside all the other fish, avoiding collisions by predicting where the other fish are heading and steering clear of them accordingly.
The Water Surface Simulation
rendering of the pond can occur, the water surface must be simulated. A variety of methods are known
and discussed in the scientific and computing literature
for doing this, but many of them are extremely computationally intensive and thus inappropriate for
The most accurate method of simulating water available relies on solutions to the 3D Navier-Stokes fluid
equations. This method is most commonly used for non-realtime
3D graphics (for example, the CGI water found in many films) and requires a prohibitive amount of computational
effort. Realtime applications, such as video games, usually involve some
simplified form of solutions to the 2D versions of the Navier-Stokes
For Koi 3D, we used a simple discrete "propagator"
equation to model wave movement along discrete surface
points of the water surface. Effectively, we represent
the surface as a series of floating points connected
by springs. This produces roughly similar results to
a complete solution to the differential equations with
less computational effort, and produces visually convincing
water movement and response.
Modelling true reflection in realtime is very difficult.
Reflection is a complicated light phenomenon in which
light rays "bounce" off of multiple objects.
To achieve this effect in realtime requires an alternative
to a realistic physical model of reflection.
For our reflection effect, we rely on Cubic Environment
Mapping, a feature built into most modern 3D graphics
cards. Cubic Environment Mapping uses a virtual "camera"
to take six images of the current scene from each of
six different angles corresponding to the six faces
of a cube (top, bottom and the four sides).
cube face images (the reflection "cube map") are loaded into your graphics card
so they can be used to generate reflections from any
possible angle. The reflection visible on a reflective surface (for example, the teapot below) can then be generated
by sampling from the correct location on this virtual cube
In Koi 3D, the reflection cube map is one of the
few pieces of data that, for performance reasons, is
not updated every frame. In fact, the reflection cube
map doesn't change at all when the view camera is stationary,
only when it is moving, and even then, it is only updated
once every few frames due to amount of computational work required
to render 6 separate images of the entire scene onto the water surface.
Refraction is a more complex phenomenon in real life.
The actual amount of distortion, or bending of the light,
created by a fluid surface depends on the index of refraction
of the material through which the light is passing (in
this case, water). The physical law that describes this
distortion is called Snell's Law, and it relates incidence
angles of light rays to refracted angles. Again, though,
simulating the real physics of water is currently computationally
prohibitive for a realtime application like Koi 3D.
As a result, we use a simplified technique called
Projective Texture Mapping to create the appearance
of refraction. This technique involves first rendering the
portion of the image below the water surface (the pond
bottom and fish in our case) onto a flat image in your
computer's memory. This flat rendered image (or texture map) is then
projected back up onto the water surface simulation. The water surface geometry
is used to vary the region of the texture map that is sampled,
creating the effect of refraction ripples around areas
where the water surface is particularly distorted. The amount of distortion that results is similar to the
index of refraction in a real liquid.
To tie all these effects together, the various rendering
techniques (reflection, refraction, and wave movement) need to be combined when drawing the finished
product, a dynamic water surface that moves and shifts
The fresnel effect, or the amount of reflected
light (or surface reflection) visible based on the viewing angle, is calculated
each frame for each of a large number of sampling points
on the water surface. This data is then used to blend the reflection
and refraction images together to produce a seamless
water surface, with bright, beautiful fish
moving fluidly beneath it.