Materials
and methods for 3d mouse ear visualization
DOWNLOAD ALL RELEVANT
PAGES IN A WINZIP ARCHIVE HERE
AVI stereo movie
of mouse ear - 441kb
QUICKTIME
stereo movie of mouse ear - 689kb
In order to better visualize the
structures in the inner ear of
mice, the 3d animation software Alias Wavefront | Maya was used
to construct 3d models from light micrographs of sequential sections
through ear tissue. Maya is not generally used as scientific visualization
software but its powerful scripting environment (Maya Embedded
Language, or MEL), variable tolerances for model precision, and
high quality renderer make it very well suited to this purpose.
microscopy
/ image preparation
First, digital micrographs were
captured of the sections. In this case, many images were taken
of a single section and then montaged together to create one single
hi-res section image, approximately 3000 pixels square. This was
repeated for each of 106 sequential sections through the tissue.
For this project we wanted the option of viewing the macroscopic
features and then zooming in on cellular detail in the 3d movie,
so high resolution section images were used. This may or may not
be necessary for other applications. A rule of thumb for image
resolution is that one wants to end up with more or less square
'voxels' (3d pixel-cubes) in the final 3d output, so the pixel
size in the section images should be approximately equal to the
distance between sections plus the section thickness. For example
if the sections are 1 micron thick and every section is to be
used, then the distance between sections (1) plus the section
thickness (0) is 1 micron. If the tissue sample is 200 microns
in diameter then the section images should be approximately 200
pixels wide. Section pixel size can be made smaller than but shouldn't
exceed the section thickness. Final output movie resolution should
also be considered when choosing source image resolution. If the
final movie is to be on video (640x480) then using 200x200 pixel
section images may cause the final render to look pixellated.
Another general rule for section image resolution is that it should
be higher than the final output movie resolution, as much as 2-3x
higher to avoid pixellation. This may cause the section image
pixel size to be larger than the section thickness, but that will
be ok. Resolution can always be reduced if necessary but not effectively
increased once the micrographs are taken. An image of a micron
calibration slide was taken at the same magnification as the rest
of the section images to calibrate the 3d scene units according
to real-world units.
Once
the section images were montaged, they were copied and these new
images were hand colored individually to highlight specific features
that were to be visualized separately in the 3d output. Hand coloring
the sections allowed them to be batch processed using Adobe Photoshop
to separate out the different colored features into batches of
images with only those specific features present. Contrast and
saturation of the section images was also tweaked to provide optimum
translucence for the 3d model, so that the tissue did not appear
as a solid mass because the value was too opaque, and alpha channels
were created to mask the textures in Maya.
from
2d to 3d
The
section images were colored as follows: Arterial
lumen was colored red, sensory
patches in the vestibular apparatus were colored purple,
coclear duct
was yellowish, and the
rest was left blue, the color of the stain. Some of the
colors changed in the 3d texture processing, however. In the final
output the arterial
lumen reconstruction became a deep red with some cyan
on the surface, sensory
patch reconstruction became ivory, and the coclear
duct reconstruction became a yellow-green. Uncolored
features reconstruction remained similar in color. All
these separate movies were composited in Adobe AfterFX to a
single reconstruction movie with all the above features.
The above movies are of the first control ear. Below are the same
links and links to KO images.
Control
and ko ear avi movies:
...The
3d process used was to stack the sections as textures with alpha
transparency on
planes. The
main hurdle to overcome when converting planes into 3d images
was the 2-dimensionality of the planes causing them to be disappear
when viewed directly from the side. The first idea was to use
a shader glow in Maya instead of standard color mapping for the
section image texture.
However, since shader glow is still dependent on how much of the
plane is visible the glow still disappears when the plane is parallel
to the camera or incident angles. The glow provided a nice luminous
transparent effect and helped bring out some subtle color differences
in tissue. Two solutions were found to overcome the 2d plane issue.
unfortunately both were very cpu intensive at render time. The
method used in this example is to apply a geometry displacement
map with a random 3d noise texture to all the frames, providing
a 3d grain effect similar to film grain, so that no matter which
way the plane stack is viewed the planes one can never see through
to the other side because the surfaces have small random bumps
on them, equal to the distance between planes. The number and
smoothness of these simulated grains will of course have a direct
effect on render time. A compromise was reached for this project
which fell short of completely eliminating the dimming at incident
angles but did improve it significantly. Process optimization,
budget, and available hardware will dictate the quality of the
final output.
Another, perhaps better solution
which was explored was to take this 3d grain idea one step further
and create a 3d particle
system for each section with randomly generated particles
aligned to those section planes. These particles were then mapped
with the section images and alpha channels. This would allow for
very accurate ray-traced shading of the features rather than the
glow emission used in our example. To match the resolution of
the section planes, unfortunately, will require approximately
10,000 texture mapped particles per plane, times 106 sections
in this example equals 1,060,000 particles, times 40 frames for
the rendered movie, times 4 different movie layers for the different
ear features. With all these multipliers the render time on a
standard 3d workstation quickly becomes inordinate so this method
was shelved pending a major hardware upgrade.
Review of Meatmorph software
from Meta Imaging
...Meta
Imaging's Metamorph is a scientific imaging application that is
designed to do what we did here with Maya. I tested a copy (briefly)
and made this 3d reconstruction
with our source images. Render time was a fraction of Maya's,
but the output quality was slightly lower. Perhaps with some color
tweaking image quality would improve.
...Meta
Imaging dealt with the 2dness of the planes in a different way
than we did. There is a blur or some similar effect applied to
the color in a direction perpendicular to the planes. When the
planes turn just a little bit to the side they will start to blur,
(to observe this stop the 3d recon movie at approx. 90°) so you
only really see the finest detail when looking at the stack head-on.
At a glance I think its a decent easy to use program, at least
the 3d reconstruction part, which is all I looked at. Considering
Maya complete is a fraction of the price of Metamorph and much
more flexible, especially if you're working with any developers,
we will continue working with Maya and developing our particle
system templates for 3d visualization from sections.
For
more information on this process, to purchase templates, or to
arrange a consultation please email info@glyphimaging.com.
