When multiple objects from the same object class (e.g. multiple specific examples of the general class of automobiles) are all set in correspondence, we can build a morphable model. This model uses a base object from the set of objects and the correspondences to generate a rich representation for the class as a whole. The parameters of the model can be adjusted to create an interpolation or extrapolation among the cars of the model.
Figure 1 shows one example of a morphable model.
The eight
3D car models shown on the top have been combined into a morphable model.
The figures on the bottom show some of the cars which the morphable model
can produce by adjusting the model parameters.
Thus, this new model can be used for the synthesis of new shapes, the
analysis of new shapes, and for constraining other algorithms.
|
|
|
|
|
|
|
|
|
| ||
|
| ||
|
| ||
|
| ||
| Figure 1: The top eight cars are the 3D models used for the model. The lower eight cars are examples of the possible cars produced by the model. | |||
Figure 1 shows a number of synthesis results for cars.
New cars, not given in the initial specifications of the model, are
constructed automatically by the morphable model by adjusting the
parameters. In
this case, the parameters adjusted were the raw parameters of the model.
These parameters may not be intuitive. However, given labelings of the
example shapes used to build the model (in the automobile case, these might
be cost of the car, model year, ``sportiness,'' etc.) standard machine
learning techniques can be used to map the raw parameters onto parameters
corresponding to these labelings. Thus instead of adjusting the raw
parameters, the user can adjust the sportiness of the car, for instance.
Given examples of flexible models, these new parameters could be descriptions
of an expression (for a human face model) or the positions of the hands
(for a clock model).
 |
| Figure 2: A morphable model of six animals |
The model can also be used for analysis. Instead of going from a set of parameters to a shape, we can go from a shape to the parameters which best describe that shape. Thus, if we are given a new car, we can get values for the learned parameters (like sportiness or model year) as estimated by the model.
As different example of analysis, we can build the model shown in figure 2. If we then ask the model, without any training or labeling of the shapes in the model, to compare the similarity of the shapes, we get the matrix shown graphically in figure 3. With the exception of the cheetah (which is the base animal in the model and thus produces a slight asymmetry), the model agrees that the cats are related and the dogs are related and that cats and dogs are not related. Thus, through unsupervised learning, we have found a distinction between cats and dogs.
 |
| Figure 3: The matrix of shape correlation in the model. Red values indicate positive correlation and blue values indicate negative correlation. |
The model encodes knowledge about the class of objects. In particular, it codes which shapes belong to the class and which do not. This can be used as a constraint for other algorithms. For instance, assume that for 3D reconstruction (in which images of an object are used to compute the 3D shape of the object), we know the object viewed belongs to a class of objects for which we have built a model. 3D reconstruction often results in noisy shapes since it is unclear which areas of the object are actually sharp and which appear to be so due to noise in the inputs. The model can be used to add exactly this kind of knowledge as a constraint on the resulting shape. This allows better reconstructions using fewer views of the object. At the moment, I don't have any ready results on 3D reconstruction.