Seeing is no longer believing.
The image you see on the evening news could well be a fake—a fabrication
of fast new video-manipulation technology.
Last year, Steven Livingston, professor of political communication
at George Washington University, astonished attendees at a conference
on the geopolitical pros and cons of satellite imagery. He didn’t
produce evidence of new military mobilizations or global pandemics.
Instead, he showed a video of figure skater Katarina Witt during
a 1998 skating competition.
In the clip, Witt gracefully plies the ice for about 20 seconds.
Then came what is perhaps one of the most unusual sports replays
ever seen. The background was the same, the camera movements were
the same. In fact, the image was identical to the original in
all ways except for a rather important one: Witt had disappeared,
along with all signs of her, such as shadows or plumes of ice
flying from her skates. In their place was exactly what you would
expect if Witt had never been there to begin with—the ice, the
walls of the rink and the crowd.
So what’s the big deal, you ask. After all, Stalin’s staff routinely
airbrushed persona non grata out of photos more than a half-century
ago. And Woody Allen ushered a variation on reality morphing into
the movies 17 years ago with Zelig, in which he inserted
himself next to Adolf Hitler and Babe Ruth. In films such as Forrest
Gump and Wag the Dog, reality twisting has become commonplace.
What sets the Witt demo apart—way apart—is that the technology
used to “virtually delete” the skater can now be applied in real
time, live, even as a camera records a scene and instantly broadcasts
it to viewers. In the fraction of a second between video frames,
any person or object moving in the foreground can be edited out,
and objects that aren’t there can be edited in and made to look
real. “Pixel plasticity,” Livingston calls it. The implication
for those at the satellite imagery conference was sobering: Pictures
from orbit may not necessarily be what the satellite’s electronic
camera actually recorded.
But the ramifications of this new technology reach beyond satellite
imagery. As live electronic manipulation becomes practical, the
credibility of all video will become just as suspect as Soviet
Cold War photos. The problem stems from the nature of modern video.
Live or not, it is made of pixels, and as Livingston says, pixels
can be changed.
 The
best-known examples of real-time video manipulation so far are
“virtual insertions” in professional sports broadcasts. Last January
30, for instance, nearly one-sixth of humankind in more than 180
countries repeatedly saw an orange first-down line stretched across
the gridiron as they watched the Super Bowl. New York-based Sportvision
created that line and inserted it into the live feed of the broadcast.
To help determine where to insert the orange pixels, several game
cameras were fitted with sensors that tracked the cameras spatial
positions and zoom levels. Adding to the illusion of reality was
the ability of the Sportvision system to make sure that players
and referees occlude the virtual line when their bodies traverse
it.
Last spring and summer, as Sportvision and rivals such as Princeton
Video Imaging (PVI) in Lawrenceville, N.J,, were airing virtual
insertion products, including simulated billboards on walls behind
major league batters, a team of engineers from Sarnoff Corp. in
Princeton, N.J., flew to the Coalition Allied Operations Center
of NATO’s Operation Allied Force in Vicenza, Italy. Their mission:
transform their experimental video processing technology into
an operational tool for rapidly locating and targeting Serbian
military vehicles in Kosovo. The project was dubbed TIGER, for
“targeting by image georegistration.” “Just dial in the coordinates
and the thing goes,” explains Michael Hansen, a young, caffeinated
Sarnoff gadgeteer who can hardly believe he was helping fight
a war last year.
Compared to PVI’s job, the military’s technical task was more
difficult—and the stakes were much higher. Instead of altering
a football broadcast, the TIGER team manipulated a live video
feed from a Predator, an unmanned reconnaissance craft flying
some 450 meters above Kosovo battlefields. Rather than superimposing
virtual lines or ads into sports settings, the task was to overlay,
in real time, “georegistered” images of Kosovo onto the corresponding
scenes streaming in live from the Predator’s video camera. The
terrain images had been previously captured with aerial photography
and digitally stored. The TIGER system, which automatically detected
moving objects against the background, could almost instantly
feed to the targeting officers the coordinates for any piece of
Serbian hardware in the Predator’s view. This was quite a technical
feat, since the Predator was moving and its angle of view was
constantly changing, yet those views had to be electronically
aligned and registered with the stored imagery in less than one-thirtieth
of a second (to match the frame rate of video recording).
In principle, the targeting step could have been hotwired to
precision guided weapons. “We weren’t actually doing that in Allied
Force,” Hansen notes. “We were just telling targeting officers
exactly where Serbian targets were and then they would vector
in planes to go strike the targets.” That way the human decision
makers could pre-empt flawed machine-made decisions. According
to the Defense Advanced Research Projects Agency, TIGER technology
was used extensively in the final three weeks of the Kosovo operation,
during which “80 to 90 percent of the mobile targets were hit.”
So far, real-time video manipulation has been within the grasp
only of technologically sophisticated organizations such as TV
networks and the military. But developers of the technology say
it’s becoming simple and cheap enough to spread everywhere. And
that has some observers wondering whether real-time video manipulation
will erode public confidence in live television images, even when
aired by news outlets. “Seeing may no longer be believing,” says
Norman Winarsky, corporate vice president for information technology
at Sarnoff. “You may not know what to trust.”
Continued...
Ivan Amato is a correspondent for National Public Radio and
the author of Stuff: The Materials the World Is Made Of
a chronicle of cutting-edge research in materials science.
Copyright © MIT's Technology Review
July/August 2000
|
