How L.A. Noire Conquered The Uncanny Valley With A Tech Called MotionScan [Video]

From the moment Rockstar unveiled their trailer for upcoming detective thriller, L.A. Noire from Aussie developers Team Bondi, we’ve known that the bar for motion capture in video games has not only been raised, but sent in the course of the roof. Animators were flirting with the brink of the uncanny valley for years, but now using some incredibly intelligent and advanced technology, the fellows from Team Bondi have created the 21st century version of Pinocchio, turning digital game characters into realistic, animated people. several weeks ago, we got to determine the technology behind this amazing breakthrough up close.

” Match me, Sydney”
- Sweet Smell of Success, 1957

You’d never realize it from the skin of the nondescript building in Culver City in La, but the united states home of Australian company Depth Analysis houses among the world’s most advanced motion capture technology. Dubbed ” MotionScan” , the technology is a mixture of high definition facial scanning and intelligent algorithms which can capture high definition video of an actor’s face in stunning, true-to-life detail, and then convert it to a virtual three-dimensional head for placement inside the game.

The story of the way this Australian technology came to be is sort of as impressive as the outcome. The brainchild of Team Bondi founder Brendan McNamara, Depth Analysis began its life back in 2004 as a small research project. Consistent with Rockstar VP of Development Jeronimo Barrera, it was born from a longing to capture more detail within the motion capture studio.

” Brendan had quite a lot of experience with motion capture and he wanted in order to capture performances, meaning the face. Doing traditional motion capture you’re capturing the bones, and he didn’t would like to capture what was on the inside, he wanted to capture what was on the surface. In order that they came up with a plan and he and Oliver Bao developed MotionScan, that’s now called Depth Analysis.”

The road from research project to functional technology has been a protracted one. Although Team Bondi and Depth Analysis had been developing the Motionscan technology since 2004, it was only this year that they actually began filming actors within the studio for L.A. Noire. Since January, more than 400 actors had been put in the course of the Depth Analysis capture room and more than 2200 pages of script were acted out for the game. Aaron Staton, who plays the game’s lead protagonist, Cole Phelps, has put in more than 80 hours of MotionScan filming himself.

” Louie, this may be the beginning of an attractive friendship”
- Casablanca, 1942

At the centre of the seemingly magical setup that makes your entire scanning process possible is a soundproof and lightproof room, kitted out with a terribly precisely aligned 32 camera rig in a 360 degree arrangement around the actor’s central chair. Like a scene from Kubrik’s 2001: A Space Odyssey, the room is uniformly lit in a bright, sterile white light, with the intention that there are absolutely no dark spots on the actor’s face. Having a well lit subject is significant to the technology working.

The 32 Machine Vision cameras are very precisely positioned in pairs around the chair, like numbers around a roulette wheel. Even the slightest bump of the rig could offset the alignment, and fixing it might probably soak up to four hours for each pair. The setup – which was moved from Sydney to LA over the Christmas holiday period last year – took Head of study and Development at Depth Analysis Oliver Bao over ten 16-20 hour days to install the 1000-plus pieces inside the rig. Each $6,000 camera is capable of sending 1080p video at 30 frames per second directly to the server room simultaneously, that can then be became a status scan in as little as 15 minutes. Bao tells us they’re a similar cameras NASA uses during space shuttle launches to provide them a couple of seconds of footage before melting away.

Each camera is fixed enthusiastic about the actor, who has to sit down extremely still and act with their face – there’s only about 50cm worth of leeway for head movement in the course of the acting process – which makes it a difficult proposition for the actors involved. Among the many main reasons the studio was moved from Sydney to LA was because most of the Australian actors being used inside the early days struggled to control the usa accent while sitting extremely still. While on the chair, the actors need to wear an orange shirt with three green balls attached, which act as reference points for the advance team back in Sydney after they add the Depth Analysis footage to the motion captured acting done at a separate facility.

” It’s very streamlined. We get geometry, colour, plus body orientation and voice, and it’s all recorded in one go. [The dots] are to track where your vertebrae is, so you know where the neck join is in animation, we use those three points to deduct it, And once it’s found, we just plunk the pinnacle onto an animation like a Lego head” explains Bao.

Besides the 32 camera setup, there’s a 33rd camera that connects directly to an adjacent room for the director to look at the performance in real time and a screen that can display the script for the actor, play back footage or allow the actor to look the director inside the next room. A lapel microphone and reference mic makes sure the actor’s voice is properly captured without any popping or background noise.

Considering the volume of information coming in from 32 FullHD cameras, the server setup intensive Analysis is breathtaking. Nine servers, each capable of 300MB/s speeds, are used for the video recording (although the ninth server is redundant), while a 45TB buffer sits alongside the server tower, working away at 650MB/s. Footage is captured to the video server, then moved to the buffer after it’s been edited, before being copied to tape and sent back to Sydney to be processed.

” Experience has taught me never to trust a policeman. Just if you think one’s o.k., he turns legit”
– The Asphalt Jungle, 1950

The Depth Analysis setup is capable of capturing 50 minutes worth of footage in a day. That won’t sound like lots, but compared to a conventional motion capture setup’s 10 to 15 minutes, it suddenly becomes a far more efficient venture. It also completely does away with the desire to animate a character’s face.

” We basically just select the processing configurations, the presets, and just let it go. So there’s no character artists or animators touching up the face. We just hand it over to them to place onto the body and do the top orientation adjustments. So basically we just cut out traditional animation – it’s an awful lot faster this manner,” explains Bao.

The culmination is not only faster – it’s also breathtakingly detailed. The team from Rockstar showed us a level from L.A. Noire called ” The Fallen Idol” , a case about a B-Grade actress and her 15 year old niece surviving an attempted murder at the hands of a movie producer. A dead ringer for every crime show you’ve ever watched on television, as the evidence comes at hand it becomes clear that there is more at play than just an attempted murder… There’s blackmail, rape, underground pornography and organised crime all mixed up within the case.

But what’s most striking in regards to the mission as it was played out – and what makes LA Noire the appropriate launch vehicle for a technology like MotionScan – is that the act of interrogating witnesses and suspects relies so much on the performance of the actors, and how it’s shown on screen. The subtlest shift of the eyes from the actress; the slightest twitch inside the character’s voice or the smug tip of a curved lip all influence your decision as a player to whether or not you might be being lied to. Unlike any online game that has ever come before it, using these facial cues is an essential component of the gameplay, and it feels natural because the expressions are so natural.

Through either fortuitous timing or an element of some Rockstar masterplan, the Depth Analysis studio was actually reshooting scenes from The Fallen Idol mission on the day of the tour. Sitting within the reception area waiting to be shown the technology that makes the complete operation possible, a gorgeous young girl with a cut across her right eye walked past. The immediate thought was that she looked familiar. It took several seconds, but eventually we realised that it was the actress who played the teenage girl, Jessica Hamilton, within the Fallen Idol mission of the game. In a city like LA you predict to work out some celebrities, but you don’t necessarily expect to recognise game characters. As the tour progressed and we watched her refilm her lines from that episode, it was an awesome just how lifelike the in-game character actually was.

An excellent better indication of the detail captured by the Motionscan process was watching a scene from the game being acted by a floating head – captured from an actor inside the Depth Analysis chair and rendered on a screen. One scene specifically, where the character seems unsure about whether or not he actually killed someone the previous evening shows the real power of the technology. By capturing a real actor’s performance, you’ll discover every nuanced expression, from the confused sadness in his eyes, to the quivering voice and trembling lips as he considered the burden of what he can have done, played out on a computer monitor by a floating head which may be manipulated in real time.

Thanks to Depth Analysis, L.A. Noire is already blurring the lines between video games and cinema, that is away from a very easy task, as Jeronimo Barrera explains:

” We’re trying something new that’s never been done. We’re not just releasing a game – everybody looks at this who hasn’t seen it in person, even from the screenshots you take a look at it and go, ‘Oh, it’s GTA with Fedoras on’. But the fact is that it’s a complete new different concept, it’s an entire new way of watching interactive entertainment. I believe we’re commencing to blur the lines between a television program and a online game.”

” I caught the blackjack right behind my ear. A black pool spread out at my feet. I dived in. It had no bottom.”
- Murder, My Sweet, 1944

One surprising aspect of the Depth Analysis scans in L.A. Noire is that they’re pretty heavily compressed. After showing a line played out by the floating head of Aaron Staton, Oliver Bao explains:

” The raw image is really significantly better quality. It’s suitable for film and post-production use. So it’s not limited to just games – This actually started as a capture for cutscenes, but we ended up using it for full game face. At the time, nobody thought this was actually possible, and nobody knew how one can use it. So once it was working, people started thinking how they may actually make use of it and that became portion of the core gameplay” .

Given that the technology used for L.A. Noire is so advanced and provides this kind of new level of immersion to gameplay, it’s your decision to assume that it’s far the longer term direction of all games from Rockstar. Not so, in line with Barrerra:

” Are we going to exploit MotionScan in every game? Not likely. But where it fits we can definitely use it.

” It’s sort of a difficult thing to do because you have a look at this thing and you instantly go, ‘Oh, they’ve got this new tech, in order that they’re gonna do a little gimmicky tech demo game’, and it’s certainly not what it is. It helps, and it’s there, nonetheless it’s also the fantastic writing by Brendan, it’s [Head of Production and Design at Team Bondi, Simon Wood]‘s amazing eye for detail, it’s all of the artists, the entire animators… It’s been a gigantic collaborative effort to make this thing.”

But on condition that LA Noire is solely the first practical implementation of MotionScan, the longer term looks pretty bright for the small Australian company.

” We’ve only got two senior programmers and one junior programmer at this stage – it’s an extremely tiny team. Basically the three of us in most cases created the entirety,” explains Bao.

” [We're] all Australian… for now. We’re gazing installing an everlasting state studio here, and we’re also staring at doing more R&D work [within the US] in addition, because we want to work more closely with clients, they have to have someone here that’s technical to work with them.”

Yet despite spending six years creating the technology and watching it slowly move from research project to a potentially game-changing motion capture technology, Bao still hasn’t subjected himself to the process.

” I refuse to. There’s nowhere to hide. Reality will probably be cruel and it’s not DA’s fault… That’s what I say” .

Yet for Rockstar and Team Bondi, the indisputable fact that there’s nowhere to hide will prove to be their greatest strength.

Republished with permission from Gizmodo Australia

Source

This post is tagged: advanced technology, australian technology, capture technology, game characters, intelligent algorithms, sweet smell of success, traditional motion