Physical Props (Proposal Breakdpwn Part 2)

The second section of my proposal spoke of physical props although this will actually include all the things to think about when capturing the initial data. For example this quote from "Puppetology: Science or Cult?" by Brad deGraf and Emre Yilmaz in regards to the initial capture with non human characters in mind for the target link: http://www.awn.com/mag/issue3.11/3.11pages/degrafmotion.php3 "Ironically, it's often better to have less data than more -- we usually use only 12 body sensors. If you had one sensor for every single moving part of the body, you'd have a lot more information tying you to the human form, but for our purposes we just want enough sensors to convey the broad lines and arcs of the body."  Through my research I found a number of interesting contraptions that were used to modify human motion for use in motion capture.. They all looked a little dangerous though. This was used to help influence the movements of the Pleo the dinosaur toy: Pleo the Dinosaur for planet of the apes andy Serkis used stilt like extensions on his arms to help mimic apelike movement more effectively. Andy Serkis Giant Studios, which motion-captured live performers in choreographed fights to provide critical data for the animators of Reel Steel Reel Steel Groot in the Guardians of the Galaxy films due to be released this year is played by vin Diesel who has been sighted getting used to the props that will help him with the extra height needed when performing. Acting Techniques Using acting techniques to manipulate the data during capture is a useful tactic and much safer for me to investigate than some of the contraptions mentioned. Andy Serkis again is a great example of this in both his role as Caesar in planet of the apes and also as Gollem in the lord of the rings and hobbit films. Andy Serkis Disney did some research into this technique but instead of taking the straight capture they took key poses form the source data and matched it to the equivalent pose for the target character. From Research paper by Yamane (2010) link: http://www.disneyresearch.com/wp-content/uploads/nonhumanoid_sca10.pdf Moving Forward I think the main focus will be the acting techniques rather than the props other than maybe basic ones to help with weighting, for example if the character is meant to have more weight on the arms I could get the actor to hold something to imitate the extra weight.

Preserving Motion Through Retargetting (Proposal breakdown part 1)

This will be quite a long post outlining the work I did researching for the proposal. (I might split it into a few posts) I generally used my notebook to keep track of ideas rather than my blog which is why it has been a bit neglected however my notes are a little all over the place so I'll do my best to keep it from being confusing.

The main sections for my literature review were as follows:

• Preserving motion through retargeting
• Physical Props
• Acting and Key Poses
• Lattice deformations and Spline IK
• Spasm: Procedural Animation and real-time retargeting.

1. Preserving motion through retargeting.

Gleicher - developed a solver which preserved desirable qualities of motion using space time constraints to allow interactive control of animation via UI. The process described in the 1998 paper delivered as part of SIGGRAPH '98 initially uses the angles of the source characters joints to drive the movement in the target character however this leads to problems when the character is scaled as shown in figure 2 taken from the journal itself. 

To fix this he used IK solvers to help apply constraints to the characters feet, forcing them to stay planted but as IK solvers consider each frame independently it doesn't know what the motion is in relation to the frames around it resulting in "high frequency jerkiness".

Spacetime constraints: (Witkin and Kass 1988) uses 3 factors to help dictate the movement needed. What the character needs to do or the mechanical action of whats being moved so from one place to another or onto something etc, How it needs to do it so characteristics of the movements. and also how the physical structure of the character would affect the movement for example how the joints are able to move, what the mass of the character is etc.

Main problem Gleicher had was that despite being able to replicate the movements of the source character it took out all personality. 

Gleichers work may be beneficial in transplanting movements from source to target for replicating the mechanics of the movement but when retargeting to a target that has a different structure to the source then I would have thought a little flex from the original movement might be useful but then again with space time constraints one of the factors is the physical structure of the character which could either break the movement entirely or cope with it.. Might be a useful thing to look into?

Moving Forward
I mentioned in my dissertation about unity's mecanim and Mayas human IK, both these tools focus on the retargetting side of things specifically. I feel like exploring the possibilities these two processes offer would be in my best interest so I'll dedicate a post specifically to research into their capabilities hopefully resulting in pipelines specific to those.

Presentation slides and notes

This is long overdue. I got really bogged down with other coursework which is no excuse.

Slide 1
As written my Aim is to explore methods of retargeting motion capture data to non human characters in the hopes of discovering a more efficient pipeline than those currently available.

Slide2

This slide shows an example of the main thing i'm hoping to avoid. Especially when it comes to biped characters a real problem is getting them to move as if they aren't just a guy in a suit at a theme park.

Slide 3

This is the first of the existing examples i found. It's taken from a short called "40 Years" and the stills on screen are taken from a breakdown video they made. While it's very nice to see it visually I haven't been able to find any documentation on what processes they needed to go through to get to this point

Slide 4

The other example is Ted, They had Seth Macfarlane on set in the motion capture suit (the one that uses sensors rather than markers which eliminates the problem of occlusion and marker swapping as they resonate at different frequencies) As you can tell by the image there's a problem with proportions when its first captured which then needs to be fixed in cleanup.

Slide5
So as a result of the research (that i have really badly documented on this blog, but i'll get better!) I have four methods i can look into to try and improve the pipeline a little. I think that it would be best if i focused on this four alone to narrow the scope of the project but at the same time if i find something interesting that's relevant and worthwhile Its better that i follow that.

Slide 6

This is an example of how props can be used. I won't be able to do something as grand as these but I would be able to get the actor to hold weights to help shift their own weight. The obvious problems that come from using props to adapt a performance are listed in the bullet points. The center of gravity might appear wrong for the character if it's a quadruped but also marker occlusion/shifting and confidence. If you're in one of those rigs i'm not sure how confidently you'd be able to move.

Slide7

so a big chunk of my research currently has been looking at the procedural methods of animation found in spore. The animators had a massive task of having to produce animation for creatures they'd never seen so it's an interesting if not very wordy read

Slide 8

Tasks that i'd need to do to complete things

Slide9

A thankful hippo

Comments

The main comment i got after presenting was that it seemed like my scope was too big for the time we have. I think this was because my slides didn't really express that i only wanted to find ways of improving it slightly using techniques i found rather than fixing it entirely. Because of this i should probably look at revising my aim and objectives to reflect that more clearly.

Research Structure - Early research

Limitations within motion capture

Develop, Will Freeman, 2013http://www.develop-online.net/tools-and-tech/motion-capture-moving-with-the-times/0117523
Markers - Limit movement possible
Motion Builder only allows for retargetting between characters with the same amount of bones. The video a little further down shows that this doesn't need to be the case.

constraints highlighted by Michael Gleicher at 1998 siggraph

"We motivate our approach with an example: retargetting motion capture data of an actress walking up to, picking up, and carrying away a box. During pre-processing, we augment the motion data by specifying constraints that are essential to the action: the hands must grab the box in the middle frame, the hands must remain the correct distance apart while carrying the box, and the feet must be planted and not skid when they are on the ground."

NOTE: Consider the affect props have on the capture data. 

Expand

Examples of Non-Human Biped Characters

 (and possible free rigs and meshes i might use during testing)

Could be characters with counter weights like wings, tails or asymmetrical features

Could be characters with legs set up more like a birds

Different centers of gravity

Proportions - Look at Ted and how they applied real motion to the short limbs of the bear

Human Rig for comparison

How to Judge success

How well the data applies to the new hierarchy without any adaptation?

consider how long the adaptation would take? 

Is the method efficient enough to improve existing pipelines?

Existing Work

Havok Engine retargetting research: 2009 

What I'd like to produceWould like to produce something like the above video for final report. Demonstrating the same motion capture data retargeted to different biped creatures. Hopefully looking a little less stiff than the examples in this video. There should probably be at least an example of a walk cycle and also maybe an interaction with an object.

Things to look intoSomething something.. methods of creating additive animation used to help find a balance between mocapped data and whats needed for the motion to be believable for the character

Chain Maps? (retargetting between different character resolutions. arms, legs, spine.) Important at all?

Motion Capture Monsters

So this is now my project. 

I'm going to look into retargetting motion capture data onto non human characters. As far as i can see, very little has been done to explore this area with the majority of research done been into realistic human motion and facial motion capture.

http://www.youtube.com/watch?v=JGNnhkk4bfU<--Using extra props such as stilt shoes and crutches to get more realistic data.

http://www.youtube.com/watch?v=xr6Ddvi0wPE <-- Example of how a person changes the way they walk to try and add a different sense of weight to performance capture. (also 2D motion capture)

Facial retargetting

Facial mocap using a phone and drawn on markers -->http://vimeo.com/27678634

MoCap possibilities?

http://www.youtube.com/watch?v=KBd2ToLt8xQ

Guy from mixamo speaking about their process.. Their markers don't stick out and each marker has it's own frequency which makes marker swaps much less likely. I'm jealous.

What are the limits of the motion capture set up? Push it to it's extreme? what can't you do with 8 cameras that you can do with more. Find examples and mimic them with our set up?

Retargetting motion capture data to non human meshes. Monster motion capture
http://vimeo.com/13132912

Motion capture fighting game? Tekken did it? Bit of a tricky one with the whole not knowing anyone who can do flips and things...plus they had the capeaibara

Motion Capture - Ikinema

This module is hard.

I want to do something using motion capture but honestly my brain shuts down when I try to think at the moment.

Idea's that have gone nowhere so far..
-Motion capture to create an environment somehow

WAIT
-2D animation using motion capture (was a nowhere but now maybe not?)

Ikinema has a free version that works on browsers. I've figured out little bits using various documentations that i've found and produced this using the tutorial assets that they provide.




I'd like to think about possible applications for 2D motion capture. I want more of a reason to play with this other than just helping massively with the motion capture pipeline.

Maybe making tutorials for creating game animations using this process? Go through the entire pipeline?



Also sidenote.
This took less than half an hour in total. My honours project is silently weeping.