More FX stuff

For the last couple weeks since there’s been no filming going on, I’ve been working on some special effects and post stuff, as you may have noticed obviously. In particular I’ve been muddling through The Library scene, trying to prep the footage for the addition of the computer generated elements - by “prep the footage” I mean cleaning it up and whatnot.

I’ve also been working at creating some of the smaller computer generated elements. The Book of the Skull prop now has a digital counterpart, as well as the Jim-Jim prop, and I’ve been fiddling with some designs for the Library set and the Librarian himself.

I thought I’d do a quick breakdown of how much work goes into a single shot, by showing it layer by layer.

First we start off with a clean plate - the original footage, totally untouched. You can see that around the outer edges of the doorframe the walls are still visible, as well as a sliver of light from the crack in the door. So first off they’ll have to be removed.
Since the shot was locked off (on a tripod and unmoving) it’s pretty easy to create a mask to block out those areas and just turn them into dark patches (which is what I want). Here you see the mask.
Instead of just using black, I actually took a 10 second clip off the camera with the lense cap still on, so instead of a jet black, I get a grainy black - which matches the footage more accurately so it doesn’t seem too weird to the eye. And combined with the mask, we get this:
Now, at the very end of the shot, which is 272 frames long (30 frames per second), you can see the Malice crosses over in front of the door. That won’t work, so I created a rotospline mask copying her movements across the door, this mask is a full 272 frame video clip since it’s masking out the motion of an actor (I think I explained this process before so I’ll skip it here, if you want, search around here for some more info on it).
Here’s a still shot of from the mask of Malice crossing in front of the door. Luckily it only requires a small part to be masked out over roughly 30 frames, so the mask doesn’t have to be completely accurate, just where the sliver of light needs to be removed.
The two masks are then combined, the still frame and the rotosplined mask to create this. Just a still image of the final composite. Not really much to see so far.
And now you can see the when Malice crosses in front of the door it works like it should. This is just the first step, All I’ve done is clean up footage at this point (note that I actually already cleaned up the burned out pixels if you recall those little buggers).
Now that’s done, I need to add the computer generated element - which is the digital version of The Book of the Skull. First I had to scan in the entire book from as many angles as possible, model it and finally turn out this. Here’s a side by side comparison of the two books, the real one on the left, the digital one on the right. It’s not perfect, but it’s close enough for what it’s needed for - the digital one will require some post work to make it match the quality of the footage.
So, we start be setting up the scene. I purposely shot the scene with the pedestal empty in order to insert the digital book later - you might have noticed that actually. Here’s a quick render of the shot with the book added in. The block you’re seeing under the book is used for camera matching - in order to get the same angle and whatnot of the original plate - so the book ends up being shot from the exact same angle. Camera matching is a long, sometimes painful, process I won’t get into.
From there I do a rough animation of the book flying up off the pedestal and hovering for a second before flying off, which Heather had to try and react to. So then I refine it a bit more, trying to make it match what’s going on in the scene, in this case instead of having Heather react to a sequence of things going on to match the “to be added” book, I just walked her through it knowing I could just add the book and make it match with her actions instead. So I just told her to look up at a spot that made sense to her, then when I added the book I just lined it up as best I could to make it appear as though she’s actually looking at it.
Then we do a final render of the book animation. The textures, lighting and an added motion blue make it look much better. I won’t go into the process of how the book was actually made, I might do that at some other point for another digital prop or actor.
Now that I have my final book animation I still need to composite it into the shot. And since there’s a few things that need to be done to the footage to hopefully make it match a little more seamlessly, I have to create a mask for the book itself.

This is pretty easy, I just render another version of the animation, removing the lights, background and textures so I get a quick mask version of the footage. Also, the reason I render the animation against the original footage, is because otherwise you tend to get a bit of a “glow” surrounding the computer generated parts.
And since the book has to appear to be on the pedestal, another quick mask and a still frame one are added to block out the pedestal.
So with a complete mask for the book I add it with the cleaned up live action footage and this is what we get. You can see the shadows and whatnot of the book are just way to harsh to make it feel like it’s actually part of the original shot.
So I do some quick touch ups, adding some noise to simulate the same noise the live action footage has, adjusting the colour balance and brightness and contrast to get as close as I can. Not always easy to do and it doesn’t always work, so sometimes you just have to settle for “as close as I can”. The changes may be a little too subtle to be noticed here, but blown up on a large scale, it'd be really noticable.
That’s pretty much it, there’s a few other things I’ll add in - like a bit of a bloom off the lantern to help bring the elements together a little better, that’s really it. Otherwise this is the process I go through with every shot that require a clean up and then additional computer generated elements. Not "Hollywood" perfect, but still not terrible looking.

By no means is this an easy or simple process, nor is it something only a “professional” can do. I am by no means a professional, it just takes a buttload of patience and practice and of course the right software certainly doesn’t hurt. I get the impression sometimes that folks who don’t really understand this side of moviemaking just pass this sort of stuff off as “oh, it’s all done by computers” - well, that might be true, but it’s a bit like assuming cars can just drive themselves around. Computer generated imagery, whether it be matte paintings, or digital actors and props, are not something that just pop out of computers, it takes an incredible amount of work to make it happen.

Having said that, I don’t mean to imply I have more talent, dedication or knowledge of this stuff then anyone else, obviously I don’t - otherwise I’d be so rich I'd pay people to write this blog for me while George Clooney and I sit on my private Caribbean beach with my supermodel girlfriends. I just have way too much time on my hands, that’s pretty much it. And a touch of OCD doesn’t hurt either I suppose.