Back to Blog

How to Use Masks in After Effects

By Kyle Hamrick
After Effects

Masks are an essential tool in After Effects. If you want to elevate your work and keep pace with the pros, this is one skill you need to pick up

As a motion designer, you're expected to be a compositing wizard. Of the many tools in our trade, masks reign supreme. This simple to learn and hard to master tool can open up your MoGraph game like none other, and it's important that you learn the basics before trying something crazy.
Masks_AE_Article.jpg
Masks are a fundamental After Effects feature that allow you to cut out and combine different elements together, build quick & easy transitions, and are the basis of visual effects and compositing work in motion design.
Today, we're going to take a look at:
  • What masks are and what they do
  • How to create them
  • The properties of masks and what each of them do
  • Working with, transforming, and animating masks
  • And then I’ll show you a few useful examples
If you want to follow along, snag the project files below. Now let's get started!

Follow along with the project files here!

Download Now

How to Use Masks in After Effects

What is a Mask?

Masks in After Effects allow you to determine the visibility of specific parts of a layer, similar to features that may already be familiar if you use Photoshop, Illustrator, or other image editing apps. You can isolate an object in a photo or video, create a cutout, make shapes, or even insert your friends into their favorite music videos!
mask-in-after-effects-1.gif
Masks are created by drawing paths on any layer with a visual component; that is, anything you can create or import into After Effects that you can actually see in a composition.
mask-in-after-effects-17.gif
It’s possible to create what are called open paths, which are just kind of a line you’re drawing on the layer, and those don’t do anything by themselves. But when you close up that path, or draw a complete shape, you’re creating a closed path, which will cut out a portion of the layer.
mask-in-after-effects-18.gif
When you do this, you’re affecting what’s known as the layer’s alpha channel, which is an additional channel that defines transparency in the clip or image. This is the basis of what’s called compositing: when you combine different elements together to create a new cohesive image or video.

How to Create a Mask

To create a mask, you first need to select the layer you want to apply it to. Then grab a shape tools from the toolbar, which you can also cycle through quickly by pressing Q. Simply click and drag in your Composition viewer, and congrats, you’ve made a mask! 
mask-in-after-effects-4.gif
One important note: if you start using these drawing tools without a layer selected, you’ll be creating what’s called a shape layer, which I love, but that’s a whole other topic. 
Your mask will draw from the top-left corner—unless you hold CMD or CTRL, in which case it’ll draw from the center. If you’re drawing a rectangle or ellipse, holding SHIFT will keep all the sides equal, allowing you to make perfect squares or circles.
mask-in-after-effects-5.gif
Many times you’ll need to draw something more organic, though, and that’s where the Pen tool comes in. You may recognize this tool from Photoshop or Illustrator, and it works very much the same way here.
mask-in-after-effects-6.gif
To create rounded corners, I’ll click and drag when creating these points. There are some variant Pen tools up here for adding, subtracting and modifying these path points. Pressing G repeatedly will cycle through these different modes. 
mask-in-after-effects-7.gif

Learning Mask Properties

I’ve shown how to create masks, now let’s see how they actually work!
You can see I already have a mask on this layer, so I’ll press the M key to reveal it on the timeline. 
mask-in-after-effects-20.gif
The first property you’ll see on a mask is the Mode. This determines what the mask actually does - Add means you’re seeing only what’s within the mask. Subtract means you’re seeing everything that’s not within the mask. None is a useful mode for when you’re working with a mask, but still need to be able to see the rest of the layer. These other modes really only come into play once you add multiple masks to a layer.
mask-in-after-effects-21.gif
Two mask layers in Add
mask-in-after-effects-22.gif
Two mask layers with one using Subtract
mask-in-after-effects-23.gif
Two mask layers using Intersect
Cool pro-tip: You can actually set these modes as you’re drawing the mask by pressing the corresponding key before releasing your mouse click. 
mask-in-after-effects-8.gif
Just like layers and effects, the stacking order matters. I’d encourage you to just grab an image, draw a couple masks, and explore this functionality to get more comfortable with it. 
If you are adding multiple masks to a layer, it's important to stay organized. You can rename any mask by clicking on the name and pressing Enter … and you can manually choose the color of each mask by clicking on its little color chip.
Next we have Mask Feather, which adjusts how hard or soft the mask edges are. This is measured in pixels and is centered on the path, so if I set this to 100, it’s feathering 50 pixels inside the mask, and 50 pixels outside.
mask-in-after-effects-9.gif
Mask Opacity adjusts how opaque or transparent this individual mask is.
Mask Expansion lets you either shrink or grow a mask - this is also measured in pixels - without actually having to change the path itself. Animating this property can be a very easy way to create an iris reveal, for example: 
mask-in-after-effects-10.gif

Working With Masks

One very important thing about masks is that they always move and transform with the layer they’re on. If you want to move the layer around on screen, but keep the mask in the same position relative to the rest of the composition, you can actually use the Pan Behind tool to do that.
mask-in-after-effects-11.gif
What if I want to keep the layer in place but move the mask? I can click on either the Mask’s name or the Mask Path property in the timeline. We’ll just move this around a little bit...
mask-in-after-effects-12.gif
To free transform the entire mask, you can double-click directly on the path. Now you’ll be able to move it around, as well as scale and rotate it. To close that out, double-click again, either directly on the path or anywhere outside of it.
Often, you’ll want individual control over one or more of these points, which After Effects calls vertexes. To select multiple points at once, you can either hold Shift and click more points, you can grab any straight path segment to move both of those points at once, or you can actually drag a box around all the points you’d like to control, and move them all at once.
mask-in-after-effects-13.gif
Since After Effects is an animation program, we can of course be making our mask change over time, which I can do by activating the stopwatch here on the Math Path property, to start creating keyframes. So, maybe I’ll start with this shape here, go forward a little bit here, and then change a few points.
mask-in-after-effects-14.gif
I know maybe that’s not the most exciting thing ever yet, but it’s the basis of all the other masking work you’ll do in the future, so it’s good to get a handle on how all of this works.

Examples

Before we wrap up, let’s take a look at a few quick examples of using masks in your everyday work.
mask-in-after-effects-15.gif
A vignette is a nice finishing step for many compositions and can help you direct the viewer’s eye to a specific part of the frame. Let's go up to LayerNewSolid … or you can press Control or Command-Y. Choose a color from our image, maybe like this very dark purple.
mask-in-after-effects-25.gif
Now we have a solid covering the whole frame, which is obviously not what we want, but if we come up here to our shape tools, choose our ellipse and double click on it … it will actually create an elliptical mask exactly the size of our layer.
mask-in-after-effects-26.gif
This still isn’t quite what we’re going for, so let's come down here, and we can either set this to Subtract, or there's also this little invert button that will quickly invert your mask.
Let's twirl this open, crank this feather up to about 350, and turn up the expansion a little bit, just to nudge this into a sweet spot.
mask-in-after-effects-27.gif
That looks pretty good. I could probably stop there, but sometimes it's nice to make your vignettes look a little bit more organic. You can just tweak these points a little bit, play with these tangent handles, and fiddle with this until you're happy.
Another easy way to put masks to work for us is creating a quick little revealer with our image. We start by drawing two rectangles: one over the left half, and one over the right.
mask-in-after-effects-30.gif
I didn't get them perfect, but that's easy to fix. I hit the quote (") key to bring up my safe zone display, which has this handy crosshair at the center of the composition. Grab the edges and move them into place. Let’s maybe go to about frame 10, open both of these up, keyframe these mask paths, move forward, and either move the entire mask, or just grab one piece and slide it off to the edge.
mask-in-after-effects-31.gif
There we go, we have a nice, quick little revealer.
mask-in-after-effects-16.gif
It is important to remember that these masks will move with the layer, so you wouldn't want to do this until your layer is definitely set in place … but this is a nice quick and easy way to wipe off the layer like this. Just by drawing your masks differently, you could make this angled, make it a little more complex, whatever works for your project.

Wrap Up

So that’s the basics of Masks in After Effects. Understanding how they work is an essential skill for any After Effects artist, and this is just the beginning.
If the possibilities of compositing using these kinds of tools has really piqued your interest, you should take a look at VFX for Motion, taught by industry legend Mark Christiansen
If you still need to master more of the fundamental After Effects skills, you should check out After Effects Kickstart, where Nol Honig gives you the ultimate intro to After Effects.
Get to creating, and we'll see you next time.
mask-in-after-effects-32.gif