Go to Top

I take a further look into the shader pipeline and reveal some more places where we can customise the rendered pixels. By the end of the article we have built a rim edged, ramp texture driven toon shader.

Motivation

You should read this article if:

  • You would like more detail on surface shaders
  • You would like to build a basic toon shader
  • You would like no know about ramp texture lookups
  • You would like to know about rim lighting

Planning the Shader

We want to make a toon shader - one that makes our models look like they are drawn as a cartoon rather than being a very realistic model.  To do that we are going to do a number of things:

  • Simplify the colors used in our model
  • Simplify the lighting so that we have well defined areas of light and dark
  • Draw an outline in black around our model

The Surface Shader Pipeline

Ok so there are a couple of bits of the surface shader pipeline that we might want to use that I simplified out of my diagram in article #1.  Let's look at what else we can do:

You can see from this diagram that there are two more stages we can write custom code for.  First we can write a custom lighting model that will allow us to apply light data to the output of our surface program to come up with a pixel value and then there is a final modification where we can just fiddle with the color before it is output to the screen.

Step 1 - Simplifying the Colours

We'll start with a basic bump mapped shader and add a few things.

To start with, let's just see how to use that finalcolor function so we can reduce the number of colours coming out of our texture.

	#pragma surface surf Lambert finalcolor:final

First we add the optional finalcolor program marker to our #pragma directive telling it we want to write a function call final.

	_Tooniness ("Tooniness", Range(0.1,20)) = 4

Then we add a _Tooniness property with a range of 0.1 to 20 and a default value of 4 - we will use this to decide how many colours we will limit our texture to.  Of course as we've defined a property we also need to add a variable with exactly the same name.

		float _Tooniness;

Now we can write our simple color modification program:

	void final(Input IN, SurfaceOutput o, inout fixed4 color) {
			color = floor(color * _Tooniness)/_Tooniness;
		}

We simple fix the range of the color by multipling it (remember it's rgba) by our tooniness, removing any floating point values and then dividing it back down again. That's it, not the best effect in the world but it will do for a start!

Here's the complete shader:

Shader "Custom/Toon" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Bump ("Bump", 2D) = "bump" {}
		_Tooniness ("Tooniness", Range(0.1,20)) = 4
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200

		CGPROGRAM
		#pragma surface surf Lambert finalcolor:final

		sampler2D _MainTex;
		sampler2D _Bump;
		float _Tooniness;

		struct Input {
			float2 uv_MainTex;
			float2 uv_Bump;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);
			o.Normal = UnpackNormal( tex2D(_Bump, IN.uv_Bump));
			o.Albedo = c.rgb;
			o.Alpha = c.a;
		}

		void final(Input IN, SurfaceOutput o, inout fixed4 color) {
			color = floor(color * _Tooniness)/_Tooniness;
		}

		ENDCG
	} 
	FallBack "Diffuse"
}

Adjusting the tooniness will change how resolved the colours are.  It's sort of a useful effect, because most of the time toon shading works best on models with low numbers of colours.  But it's not really a toon shader yet.  However at least we now know how to make a final modification to the colors.

Step 2 - Toon lighting

Ok so lets resolve to actually do some toon lighting, where the lights on things have sharp edges rather than smooth gradients.  To do that we are going to write a custom lighting program.

At this stage it's worth adding another variable.  As the current code doesn't really make it that cartoony, we should add another property called _ColorMerge that will deal with that element and we'll have _Tooniness handle the lighting - far more reasonable!

		_ColorMerge ("Color Merge", Range(0.1,20)) = 8

And its variable:

		float _ColorMerge;

Right so now we want to add a lighting program.  This is another of those coding by convention times in shader programming.  Rather than Lambert lighting we've been using up to now, we replace it with Toon.

		#pragma surface surf Toon

Note that we've removed the final color function - in a moment you'll see I've put it in the surface shader, which is better when it comes to this lighting (we get more variety and it still looks toony).

Have said we want to use Toon lighting we have to write a function called LightingToon - in other words prepend Lighting to the name of the model you use in the #pragma.

		half4 LightingToon(SurfaceOutput s, half3 lightDir, half atten)
		{
			half4 c;
			half NdotL = dot(s.Normal, lightDir); 
			NdotL = floor(NdotL * _Tooniness)/_Tooniness;
			c.rgb = s.Albedo * _LightColor0.rgb * NdotL * atten * 2;
			c.a = s.Alpha;
			return c;
		}

Lighting functions always take three parameters - the output from our surface program, the direction of the light and the attenuation to use.

They always return the color of the lit pixel.

So this is how lighting works - we take the light direction and the normal of the pixel and produce the dot product.  Remember that the dot product is 1 if the two items are facing each other -1 if the are exactly opposite and 0 at the 90 degree point.  That's very helpful for lighting of course - a pixel directly facing the light will get its full colour.  Anything beyond 90 degrees will become black and unlit and there will be an interpolation in between.

Remember Unity is summing this for each light and will automagically add the ambient light and intensity to the pixel we returned from surface - in here we want to say a pixel is black if it isn't affected by the current light.

For our Toon shading we add a very similar function to the one we had for the colour merging.  Basically we take a lovely smooth interpolated value that would be applied to the colour of the pixel and then make it have distinct steps by multiplying it by the tooniness, removing the fractional part and dividing it out again.  In other words there will be sharply defined areas of lightness.

This surface program just has the colour merging in it now:

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);
			o.Normal = UnpackNormal( tex2D(_Bump, IN.uv_Bump));
			o.Albedo = floor(c.rgb*_ColorMerge)/_ColorMerge;
			o.Alpha = c.a;
		}

The full shader code is here:

Shader "Custom/Toon" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Bump ("Bump", 2D) = "bump" {}
		_Tooniness ("Tooniness", Range(0.1,20)) = 4
		_ColorMerge ("Color Merge", Range(0.1,20)) = 8
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200

		CGPROGRAM
		#pragma surface surf Toon 

		sampler2D _MainTex;
		sampler2D _Bump;
		float _Tooniness;
		float _ColorMerge;

		struct Input {
			float2 uv_MainTex;
			float2 uv_Bump;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);
			o.Normal = UnpackNormal( tex2D(_Bump, IN.uv_Bump));
			o.Albedo = floor(c.rgb*_ColorMerge)/_ColorMerge;
			o.Alpha = c.a;
		}

		half4 LightingToon(SurfaceOutput s, half3 lightDir, half atten)
		{
			half4 c;
			half NdotL = dot(s.Normal, lightDir); 
			NdotL = floor(NdotL * _Tooniness)/_Tooniness;
			c.rgb = s.Albedo * _LightColor0.rgb * NdotL * atten * 2;
			c.a = s.Alpha;
			return c;
		}

		ENDCG
	} 
	FallBack "Diffuse"
}

Step 3 - Removing Rotational Artefacts

Ok so the problem with these sudden changes is that as the model rotates pixels may shift quickly from light to the next step darker and perhaps back again.  We really want to smooth the transitions.  To do that the best idea is to create something that will make that smoothing for us - we could try to write a function - but the easiest way turns out to be a thing called a ramp texture.

The ramp texture lets us turn our lovely smooth NdotL (normal of the pixel, dot product with the light direction) into a range of steps with slight smoothing between them.  And the great news is we can just use a simple sampler2D to convert our normal onto the texture!

We can use this technique because the UVs of the texture will be between 0..1 so we plug in the u as the value of NdotL and make v halfway down the texture and we're done.

Obviously we need a new property for our ramp texture:

		_Ramp ("Ramp Texture", 2D) = "white" {}

And a variable to hold its sampler:

		sampler2D _Ramp;

Then we update the lighting program to look like this:

		half4 LightingToon(SurfaceOutput s, half3 lightDir, half atten )
		{
			half4 c;
			half NdotL = dot(s.Normal, lightDir); 
			NdotL = saturate(tex2D(_Ramp, float2(NdotL, 0.5)));

			c.rgb = s.Albedo * _LightColor0.rgb * NdotL * atten * 2;
			c.a = s.Alpha;
			return c;
		}

We now modify NdotL by saturating (clamping between 0..1 remember) the texture lookup from the ramp texture.  Otherwise it's exactly the same.

Here's the complete source for that shader:

Shader "Custom/Toon" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Bump ("Bump", 2D) = "bump" {}
		_Tooniness ("Tooniness", Range(0.1,20)) = 4
		_ColorMerge ("Color Merge", Range(0.1,20000)) = 8
		_Ramp ("Ramp Texture", 2D) = "white" {}
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200

		CGPROGRAM
// Upgrade NOTE: excluded shader from Xbox360 because it uses wrong array syntax (type[size] name)
#pragma exclude_renderers xbox360
		#pragma surface surf Toon 

		sampler2D _MainTex;
		sampler2D _Bump;
		sampler2D _Ramp;
		float _Tooniness;
		float _ColorMerge;

		struct Input {
			float2 uv_MainTex;
			float2 uv_Bump;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);

			o.Normal = UnpackNormal( tex2D(_Bump, IN.uv_Bump));
			o.Albedo = floor(c.rgb*_ColorMerge)/_ColorMerge;
			o.Alpha = c.a;
		}

		half4 LightingToon(SurfaceOutput s, half3 lightDir, half atten )
		{
			half4 c;
			half NdotL = dot(s.Normal, lightDir); 
			NdotL = tex2D(_Ramp, float2(NdotL, 0.5));

			c.rgb = s.Albedo * _LightColor0.rgb * NdotL * atten * 2;
			c.a = s.Alpha;
			return c;
		}

		ENDCG
	} 
	FallBack "Diffuse"
}

Adding a Border

Right so now for a toon effect we want to add a border of black around our model.  In a surface shader the only real way we've got of doing that is by doing rim lighting (in black!)

Rim lighting looks for the pixels which are nearly 90 degrees away from the view direction and, in this case, turns them to black.

You've probably guessed that the dot product is going to come in handy here - but we also need to know about the direction the camera is facing, because we want this black edge to be relative to that.

Of course we are going to need a property and a variable to control our outline:

		_Outline ("Outline", Range(0,1)) = 0.4

And

		float _Outline;

We are going to be detecting these edges in our surface program, and it's there we need to get the direction of the view - luckily that's going to be magically worked out for us if we just include viewDir in our surface shaders Input structure - like this:

		struct Input {
			float2 uv_MainTex;
			float2 uv_Bump;
			float3 viewDir;
		};

Now all we have to do is detect the edge in the surf function.

			half edge = saturate(dot (o.Normal, normalize(IN.viewDir))); 
			edge = edge < _Outline ? edge/4 : 1;
			o.Albedo = (floor(c.rgb*_ColorMerge)/_ColorMerge) * edge;

First we work out the dot product to the edge by taking the normal of the pixel and the view direction.  Then if it's less than our property cut off value (remember 0 means 90 degrees from the view direction) we make it a small number (a divide by 4 seems to work well), if it's above that then we make it simply a 1 (no effect). We just multiply that value into our colour and away we go.

The full code for this shader is here:

Shader "Custom/Toon" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Bump ("Bump", 2D) = "bump" {}
		_Tooniness ("Tooniness", Range(0.1,20)) = 4
		_ColorMerge ("Color Merge", Range(0.1,20000)) = 8
		_Ramp ("Ramp Texture", 2D) = "white" {}
		_Outline ("Outline", Range(0,1)) = 0.4
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200

		CGPROGRAM
// Upgrade NOTE: excluded shader from Xbox360 because it uses wrong array syntax (type[size] name)
#pragma exclude_renderers xbox360
		#pragma surface surf Toon

		sampler2D _MainTex;
		sampler2D _Bump;
		sampler2D _Ramp;
		float _Tooniness;
		float _Outline;
		float _ColorMerge;

		struct Input {
			float2 uv_MainTex;
			float2 uv_Bump;
			float3 viewDir;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);
			o.Normal = UnpackNormal( tex2D(_Bump, IN.uv_Bump));
			half edge = saturate(dot (o.Normal, normalize(IN.viewDir))); 
			edge = edge < _Outline ? edge/4 : 1;
			o.Albedo = (floor(c.rgb*_ColorMerge)/_ColorMerge) * edge;
			o.Alpha = c.a;
		}

		half4 LightingToon(SurfaceOutput s, half3 lightDir, half atten )
		{
			half4 c;
			half NdotL = dot(s.Normal, lightDir); 
			NdotL = tex2D(_Ramp, float2(NdotL, 0.5));

			c.rgb = s.Albedo * _LightColor0.rgb * NdotL * atten * 2;
			c.a = s.Alpha;
			return c;
		}

		ENDCG
	} 
	FallBack "Diffuse"
}

Conclusion

So this article has taken our toon shader about as far as it can go using the surface shader model.  Actually the best way to create that outline (so that it works with less smooth shapes) is to run two passes - but to do that we are going to have to write a fragment shader and learn how to do lighting ourselves!  I'll leave that until next time...

, , , , , ,

7 Responses to "Noobs Guide To Shaders #4 – Toon shading (basic)"

  • Sam
    November 12, 2012 - 6:00 am Reply

    I’m getting this error when trying to do Step 3:

    Shader error in ‘Custom/Toon’: D3D shader assembly failed with: (28): error X5508: _sat not permitted on tex* instructions.
    (40): error X5204: Read of uninitialized component(*) in r3: *r/x/0 g/y/1 b/z/2 a/w/3

    It seems to have something to do with this line:

    NdotL = saturate(tex2D(_Ramp, float2(NdotL,0.5)));

    If I comment it out the error goes away.

    • whydoidoit
      November 12, 2012 - 7:47 am Reply

      That’s odd it compiles and runs fine here! It does give a warning about Flash support. Commenting out the NdotL saturate line is actually removing the toon lighting bands. Actually the saturate isn’t needed – only the tex lookup – (I used to have a multiply in there and took it out). I’ll update the article.

      • Sam
        November 12, 2012 - 10:24 am Reply

        Huh, works fine without the saturate function. Thanks.

  • Son Vu Hoang
    August 2, 2014 - 10:26 am Reply

    Hi,

    I’m stuck at step 3. I have a 3D model, but it does not have texture for Ramp Texture. Do I have to create the texture for Ramp Texture? If I have to create that one, how do I do that?

    Thank you.

    • Son Vu Hoang
      August 3, 2014 - 2:28 am Reply

      Hi, I found the ramp texture in the Toon Shading Package of Unity.

      • whydoidoit
        August 3, 2014 - 4:53 am Reply

        The ramp texture cab be saved from the image on this page using right click

Leave a Reply

%d bloggers like this: