ShaderLab Programming #1 – Sepia Blend Shader

Now that exams are over, I’ve had time to actually breathe and, more importantly, do game dev stuff. I’ve rarely touched computer graphics in the past, but it’s an area I’ve always wanted to explore in detail. This summer, I plan to mostly spend my time learning the intricacies of computer graphics, and that’s partially where Unity’s ShaderLab steps in.

ShaderLab is a graphics language in its own right, but it also encompasses Cg, a high-level shader language by Nvidia, and HLSL, a Microsoft shader language for use with DirectX, both of which were developed together. We don’t need to know too much about the structure of ShaderLab programs for what we’re doing, so I’m largely going to ignore it for now. I needed to create a post-processing effect similar to Unity’s built-in Sepia Tone, which is an on/off effect, with a tweak that allows it to fade in and out instead. It’s for a game I’m working on in which you can turn back time – more on that in the coming days when I’ve put together something a bit more concrete hopefully! Let’s deconstruct this super-simple program and explore what a sepia effect actually is. Unfortunately, Unity’s Sepia Tone shader seems to use some sort of maths wizardry for its fragment shader that I don’t understand, so I’ll also be writing mine mostly from scratch. Here’s the number bullshit right here:

fixed4 frag (v2f_img i) : SV_Target
{ 
    fixed4 original = tex2D(_MainTex, i.uv);
 
    // get intensity value (Y part of YIQ color space)
    fixed Y = dot (fixed3(0.299, 0.587, 0.114), original.rgb);

    // Convert to Sepia Tone by adding constant
    fixed4 sepiaConvert = float4 (0.191, -0.054, -0.221, 0.0);
    fixed4 output = sepiaConvert + Y;
    output.a = original.a;
 
    return output;
}

The infinite wisdom of the Internet tells us that to convert a source image to a sepia-tone version, we just define a function of the input colours as follows:

new red = min(in red * 0.393 + in green * 0.769 + in blue * 0.189, 255)
new green = min(in red * 0.349 + in green * 0.686 + in blue * 0.168, 255)
new blue = min(in red * 0.272 + in green * 0.534 + in blue * 0.131, 255)

This formula represents 100% sepia; that is, the source image completely turned into a sepia tone. But we don’t always want a completely sepia image – we need to incorporate some sort of progress measure. Using this formula, we can just interpolate between the input colour and the full sepia tone and use a separate script to control the ‘progress’ of the interpolation. I’ve talked about interpolation before, so if this really big word is confusing, it just means we’re picking a colour on an imaginary line between ‘original colour’ and ‘sepia colour’, with a parameter between 0 and 1. We can now create another function to get our final ‘blended’ sepia colour:

out red = parameter * new red + (1 - parameter) * in red
out green = parameter * new green + (1 - parameter) * in green
out blue = parameter * new blue + (1 - parameter) * blue

With the theory out of the way, we can now create the shader and the script that will control it. We’ll be utilising the post-processing effects provided by Unity, so make sure you import them from the Standard Assets using Assets->Import Package->Effects from the menu bar. We’ll start with the controller script.

There are two main ways to keep track of the progress, so I’ll cover both. The first way is to define a method such that another script can just pass in a float to act as the parameter.

using UnityEngine;

namespace UnityStandardAssets.ImageEffects
{
    [ExecuteInEditMode]
    [AddComponentMenu("Image Effects/Color Adjustments/Sepia Blend")]
    public class SepiaBlend : ImageEffectBase
    {
        private float progress = 0f;
        private bool active = false;

        private void SetProgress(float progress)
        {
            this.progress = progress;
            active = (progress == 0f);
        }

        // Called by camera to apply image effect.
        void OnRenderImage(RenderTexture source, RenderTexture destination)
        {
            if(active)
            {
                material.SetFloat("_Progress", progress);
                Graphics.Blit(source, destination, material);
            }
        }
    }
}

OnRenderImage() is the method called when we want to render a post-processing image. We set the ‘progress’ parameter of the material attached to this project – we’ll see how this works with that material’s shader shortly – and use Graphics.Blit() to apply the texture created by the shader to the screen. The second way to control the progress parameter is to allow external scripts to set the ‘active’ boolean and use a time-based interpolation to decide what the progress should be. The guts of the program will look more like this:

private float progress = 0f;
private bool active = false;

private void SetActive(bool active)
{
    this.active = active;
}

private void Update()
{
    progress = Mathf.Lerp(progress, active ? 1f : 0f, Time.deltaTime
        * 5f);
}

// Called by camera to apply image effect.
void OnRenderImage(RenderTexture source, RenderTexture destination)
{
    material.SetFloat("_Progress", progress);
    Graphics.Blit(source, destination, material);
}

Now for the actual shader! I’ll explain very quickly what the fragment shader is doing and ignore the vertex shader since it’s doing nothing in this example; I may do another post in the future explaining the details of how the shader is constructed, but I’m glossing over a lot for now. Don’t even worry about it yet.

Shader "Hidden/SepiaBlendEffect"
{
    Properties
    {
        _MainTex ("Base (RGB)", 2D) = "white" {}
        _Progress ("Progress", Float) = 0
    }
    SubShader
    {
        Pass
        {
            ZTest Always Cull Off ZWrite Off

            CGPROGRAM
            #pragma vertex vert_img
            #pragma fragment frag
 
            #include "UnityCG.cginc"

            uniform sampler2D _MainTex;
            uniform float _Progress;

            fixed4 frag(v2f_img i) : SV_Target
            {
                fixed4 original = tex2D(_MainTex, i.uv);
 
                half3x3 vals = half3x3
                (
                    0.393, 0.349, 0.272, // Values for input red.
                    0.769, 0.686, 0.534, // Values for input green.
                    0.189, 0.168, 0.131  // Values for input blue.
                );

                half3 input = half3(original.rgb);

                half progress = half(_Progress);

                half3 intermed = (progress * mul(input, vals)) + 
                    ((1 - progress) * input);

                fixed4 output = half4(intermed, original.a);

                return output;
            }
            ENDCG
        }
    }
    Fallback Off
}

We’re interested in the stuff between ‘CGPROGRAM’ and ‘ENDCG’ – this is written in Nvidia’s Cg. All we need to know about the structure of shaders for now is that the fragment shader is run on every pixel of a processed image. I’ll also ignore all the types of variables for now and cover those in another post at a later date. Jumping straight to the frag() method, which constitutes the fragment shader, we’ll see a 3×3 table of numbers, vals, that looks strikingly like the one defined for our first function; the table is the transpose of a matrix formed by those functions’s numbers. Our shader performs a matrix multiplication on a vector made of the original rgb values, input, and our magical sepia number matrix vals (which is why it had to be transposed), then we do our interpolation to get intermed. Afterwards, we put together the final output by adding the original colour’s alpha onto intermed, and boom! We have our Sepia Blend shader. Now all that remains is to attach the SepiaBlend script to the main camera and assign the SepiaBlendEffect shader to the script, then we’re good to go.

I hope this has been a helpful introduction to the world of computer graphics. I’ve uploaded the three scripts to avoid you having to copy everything out (both variants of SepiaBlend.cs are included; pick your preferred version). Let me know if everything turns into a bugfest when you attempt to run any of the examples and I’ll try to diagnose the problem.

Retro Corner: Pokemon Emerald (GBA, 2005)

Pokemon. One of the biggest gaming franchises of all time. You would have trouble finding people who haven’t heard of it, or its lovable creatures, as there is a huge merchandising industry built on the back of this videogame series, as well as the equally big trading card franchise. This classic Nintendo series came around, as the legend goes, because creator Satoshi Tajiri was a huge fan of collecting bugs as a child, and when he saw the Game Boy link cable for the first time, he imagined bugs crawling from one handheld to the other, prompting him and artist Ken Sugimori to conceive arguably the greatest JRPG of all time, starting with Pokemon Red and Green in Japan (Green was named Blue everywhere else). Now enough of the background, time to talk about one of the games in the series, Pokemon Emerald. pkmn_emerald_2

This game is the third in a trilogy that had Ruby and Sapphire preceding it, together forming the third generation of Pokemon. It was the third best-selling title in the Gameboy Advance’s lifespan, beaten only by Pokemon Ruby/Sapphire in first and Pokemon LeafGreen/FireRed in second (seems Pokemon is a big deal…).  What I loved about the third generation is the debut of double battles. These battles had a more strategic edge to them, requiring the player to choose which moves to use with deeper thought, for example using moves which attack both opponents as opposed to just one. This battle system was used a large number of times throughout the game, which is quite different to the triple battles of Pokemon Black/White (DS, 2010) which I feel where neglected hugely, appearing a couple of times only. pkmn_emerald_1

The storyline of the game is a little similar to other games in the series (evil team tries using legendary Pokemon to take over world, hero raises a team of Pokemon to stop them, while collecting Pokemon League badges and battling their rival along the way, beats evil team and subsequently the Pokemon League, the end), but it fits the game well. The game is well presented, and the graphical quality of the environments and the battles is superb, squeezing all it can from the GBA. The Pokemon introduced in Generation 3 include some of my favourite designs, such as Blaziken, Flygon (see above screenshot), Salamence and Rayquaza (the cover star of the game). blaziken

Blaziken- he’s a fire chicken. Your argument is invalid.

The multiplayer features of this game are also pretty darn cool; there’s no better way to test your skills as a Trainer than challenging a friend to a link battle, and you can’t legitimately fill the Pokedex without trading with a friend, showing Game Freak’s determination to get players to interact with one another.

The battle scene in this game and the other games of generation 3 are in full colour, which is hugely improved upon the games of the previous two generations which only had the Pokemon themselves in colour (Gold, Silver, Crystal, Yellow to an extent) or the entire game in black and white (Red, Blue). Admittedly, there’s not much of an addition, a basic platform for each Pokemon to stand on and a background made of coloured lines, but it is better than the older battle scenes. The attack animations are also improved from before, and the music is also of very high quality, adding to the largely happy atmosphere of the game.pkmn_emerald_5

The reason to buy this game over the previous two in the generation is firstly the slightly enhanced storyline (the power struggle between Team Aqua and Team Magma makes slightly more sense here), and the fact Rayquaza is the cover star- he is an amazing species to have in your team, and besides, he looks awesome! The best post-game content in the game is the Battle Frontier, a facility that allows you to pit your Pokemon against different challenges with a variety of rules to test your abilities. It is an expansion of the Battle Tower of Ruby and Sapphire, which was rather limited in its challenges. Rayquaza

Rayquaza- I don’t really know what he is, but he looks awesome anyway.

If you haven’t played this before, it is worthwhile picking it up, perhaps you can find it cheap on eBay or from a retailer. Have fun playing 🙂

-Daniel

An introduction

Hey people, this blog will be full of game reviews and the progress I am making on my own game. Since I play so many video games, I decided to make my own with all the ideas I’ve accumulated, so I’ve started learning the Java programming language. I will be showing my progress here, and also maybe help you if it’s possible (I’m only starting out with Java, but I’m hoping to get better, so maybe in the future I will have my own YouTube series showing people how to make simple games).

The games I play include the likes of Minecraft (awesome game by Mojang, if you haven’t played it, why not? Get it now), Mario, Pokemon and generally stuff by Nintendo (the only current-gen console I own is a 3DS, and it’s not a real console), but like everyone else, I have played games like Call of Duty and Halo. So if I find any games worthy of review, I’ll review them. So stick around and have fun reading!

-Daniel