Render To Texture (RTT)

For my next android application I need to apply simple Gaussian blur effect. For this to achieve I need to get current scene to texture and then perform post-production filter, in my case Gaussian blur filter.
In this post I’ll cover only FBO, Gaussian blur will be covered in next post.
To achieve this render scene to Frame Buffer Object (FBO), which is off-screen rendering technique, which allows us to render scene to a texture. We can use this texture to apply any post-processing filters.
Here is screenshot of what we are trying to achieve

In this post I’ll try to cover
1) Load images from Assets folder (android specific)
2) Initialize FBO
3) Render to FBO
4) Use rendered FBO texture in other scene

As usual code is available in Google Code.

Load images from Assets folder

This is a straight forward, not much code is involved. Below is the code

public static Bitmap GetFromAssets(GLSurfaceView view,String name)

{

    Bitmap img = null;

    //get asset manager

    AssetManager assetManager = view.getContext().getAssets();

    InputStream istr;

    try {

        //open image to input stream

        istr = assetManager.open(name);

        //decode input stream

        img = BitmapFactory.decodeStream(istr);

    } catch (IOException e) {

        e.printStackTrace();

    }

    return img;       

}

 

Initialize FBO

This is the most important part in using FBO, once initialization is done properly rest of the process is very simple.
I’ll put initialization in below steps.
a) Generate Frame Buffer
b) Generate Texture, to use with frame buffer
c) Generate Render Buffer
d) Bind Frame buffer generated in first step
e) Bind texture
f) Define texture parameters like format, dimension and min/mag filters.
g) Bind render buffer and define buffer dimension
h) Attach texture FBO color attachment
i) Attach render buffer to depth attachment
Make sure that dimensions are in POT (Power Of Two), because some devices may not support NPOT textures.
On a general note, to be on the safe side use images with POT dimensions.
Below is the code part for initializing FBO

int[] temp = new int[1];

//generate fbo id

GLES20.glGenFramebuffers(1, temp, 0);

fboId = temp[0];

//generate texture

GLES20.glGenTextures(1, temp, 0);

fboTex = temp[0];

//generate render buffer

GLES20.glGenRenderbuffers(1, temp, 0);

renderBufferId = temp[0];

//Bind Frame buffer

GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId);

//Bind texture

GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, fboTex);

//Define texture parameters

GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, fboWidth, fboHeight, 0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null);

GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);

GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);

GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);

//Bind render buffer and define buffer dimension

GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, renderBufferId);

GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16, fboWidth, fboHeight);

//Attach texture FBO color attachment

GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, fboTex, 0);

//Attach render buffer to depth attachment

GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, renderBufferId);

//we are done, reset

GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0);

GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, 0);

GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);

  Render to FBO

Rendering to FBO is same as normal rendering except for two initial steps
1) Bind the frame buffer to which we rendering
2) Set the viewport size to FBO width and height

GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId);

GLES20.glViewport(0, 0, fboWidth, fboHeight);

        ******Rendering Code*******

       

GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);

 

Use rendered FBO texture in other scene

Now that we have rendered scene to texture, we can use this texture in other scenes or for post-processing etc.,

public void onDrawFrame(GL10 arg0)

{       

    //call FBORenderer to render to texture

    fbor.RenderToTexture();

    //reset the projection, because viewport is set by FBO renderer is different

    GLES20.glViewport(0, 0, vwidth, vheight);

    float ratio = (float)vwidth/(float)vheight;

    float a = 5f;

    Matrix.orthoM(m_fProj, 0, -a*ratio, a*ratio, -a*ratio, a*ratio, 1, 10);   

    //multiply view matrix with projection matrix

    Matrix.multiplyMM(m_fVPMat, 0, m_fProj, 0, m_fView, 0);

    //below procedure is same as any other rendering

    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT|GLES20.GL_DEPTH_BUFFER_BIT);

   

    GLES20.glUseProgram(iProgId);

   

    vertexBuffer.position(0);

    GLES20.glVertexAttribPointer(iPosition, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer);

    GLES20.glEnableVertexAttribArray(iPosition);

   

    texBuffer.position(0);

    GLES20.glVertexAttribPointer(iTexCoords, 2, GLES20.GL_FLOAT, false, 0, texBuffer);

    GLES20.glEnableVertexAttribArray(iTexCoords);

   

    GLES20.glActiveTexture(GLES20.GL_TEXTURE0);

    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, iTexId);

    GLES20.glUniform1i(iTexLoc, 0);

    //since I'm multi-texturing, bind fboId to texture1

    GLES20.glActiveTexture(GLES20.GL_TEXTURE1);

    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, fboId);

    GLES20.glUniform1i(iTexLoc1, 1);

    //for rotating cube

    Matrix.setIdentityM(m_fModel, 0);

    Matrix.rotateM(m_fModel, 0, -xAngle, 0, 1, 0);

    Matrix.rotateM(m_fModel, 0, -yAngle, 1, 0, 0);

    //multiply model matrix with view-projection matrix

    Matrix.multiplyMM(m_fMVPMat, 0, m_fVPMat, 0, m_fModel, 0);   

    //pass model-view-projection matrix to shader

    GLES20.glUniformMatrix4fv(iMVPMat, 1, false, m_fMVPMat, 0);   

    //draw cube

    GLES20.glDrawElements(GLES20.GL_TRIANGLES, cube.m_nIndeces, GLES20.GL_UNSIGNED_SHORT, indexBuffer);

}

If you look at the code, FBO related initialization, rendering is separated in FBORenderer class to avoid confusion when we try to achieve Gaussian blur using 2 FBOs.
That’s it in this post. Next post will be on Gaussian blur.
If you have any related questions/queries or If you find any mistakes in this post please do leave a comment.

Multi-Texturing

Multi-Texturing is nothing but mixing multiple texture as per your requirement. here I’ll mix these two textures to get the resulting screen below. its up to coder to mix textures to get desired effect.
Texture1:

Texture2:

and mixing result:

Let’s look at the code below. First we will look at shaders because major change is in shaders only.
Vertex Shader:

String strVShader =
"attribute vec4 a_position;" +
"attribute vec2 a_texCoords;" +
"varying vec2 v_texCoords;" +
"void main()" +
"{" +
  "v_texCoords = a_texCoords;" +
  "gl_Position = a_position;" +
"}";

Nothing fancy here, simple shader that passes texture co-ordinates to Fragment shader and sets vertex position.
Fragment shader:
This is where we will make few changes to mix/blend textures.

String strFShader =
"precision mediump float;" +
"varying vec2 v_texCoords;" +
"uniform sampler2D u_texId1;" +
"uniform sampler2D u_texId2;" +
"void main()" +
"{" +
     "vec4 color1 = texture2D(u_texId1, v_texCoords);" +
     "vec4 color2 = texture2D(u_texId2, v_texCoords);" +
     "gl_FragColor = color2 * color1;" +
"}";

Here, get the color values from the texture at the co-ordinates and manipulate them as you like. here i just multiplied 2 color vectors.
There is no much changes in remaining renderer code. but, we will see draw function.

public void onDrawFrame(GL10 gl) {
  GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
  GLES20.glUseProgram(iProgId);
 
  GLES20.glVertexAttribPointer(iPosition, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer);
  GLES20.glEnableVertexAttribArray(iPosition);
 
  GLES20.glVertexAttribPointer(iTexCoords1, 2, GLES20.GL_FLOAT, false, 0, texBuffer);
  GLES20.glEnableVertexAttribArray(iTexCoords1);
 
  GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
  GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, iTexIds[0]);
  GLES20.glUniform1i(iTex1, 0);
 
  GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
  GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, iTexIds[1]);
  GLES20.glUniform1i(iTex2, 1);
 
//  GLES20.glDrawArrays(GLES20.GL_TRIANGLE_FAN, 0, 4);
  GLES20.glDrawElements(GLES20.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, indecesBuffer);

 }

Apart from passing vertex positions and texture co-ordinates, we have activated 2 textures (GL_TEXTURE0 and GL_TEXTURE1) for passing texture ids to shaders. we can go upto 32 textures. and as usual draw triangles or triangle fan to draw quad.