Saturday, February 1, 2014

Render To Texture (RTT)

For my next android application I need to apply simple Gaussian blur effect. For this to achieve I need to get current scene to texture and then perform post-production filter, in my case Gaussian blur filter.
In this post I’ll cover only FBO, Gaussian blur will be covered in next post.
To achieve this render scene to Frame Buffer Object (FBO), which is off-screen rendering technique, which allows us to render scene to a texture. We can use this texture to apply any post-processing filters.
Here is screenshot of what we are trying to achieve
clip_image002[22]
In this post I’ll try to cover
1) Load images from Assets folder (android specific)
2) Initialize FBO
3) Render to FBO
4) Use rendered FBO texture in other scene

As usual code is available in Google Code.

Load images from Assets folder

This is a straight forward, not much code is involved. Below is the code
public static Bitmap GetFromAssets(GLSurfaceView view,String name)
{
    Bitmap img = null;
    //get asset manager
    AssetManager assetManager = view.getContext().getAssets();
    InputStream istr;
    try {
        //open image to input stream
        istr = assetManager.open(name);
        //decode input stream
        img = BitmapFactory.decodeStream(istr);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return img;       
}

 

Initialize FBO

This is the most important part in using FBO, once initialization is done properly rest of the process is very simple.
I’ll put initialization in below steps.
a) Generate Frame Buffer
b) Generate Texture, to use with frame buffer
c) Generate Render Buffer
d) Bind Frame buffer generated in first step
e) Bind texture
f) Define texture parameters like format, dimension and min/mag filters.
g) Bind render buffer and define buffer dimension
h) Attach texture FBO color attachment
i) Attach render buffer to depth attachment
Make sure that dimensions are in POT (Power Of Two), because some devices may not support NPOT textures.
On a general note, to be on the safe side use images with POT dimensions.
Below is the code part for initializing FBO
int[] temp = new int[1];
//generate fbo id
GLES20.glGenFramebuffers(1, temp, 0);
fboId = temp[0];
//generate texture
GLES20.glGenTextures(1, temp, 0);
fboTex = temp[0];
//generate render buffer
GLES20.glGenRenderbuffers(1, temp, 0);
renderBufferId = temp[0];
//Bind Frame buffer
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId);
//Bind texture
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, fboTex);
//Define texture parameters
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, fboWidth, fboHeight, 0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
//Bind render buffer and define buffer dimension
GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, renderBufferId);
GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16, fboWidth, fboHeight);
//Attach texture FBO color attachment
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, fboTex, 0);
//Attach render buffer to depth attachment
GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, renderBufferId);
//we are done, reset
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0);
GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, 0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);

  Render to FBO

Rendering to FBO is same as normal rendering except for two initial steps
1) Bind the frame buffer to which we rendering
2) Set the viewport size to FBO width and height
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId);
GLES20.glViewport(0, 0, fboWidth, fboHeight);
        ******Rendering Code*******
       
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
 

Use rendered FBO texture in other scene

Now that we have rendered scene to texture, we can use this texture in other scenes or for post-processing etc.,
public void onDrawFrame(GL10 arg0)
{       
    //call FBORenderer to render to texture
    fbor.RenderToTexture();
    //reset the projection, because viewport is set by FBO renderer is different
    GLES20.glViewport(0, 0, vwidth, vheight);
    float ratio = (float)vwidth/(float)vheight;
    float a = 5f;
    Matrix.orthoM(m_fProj, 0, -a*ratio, a*ratio, -a*ratio, a*ratio, 1, 10);   
    //multiply view matrix with projection matrix
    Matrix.multiplyMM(m_fVPMat, 0, m_fProj, 0, m_fView, 0);
    //below procedure is same as any other rendering
    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT|GLES20.GL_DEPTH_BUFFER_BIT);
   
    GLES20.glUseProgram(iProgId);
   
    vertexBuffer.position(0);
    GLES20.glVertexAttribPointer(iPosition, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer);
    GLES20.glEnableVertexAttribArray(iPosition);
   
    texBuffer.position(0);
    GLES20.glVertexAttribPointer(iTexCoords, 2, GLES20.GL_FLOAT, false, 0, texBuffer);
    GLES20.glEnableVertexAttribArray(iTexCoords);
   
    GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, iTexId);
    GLES20.glUniform1i(iTexLoc, 0);
    //since I'm multi-texturing, bind fboId to texture1
    GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, fboId);
    GLES20.glUniform1i(iTexLoc1, 1);
    //for rotating cube
    Matrix.setIdentityM(m_fModel, 0);
    Matrix.rotateM(m_fModel, 0, -xAngle, 0, 1, 0);
    Matrix.rotateM(m_fModel, 0, -yAngle, 1, 0, 0);
    //multiply model matrix with view-projection matrix
    Matrix.multiplyMM(m_fMVPMat, 0, m_fVPMat, 0, m_fModel, 0);   
    //pass model-view-projection matrix to shader
    GLES20.glUniformMatrix4fv(iMVPMat, 1, false, m_fMVPMat, 0);   
    //draw cube
    GLES20.glDrawElements(GLES20.GL_TRIANGLES, cube.m_nIndeces, GLES20.GL_UNSIGNED_SHORT, indexBuffer);
}
If you look at the code, FBO related initialization, rendering is separated in FBORenderer class to avoid confusion when we try to achieve Gaussian blur using 2 FBOs.
That’s it in this post. Next post will be on Gaussian blur.
If you have any related questions/queries or If you find any mistakes in this post please do leave a comment.