O3D Tutorial 15: Getting the color of a pixel in an image

Aug 6

O3D has a peculiarity. For a graphics engine, it makes it rather tricky to handle images as simply images. This tutorial is a stepping stone for the next tutorial which I will put up which will be on how to create a landscape using a height map. mybloggingplanet.com

Before we get there, though, we need to first be able to determine the color at a particular spot on an image, which O3D does not directly allow, so you cannot just load the image into a texture, and try and get the color at a pixel location.

There is a solution though. Instead of using O3D to get the color information, we are going to load an image into an HTML canvas object, which DOES allow you to get at the color information we are wanting.

The sample application is a big departure from previous tutorials, and I have stripped most of it away. In the application, I am loading a texture onto a 2D canvas. The image represents a maze of sorts, and have created a mousemove event that updates the text onscreen depending on the color which the mouse is over. So, if the mouse is over the black part of the canvas, the text shows “border”. If you are on the white portion (the allowed area of the maze), it shows “good”, and if you are on the blue portion (which means you are in the middle of a wall), it shows “crashed”.

So looking at the code, we declare a few new globals.

var g_mazeImage;
var g_templateCanvas;
var g_templateCanvasContext;
var g_templateImageData;
var g_message = '';

The function to load the image data creates a canvas object, and a context linked to that canvas. Then the image passed to the function is inserted into the canvas, and finally an object containing the image data is created. It is this image data object that contains the color information we will use later.

function loadImageData(image)
{
  g_templateCanvas=document.createElement("canvas");
  g_templateCanvasContext=g_templateCanvas.getContext("2d");

  g_templateCanvas.width= image.width;
  g_templateCanvas.height=image.height;
  g_templateCanvasContext.drawImage(image,0,0);

  g_templateImageData = g_templateCanvasContext.getImageData(0,0, image.width, image.height);

}

The getPixelColor() function returns an array containing the red, green, blue and alpha components of the color at the specified location in the image. The color components have a range of 0 – 255.

function getPixelColor(x, y, imageData) {
  var index=(x * 4) * imageData.width + (y * 4);
  var colArray = [];

  colArray[0] = imageData.data[index];
  colArray[1] = imageData.data[index+1];
  colArray[2] = imageData.data[index+2];
  colArray[3] = imageData.data[index+3];
  return colArray;
}

Now in the mouseMove() event handler, we get the pixel color at our current location, and depending on what color we are over, sets the message to display.

function mouseMove(e) {
  g_mouseX = e.x;
  g_mouseY = e.y;

  var color = getPixelColor(g_mouseX, g_mouseY, g_templateImageData);

  if ((color[0] == 0) && (color[1] == 0) && (color[2] == 0)){
    g_message = "border";
  }else if ((color[0] == 255) && (color[1] == 255) && (color[2] == 255)){
    g_message = "good";
  } else {
    g_message = "crashed";
  }
}

We also need to load the image in the HTML, so that it is available in the DOM, so that the canvas object we create can access it.

This is what the image of the maze looks like.

mazetemplate.png

In the next tutorial, we will use this feature to create a landscape using a height map.

Here is the full listing for tutorial15.js

o3djs.require('o3djs.util');
o3djs.require('o3djs.math');
o3djs.require('o3djs.rendergraph');
o3djs.require('o3djs.canvas');
o3djs.require('o3djs.quaternions');
o3djs.require('o3djs.event');
o3djs.require('o3djs.io');

// Events
// Run the init() function once the page has finished loading.
// Run the uninit() function when the page has is unloaded.
window.onload = init;
window.onunload = uninit;

// global variables
var g_o3dElement;
var g_o3d;
var g_math;
var g_client;
var g_pack;
var g_textCanvas;
var g_mazeCanvas;
var g_paint;
var g_canvasLib;
var g_mazeRoot;
var g_viewInfo;
var g_mazeViewInfo;
var g_mazeTexture;
var g_camera = {
  eye: [0, 0, 10],
  target: [0, 0, 0]
};
var g_mouseX = 0;
var g_mouseY = 0;
var g_mazeImage;
var g_templateCanvas;
var g_templateCanvasContext;
var g_templateImageData;
var g_message = '';

//Event handler for the mousemove event
function mouseMove(e) {
  g_mouseX = e.x;
  g_mouseY = e.y;

  var color = getPixelColor(g_mouseX, g_mouseY, g_templateImageData);

  if ((color[0] == 0) && (color[1] == 0) && (color[2] == 0)){
    g_message = "border";
  }else if ((color[0] == 255) && (color[1] == 255) && (color[2] == 255)){
    g_message = "good";
  } else {
    g_message = "crashed";
  }
}

function drawText(str) {
  g_textCanvas.canvas.clear([0.5, 0.5, 0.5, 0.5]);

  // Reuse the global paint object
  var paint = g_paint;
  paint.color = [1, 1, 1, 1];
  paint.textSize = 12;
  paint.textTypeface = 'Comic Sans MS';
  paint.textAlign = g_o3d.CanvasPaint.LEFT;
  paint.shader = null;
  g_textCanvas.canvas.drawText(str, 10, 30, paint);

  g_textCanvas.updateTexture();
}         

/**
 * This method gets called every time O3D renders a frame.  Here's
 * where we update the cube's transform to make it spin.
 * @param {o3d.RenderEvent} renderEvent The render event object that
 * gives us the elapsed time since the last time a frame was rendered.
 */
function renderCallback(renderEvent) {
  drawText("(" + g_mouseX  + "," + g_mouseY + ") - " + g_message);
}

function getPixelColor(x, y, imageData) {
  var index=(x * 4) * imageData.width + (y * 4);
  var colArray = [];

  colArray[0] = imageData.data[index];
  colArray[1] = imageData.data[index+1];
  colArray[2] = imageData.data[index+2];
  colArray[3] = imageData.data[index+3];
  return colArray;
}
function loadImageData(image)
{
  g_templateCanvas=document.createElement("canvas");
  g_templateCanvasContext=g_templateCanvas.getContext("2d");

  g_templateCanvas.width= image.width;
  g_templateCanvas.height=image.height;
  g_templateCanvasContext.drawImage(image,0,0);

  g_templateImageData = g_templateCanvasContext.getImageData(0,0, image.width, image.height);

}

/**
 * Creates the client area.
 */
function init() {
  o3djs.util.makeClients(initStep2);
}

/**
 * Initializes O3D.
 * @param {Array} clientElements Array of o3d object elements.
 */
function initStep2(clientElements) {
  // Initializes global variables and libraries.
  g_o3dElement = clientElements[0];
  g_client = g_o3dElement.client;
  g_o3d = g_o3dElement.o3d;
  g_math = o3djs.math;
  g_quaternions = o3djs.quaternions;

  // Initialize O3D sample libraries.
  o3djs.base.init(g_o3dElement);

  // Create a pack to manage the objects created.
  g_pack = g_client.createPack();

  g_mazeRoot = g_pack.createObject('Transform');

  g_mazeViewInfo = o3djs.rendergraph.createBasicView(
       g_pack,
       g_mazeRoot,
       g_client.renderGraphRoot);

  //Set up the 2d orthographic view
  g_mazeViewInfo.drawContext.projection = g_math.matrix4.orthographic(
      0 + 0.5,
      g_client.width + 0.5,
      g_client.height + 0.5,
      0 + 0.5,
      0.001,
      1000);

  g_mazeViewInfo.drawContext.view = g_math.matrix4.lookAt(
      [0, 0, 1],   // eye
      [0, 0, 0],   // target
      [0, 1, 0]);  // up                                                     

  // Create the global paint object that's used by draw operations.
  g_paint = g_pack.createObject('CanvasPaint');

  // Creates an instance of the canvas utilities library.
  g_canvasLib = o3djs.canvas.create(g_pack, g_mazeRoot, g_mazeViewInfo);

  g_mazeCanvas = g_canvasLib.createXYQuad(0, 0, -1, g_client.width, g_client.height, true);

  g_mazeCanvas.canvas.clear([0, 0, 0, 1]);

  o3djs.io.loadTexture(g_pack, 'maze.png', function(texture, exception) {
    if (exception) {
      alert(exception);
    } else {
      if (g_mazeTexture) {
        g_pack.removeObject(g_mazeTexture);
      }

      g_mazeTexture = texture;
      g_mazeCanvas.canvas.drawBitmap(g_mazeTexture, 0, g_client.height);
      g_mazeCanvas.updateTexture();
    }
  });  

  // Create a canvas that will be used to display the text.
  g_textCanvas = g_canvasLib.createXYQuad(0, 0, 0, 200, 50, true);

  // Set our render callback for animation.
  // This sets a function to be executed every time frame is rendered.
  g_client.setRenderCallback(renderCallback);

  //Set up a callback to interpret keypresses
  window.document.onkeypress = keyPressedCallback;

  //Set up mouse events
  o3djs.event.addEventListener(g_o3dElement, 'mousemove', mouseMove);
}

/**
 * Removes callbacks so they aren't called after the page has unloaded.
 */
function uninit() {
  if (g_client) {
    g_client.cleanup();
  }
}

And tutorial15.html

<html>
   <head>
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <title>O3D Tutorial 15: Getting the color of a pixel in an image</title>
      <script type="text/javascript" src="o3djs/base.js"></script>
      <script type="text/javascript" src="tutorial15/tutorial15.js"></script>
   </head>
   <body>
      <h1>O3D Tutorial 15: Getting the color of a pixel in an image</h1>
      Keep the cursor on the track. This is a test to show how we can get the pixel color and act accordingly
      <br/>
      <div id="o3d" style="width: 500px; height: 400px;"></div>
      <img src="maze/mazetemplate.png" onload="loadImageData(this);" />
    </div>
  </body>
</html>

Javascript, O3D, Tutorials

Comments

Hendrik Groeneveld

September 2nd, 2009 at 8:58 PM

Serge,

Thanks so much for the great tutorial!

Just an FYI for you and your readers: This tutorial will break on Firefox if you try to load it from file://. You’ll get a security exception in getImageData() because Firefox considers all file:// references to be separate DOMs. You can install a local web server (which is a pain), or you can go to about:config (the hidden prefs page) and turn off security.fileuri.strict_origin_policy.
Manu

October 1st, 2009 at 9:35 AM

Hi Serge,

first of all, thank you again for your tutorials.

I suppose Hendrik is right about this tutorial.
I have tried it and everything loads ok. Well i had to do some changes. I put your image in the same folder as the tutorial15. and i change it’s name to maze.png, i could have change it’s src on the code but it’s the same, isn’t it?
after evrything is loaded i move arround the mouse but no messages at all, just a (0,0)- text that doesn’t change.
arghhhhhhhhhhh

what to do?

manu
Serge Meunier

October 1st, 2009 at 12:52 PM

That is quite strange. Check in the error console (if you are running Firefox) to see if any errors are being generated, and also make sure that the window focus.

Have a look, and let me know if you find anything.
Manu

October 1st, 2009 at 5:38 PM

Hi Serge,
I checked with the error console and it gives me this:

Erreur : uncaught exception: [Exception... "Security error" code: "1000" nsresult: "0x805303e8 (NS_ERROR_DOM_SECURITY_ERR)" location: "file:///D:/manuel2/Projet_o3D/data_samples/examples/samples/tutorial15/tutorial15.js Line: 101"]

line 101 is

g_templateImageData = g_templateCanvasContext.getImageData(0,0, image.width, image.height);

and

Erreur : keyPressedCallback is not defined
Fichier Source : file:///D:/manuel2/Projet_o3D/data_samples/examples/samples/tutorial15/tutorial15.js
Ligne : 183

line 183:

window.document.onkeypress = keyPressedCallback;

so i think Hendrik has a point
i’m testing your tutorials at the office so i don’t know if i can try hendrik advise but i’ll check this at home or maybe i’ll put in a ftp to check it out.

again thank you a 1000 times for your tutos,
i was using wirefusion to create 3d content imported from blender in .x3d format, but it’s limited and fps are just not there (for a big thing). this is just great and it gives me the chance to learn and improve my small coding skills.
merci beaucoup
manu
Serge Meunier

October 2nd, 2009 at 8:55 AM

Hi Manu,

I think I know what your problem is. Firefox has security issues with running some code using the file:// protocol (which happens when loading a file directing in the web browser). To bypass this, put the php file onto a webserver (either local or on the net somewhere – that does not matter), and then load the file using “http://yourserver.whatever/tutorial15.php”. If it is running on your local machine, “http://localhost/tutorial15.php” should work well too.

Hope that helps
seo salt lake city

August 8th, 2013 at 1:21 AM

For more of what you need to know see this site
http://seoforutah.blogspot.com

August 8th, 2013 at 1:38 AM

I look forward to sharing these advice
seo for utah

August 8th, 2013 at 2:30 AM

This was a helpful post

Smoky Cogs