If you have any suggestions or requests, we invite you to add comments below.

• 0.6
  • Detection of 3D objects under an iOS touch (Update: released as part of 0.5.1 on 2011-02-24. Touch positioning released as part of 0.5.3 on 2011-04-05).
  • Retina support (Update: released as part of 0.5.2 on 2011-03-13).
  • 3D particle system
    • Point-particles to be released in cocos3d 0.6.2 (mid-October 2011).
    • Mesh particles to be included in future 0.6.x release.
  • cocos2d 2D particle systems attached to CC3Billboards (can use Particle Designer)
  • CC3TargettingNodes as 3D halo objects – 3D objects that always face camera, but can be obscured by other objects (different from CC3Billboards, which always overlay all 3D objects).
  • Add any node to a CC3TargettingNode to have the child node always face a target node.
  • Multi-textuing.
  • Lighting enhancements, including attenuation and spotlights.
  • Fog effect.
  • Released Aug-Oct 2011.

• 0.7
  • Vertex skinning and bone rigging (Update: released as part of 0.6.3 on 2011-11-12).
  • Shadowing.
  • 3D Mesh-particles (point-particles were released in 0.6.2).
  • Identify location of touch event on an object.
  • Simple collision detection.
  • Expected delivery: Dec 2011.

• 0.8
  • Additional 3D model file loaders.
  • MacOS support.
  • Expected delivery: Q1/2012.

• 0.9
  • 3D physics engine. The design intention is to build a pluggable physics engine architecture that would provide the application developer with choices as to the actual physics library used, including using a third-party library, such as Bullet or Newton, or a simpler, bespoke library developed for the application. The initial release is expected to include the Bullet library.
  • Expected delivery: Q2/2012.

• 2.x
  • Support for OpenGLES 2.0.
  • Expected delivery: 2012.

• TBD
  • Further 3D model file loaders.
  • Scripting.
  • Delivery to be prioritized and scheduled based on developer-community needs.

cocos3d is a significant extension to cocos2d, a popular, well-designed framework for building iOS games and applications that play out in 2D (or 2.5D isometric projection). Although on the one hand it is possible to start with the cocos3d Application template and develop a 3D application without knowing too much about the workings of cocos2d, to get the most out of this document, you should familiarize yourself with cocos2d. You can learn more about cocos2d at the cocos2d Wiki.

CC3Layer

CC3Layer is one of two key classes that each application will subclass and customize. The other class you will subclass and customize in your application is CC3World, described below.

Although cocos3d is a full 3D modeling and rendering engine, all 3D rendering occurs within CC3Layer, a special cocos2d CCLayer subclass. Since it is a type of CCLayer, instances of CC3Layer fit seamlessly into the cocos2d CCNode hierarchy, allowing 2D nodes such as controls, labels, and health bars to be drawn under, over, or beside 3D model objects. With this design, 2D objects, 3D objects, and sound can interact with each other to create a rich, synchronized audio-visual experience. CC3Layer acts as the bridge between the 2D and 3D worlds.

You can add a CC3Layer instance anywhere in the visual node hierarchy of a cocos2d application. Although for the purposes of most games, the CC3Layer instance will usually be sized to cover the complete screen, a CC3Layer instance can be set to any size and added to a 2D parent visual CCNode. So, your application could have a 2D scene that contains a smaller 3D scene embedded into it via a small CC3Layer instance attached to a parent 2D CCLayer.

Conversely, since CC3Layer is a type of CCNode, you can add child CCNodes to it as well, mixing 2D nodes on top or below the 3D scene that is playing out within the CC3Layer. Generally, this is the way most applications will combine 2D and 3D components, with a main 3D scene overlaid with 2D controls such as joysticks, fire buttons, dashboards, etc. Any CCNode that is added to the CC3Layer using the standard addChild: method will appear overlaid on top of the 3D scene. You can also add 2D CCNodes behind the 3D action as well, by using the addChild:z: method, with a negative Z-order. Although this is generally not a common requirement, it can be used to provide a 2D skybox behind the 3D action.

When customizing your application’s subclass of CC3Layer, you will typically override the following two template methods:

• initializeControls – this method is where you can add your 2D controls into the CC3Layer, and otherwise generally initialize your layer. This method is invoked automatically from any of the init methods of the layer.
• update: – if you have scheduled regular updates using the standard CCNode scheduleUpdate method, this update: method will be called periodically. This is where you can update any of the 2D controls you’ve added, or pass data from dynamic controls such as sliders or joysticks to your CC3World instance. If you override this method, be sure to call invoke the superclass implementation so it can pass the update notice to the CC3World instance.

CC3World

Although CC3Layer forms the bridge between the 2D and 3D visual worlds, it remains primarily a cocos2D layer, and no 3D activity actually takes place within CC3Layer. All 3D activity, from model management to rendering, takes place within, and is the responsibility of, your application’s customized CC3World subclass.

As with CC3Layer, there are several key template methods that you will typically override in your application’s subclass of CC3World:

• initializeWorld – This method is where you populate the 3D models of your world. This can be accomplished through a combination of instantiting model objects directly and loading them from model data files exported from a 3D editor. The CC3World instance forms the base of a structural tree of nodes. Model objects are added as nodes to this root node instance using the addChild: method.
• updateBeforeTransform: and updateAfterTransform: – If you have scheduled regular updates through CCLayer, these CC3World methods will be invoked automatically as part of the scheduled update, respectively before and after the transformation matrices of the nodes of the world have been recalculated, respectively, and before and after the same methods are invoked on the descendants of your CC3World.
• nodeSelected:byTouchEvent:at: – If your application is configured to support allowing the user to select 3D nodes using touch events, this template callback method will automatically be invoked when a touch event occurs. See the section on touch events for more information about handling the selection of 3D nodes by user touch events.

As the names imply, the difference between the two update methods is that updateBeforeTransform: is invoked before the transformation matrix of the node is recalculated, whereas the updateAfterTransform: is invoked after. Therefore, if you want to move, rotate or scale a node, you should do so in the udpateBeforeTransform: method, to have your changes automatically applied to the transformation matrix of the node.

However, the global transform properties of each node (globalLocation, globalRotation, and globalScale) are determined as part of the calculation of the transformation matrix of that node. Therefore, if you want to make use of the current global properties, you should do so in the updateAfterTransform: method. The global properties of a node can be used to test for collisions, or end conditions of movements, etc.

Sometimes, you may find the need to change the location, rotation, or scale properties of a node in the updateAfterTransform: method, for example as part of collision detection and reaction. If you do, you should invoke the updateTransformMatrices method on the top-most node that is affected, to have those changes immediately applied to the transformation matrix.

You do not need to do anything in either update methods for 3D objects that are acting predictably, such as those on trajectories, or those controlled by CCActions. Their behaviour is handled by the nodes and actions themselves.

In addition to these main template methods, you may find the following CC3World methods useful during operation, or simply useful to understand:

• addChild: – Familiar from cocos2d, you can add child nodes (CC3Nodes) to your 3D world.
• addContentFromPODResourceFile: & addContentFromPODFile: -  These are convenience methods for loading POD files directly into the CC3World, and are added by the PVRPOD category if your application uses the module for loading 3D models from POD formatted files.
• getNodeNamed: – Retrieves the node with the specified name that was previously added to the 3D world. Useful for grabbing hold of a node that was added as part of a file load.
• activeCamera – Property returns (or sets) the 3D camera that is viewing the objects in the 3D world. You do not need to set this property. It will be automatically set to the first camera node added via one of the add... methods (even if the camera is buried deep within a loaded node hierarchy). However, if you have multiple cameras, you can flip between them by setting this property.
• ambientLight – This is the color of the ambient light of the 3D world. This is independent of any distinct lights that are added as child nodes.
• createGLBuffers – This method causes all contained vertex data held by contained mesh nodes to be buffered to Vertex Buffer Objects (VBO’s) in the GL engine and usually into hardware buffers accessible to the GPU. This is an optional step, but highly recommended for improving performance. You can also invoke this method at any level in the node structural hierarchy if for some reason you want to load some, but not all, mesh data in the 3D world into VBO’s.
• releaseRedundantData – After the createGLBuffer method has been invoked, this method can be used to release the data in main memory that is now redundant for meshes that have been buffered to the GL engine. You can also invoke this method at any level in the node structural hierarchy if for some reason you want to release some, but not all, mesh data from main memory. You can also exempt individual vertex arrays from releasing data by setting the shouldReleaseRedundantData property to NO on the individual vertex array.
• updateTransformMatrices – If you make changes to the location, rotation and scale properties of a node within the updateAfterTransform: method, those changes will not automatically be applied to the transformation matrix of the node, since it has already been calculated when the udpateAfterTransform: method is invoked. However, you can use the udpateTransformMatrices to have those changes immediately applied to the transformation matrices of a node and all its descendants.
• play & pause – these enable the operation of the update: method. When the CC3World is paused, the updateBeforeTransform: and updateAfterTransform: methods are skipped, along with updates of all other 3D nodes.
• cleanCaches – automatically invoked during low memory conditions within the application. This gives you the chance to dump any unneeded resources, for example, 3D objects that are far away, not part of this scene, etc.

Many of the responsibilities handled by CC3World are divided by the implementation into private template methods that you can override in your subclass, to customize behaviour in a modular way.

CC3Node

All the objects in the 3D world, including models, cameras, lights, and the CC3World itself, are known as nodes. CC3Node is the base of the 3D node class hierarchy. Nodes can be assembled into structural assemblies using parent/child relationships. Moving, rotating, or hiding a node moves, rotates or hides all the children (and other ancestors) in concert.

This design will no doubt feel familiar to you as being analogous to CCNodes in cocos2d, and the two node hierarchies do follow the same design pattern. However, CCNodes and CC3Nodes cannot be mixed in the same structural assembly, primarily because CC3Nodes must keep track of location, rotation and scaling in three dimensions instead of two. Nevertheless, the structural concepts between the two node families are consistent.

You assemble 3D nodes using the addChild: method. All nodes have an identifying tag and can have a name. You can retrieve a specified node in an assembly with the getNodeNamed: and getNodeTagged: methods.

All nodes have location, rotation, and scale properties (plus a couple of others). You move, rotate, and scale CC3Nodes by setting these properties. And, again, as with cocos2d nodes, the values of these properties are measured relative to the node’s parent node.

You can override the updateBeforeTransform: and updateAfterTransform: methods of a node subclass that acts predictively, such as those following a trajectory, to update these transform properties. See the discussion on CC3World above regarding the difference between these two methods.

So, the wheels of a car can be child nodes on a car node. Each wheel node can rotate on its axis, and move up and down relative to the car as if on suspension via the location property of each wheel node, and all the while, the whole car assembly may be travelling and bouncing down a dirt road simply by manipulating the location property of the car node.

Any node can be the target of a CCAction, to simply and easily control the movement and behaviour of the node in sophisticated ways. In addition, any node may be configured to respond to finger touch events by the user. See the sections on actions and touch events for more information on this interactivity.

You can use the createGLBuffers method to cause all contained vertex data held by contained mesh nodes to be buffered to VBO’s in the GL engine. Generally, you should invoke this method at the highest level in the structure, CC3World, but you can also invoke this method at any level in the node structural hierarchy if for some reason you want to load some, but not all, mesh data in the 3D world into VBO’s. Once data is loaded into GL VBO’s, you can use the releaseRedundantData method to release the data from your main application memory.

CC3MeshNode

CC3MeshNode puts the ’3′ in 3D. An instance of CC3MeshNode contains the 3D mesh data for a 3D object, plus the material, texture, or solid color that covers the surface of the object.

Each CC3MeshNode instance can be covered in only a single material, texture, or color. However, like any node, mesh nodes can be assembled into a node structure. Moving the parent node will move all of the child mesh nodes in concert, along with their materials and textures. So, a multi-colored beach-ball could be one parent node and several child mesh nodes, each corresponding to a differently colored panel on the beach-ball. The parent ball node can be moved, rotated and scaled, and all component mesh nodes will be affected in unison.

Materials, Textures and Colors

The visible characteristics of the surface of a mesh node is determined by either the material it is covered with, or the pure, solid color it is painted with. As mentioned above, each mesh node can be covered with only one material, or one solid color.

The mesh node holds an instance of a CC3Material to describe the visual characteristics of the surface of the mesh. CC3Material includes properties to set the various coloring characteristics of the surface, including the ambient, diffuse and specular reflective colors, the emissive color, and the surface shininess. It also includes properties to determine how the colors should blend with the colors from objects behind the mesh node, permitting effects such as translucency.

In addition to basic coloring, each CC3Material instance can hold an instance of CC3Texture to cover the surface of the mesh with a texture image.

The combination of coloring properties, blending properties, and textures interact with lighting conditions to create complex and realistic surface visual characteristics for the mesh.

There are two mechanisms for changing the coloring and opacity of a material covering a mesh node.

• To achieve the highest level of detail, accuracy and realism, you can individually set the explicit ambientColor, diffuseColor, specularColor, emissiveColor, shininess, sourceBlend, and destinationBlend properties. This suite of properties gives you the most complete control over the appearance of the material and its interaction with lighting conditions and the colors of the objects behind it, allowing you to generate rich visual effects.
• At a simpler level, CC3Material also supports the cocos2d <CCRGBAProtocol> protocol. You can use the color and opacity properties of this protocol to set the most commonly used coloring and blending characteristics simply and easily. Setting the color property changes both the ambient and diffuse colors of the material in tandem. Setting the opacity property also automatically sets the source and destination blend functions to appropriate values for the opacity level. By using the color and opacity properties, you will not be able to achieve the complexity and realism that you can by using the more detailed properties, but you can achieve good effect with much less effort. And by supporting the <CCRGBAProtocol> protocol, the coloring and translucency of nodes with materials can be changed using standard cocos2d CCTint and CCFade actions, making it easier for you to add dynamic coloring effects to your nodes.

If the mesh node does not contain a material, it will be painted with the color defined in the pureColor property. This color is painted as is, pure and solid, and is not affected by the lighting conditions. In most cases, this looks artificial, and is not recommended for realistic scene coloring. But in some circumstances, such as cartoon effects, it may be useful.

CC3MeshModel

Underpinning CC3MeshNode is CC3MeshModel, which in turn holds the raw vertex data in several CC3VertexArray instances. The management of mesh data is spread across these three classes (CC3MeshNode, CC3MeshModel, and CC3VertexArray) to enable data reuse and reduce memory requirements. A single mesh model instance can be used in any number of individual mesh nodes, each covered with a different material, and placed in a different location. Whether you’re talking about hordes of zombies, or a twelve-place dinner setting laid out on a table, a single mesh model instance, with one copy of the raw vertex data can be reused in an number of similar nodes.

CC3VertexArrays

Each mesh model instance (typically an instance of the concrete subclass CC3VertexArrayMeshModel) holds several CC3VertexArray instances, one for each type of vertex data, such as vertex locations, normals, vertex colors, texture coordinate mapping, and vertex indices. And reuse can be applied at this level as well. A single vertex array instance can be attached to many mesh models. So, if you have the need for two teapot meshes, one textured, and one painted a solid color, you can use two separate instances of CC3VertexArrayMeshModel, each containing the same instances of the vertex locations and vertex normals arrays (CC3VertexLocations and CC3VertexNormals, respectively), but only the mesh model for the textured teapot would also contain a texture coordinate vertex array (CC3VertexTextureCoordinates) instance. With this arrangement, there is only ever one copy of the underlying vertex data.

CC3LineNode & CC3PlaneNode

CC3LineNode and CC3PlaneNode are specialized subclasses of CC3MeshNode that simplify the creation and drawing of lines and planes using vertex arrays, respectively.

Duplicating 3D Models

In order to allow you to create hordes of invading armies, the CC3Node class supports the <NSCopying> protocol. To duplicate a node is simply a matter of invoking the copy method on that node. In addition, there is a copyWithName: method that duplicates the node, and gives the new copy its own name.

Copying a CC3Node creates a deep copy. This means that not only is the node itself copied, but copies are created of most components of the node, including the material, and any descendant child nodes. This allows the properties and structure of the duplicate node to be changed separately from those of the original node. For instance, the new node can be positioned, rotated, scaled, colored, or assigned a different texture from that of the original. Similarly, the child nodes of the duplicate may be modified separately, without affecting the original, or any other copies. You can copy an automobile node, and remove one of the wheels of the duplicate, while retaining all four wheels on the original. If a deep copy were not performed, changing parameters in the duplicate node, or its children, would result in the same changes appearing in all other copies of the original node.

There is one big exception to this deep copying. Since mesh data is for the most part static, and is designed to be shared between instances of CC3MeshNode, mesh data is not copied, but is automatically shared between the original node and all its copies. Specifically, when an instance of CC3MeshNode is copied, a copy is made of the encapsulated CC3Material instance and is retained by the duplicate node, but the encapsulated CC3MeshModel is not duplicated. Instead, the single CC3MeshModel instance is retained by both the original node and the duplicate, and is henceforth shared by both nodes. This shallow-copying of the mesh data ensures that only one copy of the mesh data appears in the device memory.

The following additional rules apply to duplicating a CC3Node:

• The tag property is not copied. The duplicate node is assigned a new unique value for its tag property. This is to ensure that the tag property is unique across all nodes, including duplicates, and allows you to identify a duplicate as distinct from the original, even if the name property is left the same.
• The duplicate will initially have no parent. That will automatically be set when the duplicate node is added as a child to a parent node somewhere in the world. This applies to the specific node on which the copy or copyWithName: method was invoked. Any descendants of this node will be assigned to their parents after they are copied, so that the overall structure of the node assembly below the invoking node will be replicated in its entirety.
• Like mesh data, underlying textures are not duplicated (although the CC3Texture instance itself is). Nor is node animation data duplicated.

If you create your own subclass of CC3Node, or any of its existing subclasses, and your new subclass adds state, you must implement the populateFrom: method, including invoking the same superclass method as part of it, to ensure that state is transferred correctly from the original node to the duplicate, whenever an instance of your node class is duplicated.

CC3Camera & CC3Light

These are special nodes that represent the camera and lights in the 3D world. Cameras and lights are directional, and are examples of CC3TargettingNode, which, in addition to being able to move and rotate like other nodes, can be assigned a target or targetLocation at which to point. A target can be any other CC3Node in the scene, and the camera or light can be told to track the target as it moves.

Controlling CC3Nodes with CCActions

If you’ve used cocos2d, you’re no doubt familiar with the family of CCActions, used to control the movement, coloring, visibility, and activities of CCNodes.

In cocos3d, CC3Nodes can similarly be manipulated and controlled by CCActions. Some features, such as tinting and fading can manipulated by standard cocos2d actions. However, because the 3D coordinate system is different than the 2D coordinate system, specialized 3D versions of movement, rotation, scaling, and animation actions are required. This family of 3D actions can be found as subclasses of the base CC3TransformTo interval action. In addition, there are several cocos3d action subclasses that handle material-tinting.

The 3D family of actions can, of course, be used with standard cocos2d combination actions such as the family of ease actions, sequence actions, etc.

CC3Node Animation

Any CC3Node can be animated with key-frame animation data held in a CC3NodeAnimation instance within the CC3Node instance. Typically, this animation data is loaded from the 3D model files exported from your 3D editor, but you can also assemble this data programmatically using the simple CC3ArrayNodeAnimation subclass of CC3NodeAnimation. Like mesh data, the CC3NodeAnimation instances are stateless, and each may be shared by multiple CC3Node instances, so that animation data need not be duplicated redundantly.

Any or all of the transformation properties: location, rotation, and scale, may be animated using CC3NodeAnimation or CC3ArrayNodeAnimation. If you need to animate other characteristics of your nodes, you can subclass these classes to add additional animation capabilities.

Once animation data is associated with a CC3Node, the node can be animated with repeated invocations of the establishAnimationFrameAt: method on the node itself, passing in a frame time, which is a floating point value between zero and one, with zero representing the first animation frame, and one representing the last animation frame.

In order to perform smooth animation, by default, the CC3NodeAnimation will interpolate animation data between frames if the timing value passed in through the establishAnimationFrameAt: method is between actual frame times. This is controlled by the shouldInterpolate property on the CC3NodeAnimation instance. By default, this property is set to YES, but may be set to NO if heavy interpolation interferes with frame rates.

For node assemblies, you will typically want to animate the whole assembly in concert. To support this, the establishAnimationFrameAt: method automatically invokes the same method on each child node, passing the same animation frame time to each child node. Thus, animating a character, by invoking that method on the character node itself, will automatically forward the same invocation to each component child node of the character, such as the character’s limbs, or any weapon the character might be holding. As a result, all components of the character will move in sync with the character, as it is animated.

However, you can turn animation of any node in the assembly off, including selectively turning off animation of specific child nodes, via the disableAnimation method on that child node. You can turn the animation of whole sub-assemblies of nodes off via the disableAllAnimation method.

Fractional animation of individual movements is possible by simply limiting the range of frame times that are sent with the establishAnimationFrameAt: method. For example, if your character animation includes a run frame sequence and a jump frame sequence, you can invoke one or the other by simply starting and ending your animation at the frame times associated with the specific movement you want to invoke.

Controlling Animation with Actions

Although you can arrange to invoke the establishAnimationFrameAt: method of your animated CC3Node directly, in most cases it is much easier to use an instance of CC3Animate to control the animation of a node.

CC3Animate is a type of CCActionInterval, and will run through the animation frames automatically over a configurable time duration. Moreover, when instantiating the CC3Animate, you can restrict the frames to a particular range, to allow fractional animation of specific movements.

Selecting CC3Nodes with User Touch Events

To allow the user to interact with the objects in your 3D world, you can enable the selection of 3D objects using standard iOS finger touch events.

The first step in enabling touch activity is to set the isTouchEnabled property of the CC3Layer to YES. Typically you will do this in the initializeControls method of your customized CC3Layer class. This causes the CC3Layer to register itself to receive touch events from iOS.

As such, your CC3Layer will receive and process kCCTouchBegan, kCCTouchEnded, and kCCTouchCancelled events. By default, your CC3Layer will not receive or process kCCTouchMoved events, because these are both quite voluminous and seldom used. However, you can configure your customized CC3Layer subclass to receive and handle kCCTouchMoved events by copying the commented-out ccTouchMoved:withEvent: method from the CC3Layer implementation and paste it into the implementation of your customized CC3Layer subclass, with the commenting removed.

The second step is to set the isTouchEnabled property to YES, in any CC3Nodes that you want the user to be able to select with a touch event. So, for example, let’s say your 3D world had a bowl of fruit on a table, you might want the user to be able to select a piece of fruit, but not the bowl or the table objects. In this case, you would set the isTouchEnabled property to YES on the CC3Node instances representing each piece of fruit, but not on the CC3Node instances that represent the bowl or table. Please also note that, to be touchable, the node must have its visible property set to YES.

Once these two steps have been completed, the nodeSelected:byTouchEvent:at: callback method of your customized CC3World subclass will automatically be invoked on each touch event. This callback includes the CC3Node instance that was touched, the type of touch, and the 2D location of the touch point, in the local coordinates of the CC3Layer.

If the touch occurred at a point under which there is no touchable CC3Node, the callback nodeSelected:byTouchEvent:at: method will still be invoked, but the node will be nil, to indicate that a touch event occurred, but not on a touchable node.

For node assemblies, the node passed to the nodeSelected:byTouchEvent:at: method will not necessarily be the individual component or leaf node that was touched. Instead, it will be the closest structural ancestor of the leaf node that has its isTouchEnabled property set to YES.

For example, if the node representing a wheel of a car is touched, it may be more desireable to identify the car as being the object of interest to be selected, instead of the wheel. In this case, setting the isTouchEnabled property to YES on the car, but leaving it as NO on each wheel, will allow a wheel to be touched, but the node received by the nodeSelected:byTouchEvent:at: callback will be the node that represents the car as a whole.

Selection Artifacts

When using translucent nodes, you might notice that when a translucent node is displayed over the raw background color of the CC3Layer, it will appear to flicker slightly when a touch event occurs. This is a side effect of the mechanism used to identify the 3D node that is under the 2D touch point.

The selection mechanism uses a color-picking algorithm, which momentarily paints each node with a unique color and then reads the color at the touch point to identify the node. The scene is then immediately drawn a second time with proper coloring. Since this second scene rendering occurs within the same rendering frame as the first, the second rendering completely overwrites the first, and the user is completely unaware than the first pass occurred.

The only exception to this is when a translucent node has no opaque nodes behind it. In this case, because the underlying layer background color is not redrawn on the second rendering pass, some of the solid coloring used to paint the node during the selection rendering pass will “leak through” the transparency of the second rendering pass. In effect, for one frame, the translucent node does not have the layer background color behind it, causing a momentary flicker.

It is important to realize that this effect does not occur when the translucent node has opaque nodes behind it in the 3D scene. This is because the opaque background nodes will be redrawn as well, and it will be these background nodes that will be seen through the translucent node, as they should be.

Therefore, if you have translucent nodes and are using touch selection, to avoid this slight flicker be sure to include an opaque skybox node at the back of your 3D scene, over which the other nodes of your scene will be drawn.

Projecting 3D Locations to 2D

The projectedLocation property of CC3Node holds the location of the 3D node in the 2D coordinate system of the window. It acts as a bridge between the 3D coordinate system and the 2D coordinate system. Knowing the projectedLocation of a 3D node allows you to relate it to a 2D controls such as a targetting reticle, or to a touch event.

The projectedLocation is a 3D vector. As you would expect, the X- and Y-components provide the location of the 3D node on the screen, in the 2D coordinate system of the CC3Layer. If the 3D node is located somewhere out of the view of the camera, and therefore not displayable within the layer, these values will be outside the range given by the contentSize of the CC3Layer. Following from this, either coordinate value may be negative, indicating that the node is located to the left, or below, the view of the camera.

The Z-component of the projectedLocation contains the straight-line distance between the 3D node and the camera, as measured in the coordinates of your 3D world. This value may be negative, indicating that the 3D node is actually behind the camera. In most cases, of course, you will be interested in nodes that are in front of the camera, and the Z-component of the projectedLocation can help you identify that.

CC3Node also supports the related projectedPosition property. It is derived from the projectedLocation property and, for the most part, contains the same X- and Y- coordinates as projectedLocation, but as a 2D CGPoint, which is immediately usable by the cocos2d framework.

However, as a 2D point, the projectedPosition lacks the needed ability to distinguish whether the node is in front of, or behind, the camera. Since this information is almost always needed, the point is encoded so that, if the 3D node is actually behind the plane of the camera, both the X- and Y-components of the projectedPosition will contain the large negative value -CGFLOAT_MAX. If you use projectedPosition, you can test for this value to determine whether the 3D node is in front of, or behind, the camera.

For most nodes, these properties are not calculated automatically. In the update: method of your CC3World you can have them calculated for a node of interest by passing the node to the projectNode: method of the activeCamera.

However, for CC3Billboard nodes, the projectedLocation and projectedPosition properties are calculated automatically on each update. A CC3Billboard is a 3D node that can hold an instance of a 2D cocos2d CCNode, and display that 2D node at the projectedPosition of the CC3Billboard node. Since the 2D node is drawn in 2D, it always appears to face the camera, and is always drawn over all 3D content. The contained CCNode can be any cocos2d node, and it can be configured to automatically scale to the correct perspective sizing, shrinking as the CC3Billboard moves away from the camera, and growing as the CC3Billboard approaches the camera. A common use of CC3Billboard is to display information about a 3D node, such as a textual name label or a health-bar for a game character.

Pluggable 3D Model Loading

Sophisticated 3D games are dependent on loading 3D models from resources created in 3D editors such as Blender, Cheetah3D, Maya, 3ds Max, and Cinema 4D. Since the number of possible file formats is significant, and can evolve, the loading of model data is handled by pluggable loaders and frameworks. When building your application with cocos3d, you only need to include the loaders that you use.

The pluggable loading framework is based around two template classes:

• The resource class, which takes care of loading and parsing the 3D data file, creating instances of nodes, mesh models, vertex arrays, materials, textures, cameras, lights, etc., and assembles them into a node hierarchy. This will be a subclass of CC3Resource, and is tailored to parsing and assembling a specific data file format.
• The resource node class. This will be a subclass of CC3ResourceNode, and as such, is actually a type of CC3Node. It wraps the CC3Resource instance, extracts populated nodes from it, and adds them as child nodes to itself. This node can be used like any other CC3Node instance, and is usually simply added as a child to your CC3World instance. You can move, rotate, scale, or hide all the components loaded from a file simply by manipulating the properties of the CC3ResourceNode instance.

In addition to these two template classes, a pluggable loading package will usually contain specialized subclasses of  CC3MeshNode, CC3Material, CC3Camera, etc., in order to set the properties easily from the data loaded from the file. But once created, these objects behave like any other node of their type.

A pluggable loading package may also add categories to base classes as a convenience. Thus, the PowerVR POD file loading package adds a category to CC3World that adds the method addContentFromPODResourceFile: as a convenience for loading POD files directly into your CC3World.

The initial loading package available with cocos3d is for PowerVR POD files.

Automatic Frustum Culling and Bounding Volumes

In a 3D world, the camera is always pointing in some direction, and generally, is only seeing a fraction of the objects in the 3D world. It is therefore a waste of precious time and resources to try to draw all of these objects that will not be seen.

The task of determining which objects don’t need to be drawn, and then removing them from the rendering pipeline is known as culling. There are two types of culling: frustum culling, which is the removal of objects that are not within the field of view of the camera, and occlusion culling, which is the removal of objects that are within the field of view of the camera, but can’t be seen because they are being visually blocked by other objects. For example, they might be located in another room in your the map, or might be hiding behind a large rock. At this time, cocos3d does not perform any kind of occlusion culling.

cocos3d performs frustum culling automatically. Objects that are outside the view of the camera will not be drawn. This is accomplished by specifying a bounding volume for each mesh node. Each node holds onto an instance of a CC3NodeBoundingVolume, which it delegates to when determining whether it should draw itself to the GL engine. All mesh nodes in cocos3d have a default bounding volume that is reasonably accurate, and for the most part, you won’t even need to think about bounding volumes. However, we’ll describe the operation of bounding volumes here, for those situations where the application may want to improve on the default behaviour.

The trick with bounding volumes is to strike a balance between the time it takes to determine whether a node should be rendered, and the time it would take to simply blindly render the object. To that end, different kinds of bounding volumes can be specified, some of which are very fast, but perhaps less accurate, and others that are very accurate, but time consuming.

Accuracy is important because acn inaccurate bounding volume can either result in the effort of rendering an object being performed when the objects won’t be seen, or worse, an object might be culled, when in fact at least part of it would really be seen if it were drawn. This second type of inaccuracy, over-zealous culling results in objects strangely popping in and out of existence, particularly around the edges of the camera’s field of view. Since this behaviour is generally undesirable, you should stay away from creating bounding volumes that are overly-zealous. The bounding volumes included by default in cocos3d always ensure that all vertices are included within the bounding volume, so that objects are never culled when they are actually partially visible.

The family of cocos3d bounding volumes includes a number of different subclasses, each performing a different type of boundary test. The two most commonly used are a spherical bounding volume, represented by subclasses of CC3NodeSphericalBoundingVolume, and axially-aligned-bounding-box (AABB) bounding volumes, represented by subclasses of CC3NodeBoundingBoxVolume.

Checking spherical boundaries against the camera frustum is very fast. However, for most non-spherical real-world 3D objects, such as a human character, or automobile, a sphere that completely envelopes the object is much larger than the object. The result is that near the periphery of the camera’s view, the sphere may be partially inside the frustum, but the object is not. The result is that the object will be rendered (because the sphere can be “seen” by the camera), even though the object itself will not be seen.

Bounding boxes are often more accurate for many real-world objects, but are more expensive to test against the camera frustum, because all eight points that define the volume must be tested against the frustum.

At the extreme end of the scale, the most accurate bounding volume would be to test every single vertex in the mesh against the camera’s frustum. By definition, this will result in almost perfect accuracy, but at the large cost of testing all the vertices. In many cases, since the GPU can generally do this faster than the CPU, there is no cost savings in testing all vertices. For this reason, cocos3d does not yet include a default all-vertex bounding volume test. However, such a bounding volume could easily be added and used by the application.

Bounding volumes, whether they be point, spherical, boxes, or custom, are calculated and defined automatically by the vertices in the mesh node. In addition, they scale automatically as the node is scaled.

Finally, to make use of both fast boundary tests to exclude objects that are clearly far from the camera’s field of view, and also more accurate boundary tests for objects that are at the edges of the camera’s field of view, cocos3d also includes the CC3NodeTighteningBoundingVolumeSequence bounding volume. Instances of this class hold an array of other bounding volumes, and tests the node against them in sequential order. As soon as one contained bounding volume indicates that the node is definitely outside the camera’s field of view, the node is rejected and not drawn.

The trick is to order the contained bounding volumes so that fast, but broad, tests are performed early in the sequence, so they can reject the node if it is clearly nowhere near the camera’s field of view. Subsequent bounding volumes in the sequence should be less broad, but will not be tested unless the earlier bounding volumes have accepted the node as being visible.

By default, mesh nodes in cocos3d make use of a tightening sequence that contains first a spherical bounding volume and then an AABB bounding volume. For the most part, this will likely be sufficient, and you won’t even have to think about bounding volumes. However, some applications may benefit from custom bounding volumes.

Rendering Order

The underlying OpenGL ES engine uses a pipeline of commands for manipulating the state of the rendering engine. The switching of some states can be a costly exercise. Consequently, it is often useful for the application to consider the order in which objects are rendered to the engine. For example, if several objects make use of the same texture, it is usually beneficial to draw all of these objects one after the other, before drawing an object that is covered with a different texture.

To that end, the drawing order of the objects within a CC3World can be configured by the application for optimal performance. The CC3World instance holds onto an instance of a CC3NodeSequencer, and delegates the ordering of nodes for drawing to that sequencer. If no node sequencer is supplied to the CC3World, the world will draw nodes hierarchically, following the structural hierarchy of its children. In most cases, this will not be the optimal ordering.

As with bounding volumes discussed above, there are a number of different types of node sequencers, and establishing a different ordering is simply a matter of plugging and configuring a different sequencer into your CC3World.

There are two key types of sequencers, CC3NodeArraySequencers, which hold a collection of nodes, grouped in some defined order, and CC3BTreeNodeSequencers, which hold a B-tree of other sequencers, allowing groupings of groupings of nodes, ad infinitum. You can assemble a complex sequence definition, by assembling different types of sequencers into a structural B-tree hierarchy.

Every sequencer contains in instance of a CC3NodeEvaluator subclass. A node evaluator tests a node against some criteria, and returns a boolean result indicating whether the node is accepted or rejected, effectively giving each sequencer a chance to answer the question “Do you want this node?”. Using this mechanism, a B-tree sequencer can determine which of its child sequencers wants to take care of ordering any particular node.

A simple example might help here. Both Imagination Technologies (the supplier of the iOS GPU’s) and Apple recommend that all opaque objects should be drawn before translucent objects. In addition, the translucent objects themselves should be drawn in Z-order, which is the reverse order of distance from the camera, with the farther translucent objects being drawn first, and the nearer translucent objects drawn over them.

This drawing order can be accomplished with a B-tree sequencer containing two array sequencers: the first with an evaluator that tests the node for opacity, and the second with an evaluator that tests the node for translucency. The array sequencer holding the opaque nodes can simply hold them in the order they are added. However, the array sequencer holding the translucent nodes needs to order the nodes by their Z-order.

This structure looks as follows:

• CC3BTreeNodeSequencer (CC3LocalContentNodeAcceptor)
  • CC3NodeArraySequencer (CC3OpaqueNodeAcceptor)
  • CC3NodeArrayZOrderSequencer (CC3TranslucentNodeAcceptor)

When testing a node, the B-tree sequencer will first ask the child sequencer with the opacity test if it is interested in the node, before asking the second sequencer. The result is that the first sequencer will contain opaque nodes in the order they were added, and the second will contain translucent nodes in Z-order.

Because nodes may be moving around from frame to frame, be aware that Z-ordering requires that the nodes in the CC3NodeArrayZOrderSequencer be sorted on each and every frame update. This can have significant performance implications if there are a large number of translucent nodes. To improve performance, ensure that translucent nodes really are visibly translucent (don’t waste time sorting translucent nodes that can’t be seen as translucent by the user), and keep the number of translucent nodes to a minimum.

The requirement that this example illustrates is so common that CC3BTreeNodeSequencer includes the class method sequencerLocalContentOpaqueFirst, to create just such a sequencer assembly. There are also the class methods sequencerLocalContentOpaqueFirstGroupTextures and sequencerLocalContentOpaqueFirstGroupMeshes, which take the concept a bit further and group the opaque nodes so that opaque nodes with the same texture or mesh are drawn together, one after the other, before other opaque nodes are drawn. As above, in each of these sequencers, the translucent nodes are sorted by Z-order.

By default, CC3World uses a sequencer created by the sequencerLocalContentOpaqueFirst method, thereby drawing opaque nodes first, in the order they were added, then translucent nodes in Z-order.

Updating vs Drawing

By design, both cocos2d and cocos3d support a clear separation between updating and drawing activities, and each is invoked on a separate loop. In addition to helping to keep your application code organized, there are good performance reasons for separating updating from drawing. OpenGL ES is designed as a state engine pipeline, designed for a steady stream of rendering commands, with ideally no data traveling back upstream to the application. This allows the GPU to keep its own time. Keeping the application rendering loop lean and mean, helps the GPU operate at the full frame rate, and leaves open options for scaling back the update rate separately if needed, or breaking the update loop into several passes, or possibly even multi-threading the updates, in extreme cases.

With all this in mind, you should keep the code you write in the updateBeforeTransform: and updateAfterTransform: methods of any CC3Node subclass free of drawing or rendering operations. Similarly, should you override the drawWithVisitor: or draw methods in any CC3Node subclass, you should only perform rendering operations, and keep those methods free of model updates.

OpenGL ES State Management

As mentioned above in the discussion of rendering order, the performance of the OpenGL ES state machine pipeline is slowed down by frequent state changes. And the OpenGL ES engine incurs costs when performing state machine changes that really didn’t need to happen because the engine already was in that state. To that end, the application should do its best to ensure that GL engine state changes are only made if state really is changing.

To ensure that, cocos3d shadows the state of the GL engine outside of the GL engine itself. When an attempt is made to change state, cocos3d first checks to see if the GL engine is already in that state, by checking the value of the shadow state. Only if the state is changing from that of the shadow state is it propagated to the GL engine.

This state shadow tracking is automatic, and most application developers can safely ignore it. However, some sophisticated applications may need to interact with this framework, so I’ll give an overview of it here.

In cocos3d, this shadow state is managed by a singleton instance of CC3OpenGLES11Engine. All calls to the GL engine are made through this singleton instance.

The CC3OpenGLES11Engine singleton instance contains a number of instances of subclasses of CC3OpenGLES11StateTrackerManager, each geared towards tracking a particular grouping of GL state, such as capabilities, or lighting, materials, etc. Each of these tracker managers is accessed through a property in the CC3OpenGLES11Engine singleton instance. Each tracker manager exposes individual elements of state through properties.

For shadow tracking to work, it is absolutely critical that all GL state changes are handled by the shadow tracker. So, when adding functionality to cocos3d for your application, if you need to make calls to the GL engine, be sure to do so through the proper shadow tracker! A few examples of how to make a GL call through a shadow tracker, and the GL call that it replaces are:

• [[CC3OpenGLES11Engine engine].serverCapabilities.texture2D enable] <-> glEnable(GL_TEXTURE_2D)
• [CC3OpenGLES11Engine engine].materials.ambientColor.value = kCCC4FBlue <-> glMaterialfv(GL_FRONT_AND_BACK, GL_AMBIENT, (GLfloat*)&kCCC4FBlue)
• [CC3OpenGLES11Engine engine].state.scissor.value = self.activeCamera.viewport <-> glScissor(vp.x, vp.y, vp.w, vp.h), where vp = self.activeCameraViewport

Since it is critical that all GL calls be processed through the state shadow trackers, but cocos2d does not use state shadow tracking, you may be wondering how the shadow trackers know what the value a GL state has at the time that rendering control is passed to the cocos3d framework from cocos2d.

Each primitive state tracker has an originalValueHandling property, which tells that tracker how to determine the original value. The enumerated value of this property can tell the tracker to do one of the following:

• simply ignore the current state at the beginning of each rendering frame
• read the current state at the beginning of each rendering frame
• read the current state at the beginning of the first rendering frame and assume that value always

In addition, for the last two options, the enumeration can be modified to tell the tracker to restore that original state back to the GL engine, because cocos2d is assuming it to have stayed the same.

The original value state is set to the optimal value for combining cocos2d and cocos3d, generally either ignoring the value, or reading it once on the first frame. However, if your application varies from frame to frame and you find that the initial value can change from frame to frame at the point that cocos3d takes over, you can modify the originalValueHandling property of the associated state tracker to read the value on every frame.

Keep in mind that this reading interrupts the the flow of the GL pipeline significantly, and should generally be avoided at all costs. A better solution is to ensure that your cocos2d code leaves GL state with consistent values from frame to frame at the time cocos3d takes over.

However, there are occasions where the functionality provided by UIViewController can be quite useful. In particular, the following functionality is best handled by adding a UIViewController into the mix:

• Auto-rotation – in some games, or other OpenGL ES applications, it makes sense to change the orientation of the application display to match changes in the orientation of the physical device. This means that when the user flips the device from landscape to portrait mode, or back again, the application display will adjust itself accordingly. The auto-rotation works on cocos2d nodes directly and does not rotate the underlying iOS Cocoa UIView. If you are interested in combining cocos2d nodes with UIViews and having them auto-rotate together, please review cocos2d support for that here.
• Device camera overlay – an increasingly popular use for iOS devices is to overlay a game or app view on top of the image visible through the device’s camera. This is known as augmented reality, and provides the user with a combined view of the real world overlaid with images generated by the application. For this use, a UIViewController is mandatory, since it controls the merging of the real-world and generated image streams.

Here at The Brenwill Workshop, some of our iOS applications actually use both of these features, and we’ve developed a small framework that can easily add the capabilities above to any cocos2d application. This article describes the design and use of this framework.

Examples

Since a picture is worth a thousand words, let’s start with sample screenshots from the example game, showing the automatic rotation of the game layer when the device orientation is changed, and the game layer overlaying the device camera image:

Game layer in portrait orientation with colored backdrop.

Game layer rotated to landscape orientation. The player and sunshine button have moved to their correct positions relative to the new orientation.

Game layer overlaying the device camera image.

Although it is not shown here, the two features are compatible, of course, and the device camera overlay will also rotate along with changes in the device orientation.

Notice that, in the rotation from portrait to landscape orientation, the player image on the left has moved to the vertical center of the new orientation and the sunshine button in the lower right corner has stayed on-screen and in the same position relative to that corner. The game layer subclass is automatically notified of changes in the device orientation to give it a chance to adjust the positions of its child nodes to suit the new orientation.

Quick Start

This framework is centered on two classes: a node controller, CCNodeController, and a controlled layer, ControllableCCLayer. Setting up the controller is simple, and is handled in the applicationDidFinishLaunching: method of your UIApplicationDelegate implementation, during normal cocos2d startup. Add the following code at the end of this method (replacing the existing line: [[CCDirector sharedDirector] runWithScene: [[CCScene node] addChild: myLayer]];):

GamePlayLayer* gamePlayLayer = [GamePlayLayer node];
controller = [[CCNodeController controller] retain];
controller.doesAutoRotate = YES;
controller.isOverlayingDeviceCamera = YES;
[controller runSceneOnNode: gamePlayLayer];

That’s it! You now have a cocos2d application that will rotate itself automatically as the device is rotated, and display itself over the camera view if the device has a camera.

In the first line of code above, GamePlayLayer is your subclass of ControllableCCLayer. In the second line, you create and retain the controller in a controller instance variable you add to your UIApplicationDelegate class (don’t forget to release it in the dealloc method).

In the third and forth lines you tell the controller to automatically rotate the game layers and overlay those layers on top of the device camera. No need to worry about whether the device actually has a camera. The isOverlayingDeviceCamera property can only be set to YES if the device has a camera, and it will be ignored otherwise.

In the last line, the controller sets the GamePlayLayer as its controlled node, wraps it in a CCScene, and tells CCDirector to run the scene. This last line replaces the typical [[CCDirector sharedDirector] runWithScene: [[CCScene node] addChild: myLayer]]; line normally found at the end of the applicationDidFinishLaunching: method within a cocos2d application.

CCNodeController, ControlledCCNodeProtocol, and ControllableCCLayer

Central to this framework is the CCNodeController class, which is a subclass of UIViewController and provides the bridge to iOS to receive automatic notifications of changes in the orientation of the device and to control the device camera display. When using this framework, you will need to instantiate a CCNodeController, but you will usually have no need to subclass CCNodeController, unless you have very specialized requirements.

As the name suggests, UIViewController is designed to control a UIView, also from Apple’s Cocoa UIKit framework. Although cocos2d does make use of a UIView under the covers and the CCNodeController does in fact control that UIView, practical OpenGL rendering is handled by the cocos2d CCNode hierarchy. Therefore, to make the controller useful to cocos2d, we need a bridge between the controller and the CCNode hierarchy.

Enter the ControlledCCNodeProtocol protocol, which defines the requirements that a CCNode must support for it to be controlled by a CCNodeController. These requirements are actually relatively simple, amounting to tracking the controller itself via a controller property, and handling notification of changes in the device orientation through a deviceOrientationDidChange: method.

In Apple’s UIKit framework, although a UIViewController can control any UIView subclass, in practice controllers are almost always applied to control the top-most UIView within a window scene. Similarly, although a CCNodeController can control any CCNode subclass that supports ControlledCCNodeProtocol, in practice it is the main CCLayer at the level of the scene that is of most interest.

In recognition of this, the framework here includes a ControllableCCLayer class that adds the behaviour defined by ControlledCCNodeProtocol to CCLayer. This class is subclassed from CCColorLayer and supports initialization either with or without a background color. In your application, you derive subclasses of ControllableCCLayer instead of CCLayer, to allow your layers to rotate automatically and overlay the device camera.

ControllableCCLayer automatically resizes as it rotates, and there are hooks for the subclasses to latch onto to reposition child nodes if needed. It is also smart enough to know that if it has been configured with a colored background, it should not draw it when overlaying the device camera, but should present a transparent background instead.

Typically, your cocos2d application will have a number of scenes, each with a primary layer. Since cocos2d reuses a single UIView instance on which to draw scenes, you will need only a single CCNodeController instance. But as the layers are swapped in and out during game play by the CCDirector, the CCNodeController must be kept in sync so that it knows which layer is currently active. You can use the controlledNode property on the CCNodeController to set the active layer. Or, to make this synchronization task easier, your application can use runSceneOnNode:, a convenience method on CCNodeController, to start or swap out a layer in both the CCDirector and the CCNodeController simultaneously with a single call.

Auto-rotation on Device Orientation Changes

The doesAutoRotate property of the CCNodeController enables and disables automatic rotation of the controlled node. When this property is set to YES, the controller will start to listen for notifications of device orientation change from iOS and propagate those notifications to the cocos2d framework as they arrive. When this property is set to NO, the controller will stop listening for iOS notifications. To automatically rotate a ControllableCCLayer in your application, all you need to do is set this property to YES, and attach the layer to the controller, typically using the controller’s runSceneOnNode: method, as described in the previous section.

With the doesAutoRotate property active, when the CCNodeController is notified by iOS that the device orientation has changed, it informs the cocos2d framework of the change, and also calls the deviceOrientationDidChange: method on the controlled node.

ControllableCCLayer is the most commonly used controlled node class. In most cocos2d applications, layers cover most or all of the device screen and, when the device orientation changes, the contentSize of the layer should generally change to match the new orientation. The ControllableCCLayer property alignContentSizeWithDeviceOrientation controls whether the layer will automatically adjust its contentSize to align with the new orientation. In particular, if this property is set to YES, and the device orientation changes from portrait to landscape or vice versa, the contentSize of the layer will automatically change to its transpose. When changing between two landscape orientations (landscape-right or landscape-left) or two portrait orientations (normal or upside down), the contentSize will remain unchanged. This property is initially set to YES. However, if your application has a layer subclass that is smaller than full screen, you might find occasion to set this property to NO so that the contentSize (and shape aspect) will remain steady through rotations between portrait and landscape orientations.

To provide a hook for subclasses of the ControllableCCLayer to know when the contentSize has changed, the ControllableCCLayer method didUpdateContentSizeFrom: is called automatically whenever the contentSize of the layer changes. Your subclass can override this method to adjust the positions of any child nodes after a rotation. For example, to keep a button or label in, say, the center or the bottom-right corner of the layer after a change in the contentSize of the layer, the subclass can update the position of the button or label accordingly in didUpdateContentSizeFrom:.

Finally, note that the auto-rotation works on cocos2d nodes directly and does not rotate the underlying iOS Cocoa UIView. If you are interested in combining cocos2d nodes with UIViews and having them auto-rotate together, please review cocos2d support for that here.

Overlay the Device Camera View

The CCNodeController property isOverlayingDeviceCamera controls whether the controlled node will be displayed over the view of the device camera. Your application only need only toggle this value to cause the overlaying effect to be applied or removed. The CCNodeController takes care of coordinating all the activities needed to make it happen. This property cannot be set to YES if the device does not actually have a camera. Attempting to do so is safe, because the action is simply ignored and the controlled node will continue to behave as it does when not overlaying the device camera. To find out if this property can successfully be set, you can check the isDeviceCameraAvailable property of CCNodeController, or you can simply set the isOverlayingDeviceCamera property and then check its value to see if the change was successful. The value of the isOverlayingDeviceCamera property can be changed at any time, and it is up to your application to determine when, if ever, that should happen. For instance, you might change it upon a scene transition, or even during active game play via a user-controlled on-screen button.

Within iOS, applying or removing an overlay on the device camera is not a trivial activity. Although your application interacts only with the isOverlayingDeviceCamera property, a lot happens under the covers. One important aspect is that in order to display an overlay on the camera image, a second specialized UIViewController (UIImagePickerController) is created and displayed modally. This does not specifically affect the use of cocos2d, but may impact applications that combine cocos2d visual components with standard UIKit views and controls. It also impacts the visual flow of your game, since going from not overlaying to overlaying can take a few seconds and involves a mandatory display of a camera iris image by iOS.

Another important aspect of swapping the device camera overlay in and out is that the running scene must be stopped and restarted on each change between overlaying and not overlaying or back again. This is needed to ensure that active actions and touch events are cleaned up correctly between transitions. For the most part, this effect is typically transparent to the application. As part of the stopping and restarting of the scene, the onExit and onEnter methods on the controlled node are called just before and just after the switchover, respectively. Your controlled node can make use of these calls as hooks to know when the switchover occurs in order to update state accordingly. Typically this might include hiding some visual items and showing others. For example, when the device camera image is visible, you might hide a background image or skybox and show some camera controls.

Fantastic! Where Can I Get Me Some of That?

You can download the source code for this framework in the download area at the top-right of this article. It is distributed under an MIT license, which makes it free for you to use in your projects. However, if you find this code is useful, please remember to make a donation above to help us fund the ongoing development and support of frameworks such as this.

Once downloaded, unzip the file, and drag the extracted directory to your Classes group within your Xcode project. On the dialog that pops up when you do that, make sure both the Copy items into destination group’s folder and Recursively create groups for any added folders options are both selected.

You can also download a full Xcode project of a simple example game that demonstrates the use of node controllers for both auto-rotation and device camera overlay. Simply unzip the downloaded file to extract the full project directory. The game is based on an example developed by Ray Wenderlich and published by him as a tutorial. Our version also makes use of a framework of cocos2d UI Controls that we developed, and which also may be of interest to you for your cocos2d projects.

• Joystick – a flexible joystick for user control in two dimensions with a single finger
• CCNodeAdornments – a framework for assigning adornments to CCNodes to temporarily change the appearance of the CCNode, such as scaling, fading, adding visual embellishments, etc. This is most useful when used with menu items to temporarily change the appearance of the menu item during user selection.

Joystick

For many games, controlling the action using a joystick is an integral and indispensable part of the game.

The joystick developed by us here is a subclass of CCLayer that contains two CCNode children, each of which is typically a CCSprite image. The thumb image is a smaller image that the user moves by touching and dragging. The second is an optional background image over which the thumb image is moved. The sizes of the thumb and background nodes can be set independently, or in the case of a joystick with no background image, the size of the Joystick node itself is set during initialization instead.

The following figure shows a Joystick control with a round green thumb and no background image controlling the player movement in a first-person game within a three-dimensional world. Like most first-person games, the player can be moved forward or backward and turn left or right by user movement of the joystick control.

A simple round green Joystick thumb, without a background image, controlling player motion in a simple first-person perspective game.

This is a visually trivial example of the Joystick control, stripped to the basics. In a real game, much more interesting thumb and background images would of course be applied to match the look and feel of the game.

To use the joystick, the user touches within the bounds of the Joystick node. Movement of the thumb image is constrained to the size of the Joystick node, however, the user may continue dragging outside the bounds of the joystick without losing control of the joystick. When the user lifts the finger from the screen, the joystick thumb will glide smoothly back to the center of the joystick (complete with a little bounce as it centers), just like a real physical joystick.

Creating a Joystick in your code is simple, as shown in the following, which sets up the Joystick shown above:

CCSprite* jsThumb = [CCSprite spriteWithFile: kJoystickThumbFileName];
// jsThumb.scale = 0.80; // change thumb size if you like
joystick = [Joystick joystickWithThumb: jsThumb
andSize: CGSizeMake(kJoystickSideLength, kJoystickSideLength)];
joystick.position = ccp(kJoystickPadding, kJoystickPadding);
[self addChild: joystick];

If a Joystick with a background image is desired instead, just use a different creation method:

CCSprite* jsThumb = [CCSprite spriteWithFile: kJoystickThumbFileName];
//  jsThumb.scale = 0.80;       // change thumb size if you like
CCSprite* jsBackdrop = [CCSprite spriteWithFile: kJoystickBackdropFileName];
joystick = [Joystick joystickWithThumb: jsThumb
andBackdrop: jsBackdrop];
joystick.position = ccp(kJoystickPadding, kJoystickPadding);
[self addChild: joystick];

The application can read the position of the Joystick thumb, relative to its resting position, at any time from either the velocity property or the angularVelocity property.

The velocity property returns a CGPoint reporting the relative position of the thumb in cartesian X-Y coordinates. The X-Y values returned range from -1 to +1, with zero being the center position. However, the magnitude of the velocity vector is clamped to unit length, so the set of possible points returned by the velocity property therefore covers the area of a circle of unit radius, centered on zero. This means, for instance, that pulling the thumb as far as it will go up and right will not take both X and Y each to a value of one at the same time. Instead, the thumb will stop when the radial distance from the origin is equal to one. This design provides a more natural range of values for controlling motion within a game.

The angularVelocity property returns a structure reporting the position of the thumb in an angular coordinate system, in terms of an angle plus a radius. The angle is given in positive or negative degrees from vertical (forward), and the radius ranges from zero to one. As with the velocity property, the set of points returned by the angularVelocity property therefore also covers the area of a circle of unit radius.

To add a Joystick to your game, download and install the cocos2d UI Controls framework code into your Xcode project as described below, then #import the Joystick.h file into your code where it is needed, typically into a custom CCLayer subclass file. The Joystick.h header file contains extensive documentation on the Joystick API and is your best source for understanding how to use it in your project.

CCNodeAdornments

Adornments are visual components that can be added to a subclass of CCNode to provide a visual appearance that can be activated and deactivated on demand. Familiar examples of this might be the icon tags associated with iOS app icons. Or within a cocos2d game, adornments can provide visual feedback when a user touches a game element, menu item, or other button-like nodes. The example shown below is the same scene shown in the Joystick description above, but in this case, the user is currently pressing the sunshine button (actually a CCMenuImageToggle) on the right. Doing so has activated the adornment and a yellow ring has faded in around the button. When the user releases the button, or drags his finger away from the button, the adornment will be deactivated, and the ring will fade away again.

The sunshine button on the right is being pressed by the user and a yellow ring adornment has faded in around the button.

To be clear, this image shown here is not the alternate toggled image of the CCMenuItemToggle. Nor is it the selected image of a CCMenuItemImage, which cannot be made to fade in and out in an animated manner. The adornment is a separate image that appears over or under the basic images of the menu item. In fact, in the action above, once the user releases the button, the CCMenuItemToggle itself will toggle to a different image altogether to show the changed state. The adornment is displayed only during the transitional stage while the user is actually pressing the button.

Instead of a ring, the adornment image could also have been a semi-transparent colored “shine” image that faded in and out over the main button image as the user touches and releases the button.

Another type of adornment scales the adorned node when activated. For example, the button could be made to grow in size when touched, and shrink back again when released, with the scaling up and down being animated for effect. Should fading and scaling seem boring to you, other adornment effects can easily be developed should you have other interesting effects in mind (spinning, changing color, etc).

Using adornments is fairly straightforward. At the very basic level, an adornment can simply be added as a child to any CCNode, and then tracked, activated and deactivated manually by the application. But where adornments really become interesting is when a CCNode is made adornable so it can automatically activate and deactivate its adornment at the appropriate times.

To make a CCNode adornable, you first create a subclass of the CCNode and add the AdornableCCNodeProtocol protocol to it. In this framework, we’ve done that already for CCMenuItemToggle and CCMenuItemImage, and tied the activation and deactivation of its adornment to the selected and unselected actions of the adornable subclasses. So when an AdornableMenuItemToggle is selected by the user, its adornment is automatically activated, and when the menu item is unselected, the adornment is automatically deactivated. As described above, depending on the adornment type, when the user selects and unselects the menu item, this might expand and shrink the menu item, or might fade in and out a shine or ring on top of the menu item.

The adornments themselves are CCNodes that support the CCNodeAdornmentProtocol protocol. To support development of new adornments, we’ve included in the framework an abstract CCNodeAdornmentBase class, along with concrete CCNodeAdornmentOverlayFader and CCNodeAdornmentScaler classes to perform the overlay fading and node scaling I’ve been describing above.

Putting this into practice then, the scene displayed in the picture above, with the yellow adornment ring, was created with the following code in the custom CCLayer that forms the scene:

// Toggle between two subitems: one for shutter button, one for sky button
CCMenuItem* camMI = [CCMenuItemImage itemFromNormalImage: kShutterButtonFileName
selectedImage: kShutterButtonFileName];
CCMenuItem* skyMI = [CCMenuItemImage itemFromNormalImage: kSkyButtonFileName
selectedImage: kSkyButtonFileName];
backdropToggleMI = [AdornableMenuItemToggle itemWithTarget: self
selector: @selector(changeBackdropSelected:)
items: camMI, skyMI, nil];
backdropToggleMI.position = ccp(self.contentSize.width - kBackdropToggleMenuInset, kBackdropToggleMenuInset);

 // Instead of having different normal and selected images, the toggle menu item uses an
// adornment, which is displayed whenever an item is selected.
 CCNodeAdornmentBase* adornment;

 // The adornment is a ring that fades in around the menu item and then fades out when
 // the menu item is no longer selected.
CCSprite* ringSprite = [CCSprite spriteWithFile: kButtonRingFileName];
adornment = [CCNodeAdornmentOverlayFader adornmentWithAdornmentNode: ringSprite];
 adornment.zOrder = kAdornmentUnderZOrder;

 // Attach the adornment to the menu item and center it on the menu item
 adornment.position = backdropToggleMI.anchorPointInPixels;
 backdropToggleMI.adornment = adornment;

CCMenu* bgMenu = [CCMenu menuWithItems: backdropToggleMI, nil];
bgMenu.position = CGPointZero;
[self addChild: bgMenu];

If you would rather see the “shine” adornment described above, replace lines 17-19 above with:

CCSprite* shineSprite = [CCSprite spriteWithFile: kButtonShineFileName];
shineSprite.color = ccYELLOW;
adornment = [CCNodeAdornmentOverlayFader adornmentWithAdornmentNode: shineSprite
peakOpacity: kPeakShineOpacity];

Or to have the menu item grow and shrink when pressed by the user, replace the original lines 17-19 above with:

adornment = [CCNodeAdornmentScaler adornmentToScaleUniformlyBy: kButtonAdornmentScale];

To add adornments to your project, download and install the cocos2d UI Controls framework code into your Xcode project as described below, then #import the CCNodeAdornments.h file into your code where it is needed. That header file also contains extensive documentation on the adornments API and is your best source for understanding the adornments framework. And once you get some experience with the framework, don’t be timid about extending the framework by inventing your own adornments!

Fantastic! Where Can I Get Me Some of That?

You can download the source code for this framework in the download area at the top-right of this article. It is distributed under an MIT license, which makes it free for you to use in your projects. However, if you find this code is useful, please remember to make a donation above to help us fund the ongoing development and support of frameworks such as this.

Once downloaded, unzip the file, and drag the extracted directory to your Classes group within your Xcode project. On the dialog that pops up when you do that, make sure both the Copy items into destination group’s folder and Recursively create groups for any added folders options are both selected.

Credit Where Credit is Due

Our Joystick code was inspired by code donated to the cocos2d community by cocos2d user adunsmoor. The original inspirational code can be found here.

However, the use of NSLog suffers from two drawbacks:

1. There is only one level of logging with NSLog. So during development, you get a record of all of the NSLog calls that you’ve inserted, some of which might be quite detailed and frequent, and it’s easy to be overwhelmed by the clutter. There is no way of selectively turning off some of the output.
2. The NSLog function is called every time it is encountered, including within executing production release code. To avoid this, developers must typically remove all calls to NSLog before compiling for a production release. This is a headache fraught with possible mistakes and inhibits debugging future releases.

To remedy this, we’ve developed a logging framework that solves both of these issues. This library consists of a single Logging.h header file which adds, to iOS, flexible, non-intrusive logging capabilities that are efficiently enabled or disabled via compile switches.

Quick Start

To use this enhanced logging functionality, simply #import the Logging.h header file into any code file to which you want to add enhanced logging, and set the compiler switch LOGGING_ENABLED=1, either in a #define statement or in the Xcode build setting GCC_PREPROCESSOR_DEFINITIONS.

Logging Levels

To address the first issue described above, that of a single level of logging, this library defines four levels of logging: Trace, Info, Error and Debug, and each is enabled independently via a distinct compiler switch. Invoking any level is as simple as calling the appropriate variadic (variable number of arguments) function as shown in the following examples:

LogTrace(@"Simple message");
LogTrace(@"Message with %i argument...oops I mean %.1f arguments", 1, 2.0f);
LogInfo(@"Do you object to this %@ date object?", [NSDate new]);
LogDebug(@"I'll take this out if ever I solve this bug");
LogError(@"Don't do that: %@", [NSException exceptionWithName: @"CRYING" reason: @"Spilled milk" userInfo:nil]);

In the examples above, you can see the four logging levels in action, along with the use of variable arguments that match against templates in the message string, in exactly the same manner as the NSLog or printf functions. The first argument is an NSString that optionally includes embedded format specifiers, and subsequent optional arguments indicate data to be formatted and inserted into that string. As with NSLog and printf, there must be an equal number of embedded format specifiers and optional arguments. For more information on string formatting, see the documentation for NSLog, String Format Specifiers and printf formatting.

The output of the above code is:

2010-07-23 18:27:23.398 Logging Sample[35700:20b] [trace] Simple message
2010-07-23 18:27:23.399 Logging Sample[35700:20b] [trace] Message with 1 argument...oops I mean 2.0 arguments
2010-07-23 18:27:23.402 Logging Sample[35700:20b] [info] Do you object to this 2010-07-23 18:27:23 -0400 date object?
2010-07-23 18:27:23.402 Logging Sample[35700:20b] [DEBUG] I'll take this out if ever I solve this bug
2010-07-23 18:27:23.403 Logging Sample[35700:20b] [***ERROR***] Don't do that: Spilled milk

In this output listing, Logging Sample is just the name of the Xcode project. As you can see, each log message identifies the level of logging. Trace logging is recommended for detailed tracing of program flow, such as inside a loop, or on each frame render in a graphic engine. Info logging is recommended for general, infrequent, information messages such as initialization or shutdown activities, file loads, or user settings changes. Error logging, as the name suggests, is recommended for use only when there is an error to be logged. And finally, debug logging is recommended for temporary use during debugging. When trying to track down a coding error, you might temporarily sprinkle even your most detailed code with debug log messages. But once the problem is resolved, you should remove them, or possibly convert the most useful of them into info or trace logs.

Each of the logging levels is independently turned on or off by setting a boolean compiler switch to either 1 or 0, respectively, as follows:

• LOGGING_LEVEL_TRACE enables or disables the LogTrace function.
• LOGGING_LEVEL_INFO enables or disables the LogInfo function.
• LOGGING_LEVEL_ERROR enables or disables the LogError function.
• LOGGING_LEVEL_DEBUG enables or disables the LogDebug function.

By selectively turning on or off each of these switches, you could choose to output only error log messages so that logging will only occur if something goes wrong, or perhaps include info messages, so that you can keep track of high-level events such as file loads, or initialization and shutdown code. And when attempting to debug the details of a module, where you want more copious and detailed logging, you might enable either or both of the trace or debug logging levels.

The LOGGING_ENABLED compiler switch controls overall logging. This is normally turned on in development code to make use of the logging functionality. But in production code, you can disable all logging in one fell swoop by turning off the LOGGING_ENABLED compiler switch. This has the effect of overriding all the individual level switches, and disables all logging. None of the levels will produce log messages with the LOGGING_ENABLED compiler switch turned off.

To assist with tracking down bugs, you can opt to have each logged entry automatically include class, method and line information by turning on the LOGGING_INCLUDE_CODE_LOCATION compiler switch. Doing so will modify the log output above to appear as:

2010-07-23 18:28:17.238 Logging Sample[35732:20b] -[Logging_SampleAppDelegate applicationDidFinishLaunching:][Line 20] [trace] Simple message
2010-07-23 18:28:17.239 Logging Sample[35732:20b] -[Logging_SampleAppDelegate applicationDidFinishLaunching:][Line 21] [trace] Message with 1 argument...oops I mean 2.0 arguments
2010-07-23 18:28:17.247 Logging Sample[35732:20b] -[Logging_SampleAppDelegate applicationDidFinishLaunching:][Line 22] [info] Do you object to this 2010-07-23 18:28:17 -0400 date object?
2010-07-23 18:28:17.248 Logging Sample[35732:20b] -[Logging_SampleAppDelegate applicationDidFinishLaunching:][Line 23] [DEBUG] I'll take this out if ever I solve this bug
2010-07-23 18:28:17.249 Logging Sample[35732:20b] -[Logging_SampleAppDelegate applicationDidFinishLaunching:][Line 24] [***ERROR***] Don't do that: Spilled milk

Like most compiler switches, each of the switches discussed in this article is turned on by setting its value to 1 and turned off by setting its value to 0. This can be done via #define statements in your code (or in Logging.h), or via the compiler build setting GCC_PREPROCESSOR_DEFINITIONS in your Xcode build configuration. Using the GCC_PREPROCESSOR_DEFINITIONS Xcode setting is the preferred choice, so that logging can be configured differently for different Xcode targets or build configurations, and particularly to ensure that logging is not accidentally left enabled in production release builds.

Logging Overhead

To address the second issue described above, namely the memory and processing overhead of the logging function calls themselves, the logging functions are implemented here via macros. Disabling logging, either entirely, or at a specific level, completely removes the corresponding log invocations from the compiled code, thereby eliminating both the memory and CPU overhead that the logging calls would add. So, there is absolutely no runtime penalty to be paid by leaving copious quantities of LogTrace calls spread thickly throughout your code, as long as you simply disable either trace logging or all logging by turning off the LOGGING_LEVEL_TRACE or LOGGING_ENABLED compiler switch, respectively, at build time.

Fantastic! Where Can I Get Me Some of That?

You can download this logging framework (a single Logging.h file) in the download area at the top-right of this article. It is distributed under an MIT license, which makes it free for you to use in your projects. However, if you find this code is useful, please remember to make a donation above to help us fund the ongoing development and support of frameworks such as this.

Credit Where Credit is Due

Thanks to Nick Dalton for outlining the underlying ideas for using variadic macros as well as for outputting the code location as part of the log entry.