JavascriptGraphicsAPI
Rationale for a common API to 3d graphics systems
Objects are sent across the thread barrier to alter the current scene graph being displayed--here's why:
In modern engines, graphics framerates should not be tied to physics framerates and networking events and decoding of said events should happen at the correct pace to keep up with the networknig adapter. Graphics, however, is tied to the DOM and therefore must be on the main thread. This forces both networking and physics to be on webworker threads if there is to be any threading.
These threads need to advertise state changes to the main graphics thread so that the scene graph may be altered at the graphics rate. This requires that the individual physics and networking threads send timestamped events to the graphics system which drive changes to it.
Since graphics can run at different rates and the updates from the network may be irregular, the graphics (main) thread needs to have a smooth interpolation scheme interpolating the current position with the timestamped updates sent by the network/physics thread(s).
Below are some example objects that may be sent cross-thread. The objects are listed in JSON format so the type information should be clear from the example.
API To Graphics System
Graphics should provide a function called InitializeGFX(name,callback) The function definition should look as follows
InitializeGFX=function(name,callback) {
if (name in GraphicsPlugins) {
GraphicsPlugins[name](callback);
return true;
}
return false;
}
The plugin should also carefully create the list GraphicsPlugins if that list is not there and add itself to the list of graphics plugins
callback is a function taking a single serializable object and passing that message to the rest of the system as a manner of providing a mechanism for the graphics system to call back with key and mouse events
calling GraphicsPlugin[name] must create a global method called SendGFX(object) which routes one of the below cross thread message to the graphics subsystem.
Cross thread communication from Physics and Networking to graphics
Object Management
Creating a new graphics object
{
msg:"Create"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
time: 2181298451298491284,//milliseconds since 1970
pos:[1,2,3],
vel:[.25,0,0],
orient:[.5,0,0,.5]
rotaxis:[0,0,1]
rotvel:.25,
scale:[1,1,1],//defaults to 1,1,1 if absent
parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479",//<-- optional (defaults to empty--toplevel if absent
parentbone:"Hand"//name of the bone on the parent object that this is attached to. Assume root transform otherwise
}
Moving a graphics object
//should we define that the graphics system has some sort of interp--otherwise velocity may be useless?
{
msg:"Move"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
time:39852398592385,//milliseconds since 1970
pos:[1,2,3],
vel:[.25,0,0],
orient:[.5,0,0,.5],//defaults to (identity) if absent
rotaxis:[0,0,1],//defaults to 0,0,1 if absent, forcing rotvel to 0
rotvel:.25,//defaults to 0 if absent
scale:[1,1,1],//defaults to 1,1,1 if absent
parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479"//<-- optional (defaults to previous state if absent, to clear pass empty string)
parentbone:"Hand"//name of the bone on the parent object that this is attached to. Defaults to previous state if absent, to clear pass empty string
}
Destroying a graphics object
{
msg:"Destroy"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
Managing object appearance properties
Adding/changing mesh property for an object
{
msg:"Mesh",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
mesh:"http://example.com/test.dae",//the mesh should be rescaled to fit inside a unit sphere centered at 0,0,0
billboard:false//defaults to false, if set to true the object will stay aligned with camera
}
Removing mesh property for an object
{
msg:"DestroyMesh",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
}
Adding/changing light property for an object
{
msg:"Light"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
diffuse_color:[.25,.5,1],
specular_color: [.2,1,.5],
power=1.0: //exponent on the light
ambient_color: [0,0,0],
light_range: 1.0e5
constant_falloff: 0.5,
linear_falloff: 0.2,
quadratic_falloff: 0.1,
cone_inner_radians: 0,
cone_outer_radians: 0,
cone_falloff: 0.5,
type: "POINT",//options include "SPOTLIGHT" or "DIRECTIONAL"
casts_shadow: true
}
Removing light property for an object
{
msg:"DestroyLight"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
Camera Management
Attaching a camera to an object
{
msg:"Camera"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
primary:true//whether the camera should be attached to the canvas rendering surface (may be overridden with future calls to Camera)
}
Detaching a camera from an object
{
msg:"DestroyCamera"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
Attach a camera to an object's texture"
{
msg:AttachCamera",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
camid:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
texname:"example.png"//overwrites this texture
}
Attaching UI elements to graphics objects
The UI will naturally need to be in HTML since that's the best established cross platform, sandboxed UI system.
The user may specify a 3d location, orientation and scale for a UI dialog to be. The graphics system should do its best to scale and position the UI in the appropriate place, but the UI may be restricted to always face the camera and always be horizontal compared with the bottom of the screen on many system. The UI should not be displayed if it is completely invisible from the camera angle or smaller than 10 pixels.
Creating/Updating UI Element
{
msg:"IFrame"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
uri: "http://example.com"
}
Notification from UI Element
The iframe should have a parent method called "UISend" which takes a serializable object and which will invoke the callback that was passed into the graphics system with an object like
{
msg: "IFrameCallback"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
argument:{name:"Clicked button",location:[4,5]}
}
Destroying UI Element
{
msg:"DestroyIFrame"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
}
Skeleton Management
Animating a skeleton based on a time based animation
{
msg:"Ani",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
time:489192048120984102,///milliseconds since 1970 that the animation should be started from (skip frames if now is later)
animation:"http://example.com/animation.dae",
loop:false,
weight:1.0 ///how strong this animation should compare with other animations that use the same bones
fadein:2.3 //how many seconds to fade in
}
Stopping a skeleton based on a time based animation
{
msg:"AniStop",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
animation:"http://example.com/animation.dae"
fadeout:1.0//how many seconds to fade out
}
Streaming some joint locations
{
msg:"AnimateBone",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
animation:"uniqueAnimationIdentifier",//so this movement can be associated with one animation and blended with others
weight:1.0,//the weight for prospective blending, defaults to 1.0
time:1250120951209510295;//milliseconds since 1970
bone:["ankle","arm"]
pos:[[1,2,3],[2,3,4]]
vel:[[.25,0,0],[0,0,0]]
orient:[[.5,0,0,.5],[1,0,0,0]]
rotaxis:[[0,0,1],[0,1,0]]
rotvel:[.25,0]
}
Scene Queries
Requesting intersection
{
msg:"RayTrace",
pos:[2,3,4],
dir:[.24,.33,.5],
multiple:true//if false, only return first hit
infinite:false//if false use length of dir to specify ray length
}
Intersection callback
{
msg:"Intersections",
pos:[[2,3,4],[2.23,3.32,4.49]]
id:["f47ac10b-58cc-4372-a567-0e02b2c3d479","a33ff133-58dd-2272-dd6a-12aadc31d173",
}
Particle System
Adding a particle system to an object
This mimics the ogre interface and we introduce a number of billboard types
point The default arrangement, this approximates spherical particles and the billboards always fully face the camera. oriented_common Particles are oriented around a common, typically fixed direction vector (see common_direction), which acts as their local Y axis. The billboard rotates only around this axis, giving the particle some sense of direction. Good for rainstorms, starfields etc where the particles will traveling in one direction - this is slightly faster than oriented_self (see below). oriented_self Particles are oriented around their own direction vector, which acts as their local Y axis. As the particle changes direction, so the billboard reorients itself to face this way. Good for laser fire, fireworks and other 'streaky' particles that should look like they are traveling in their own direction. perpendicular_common Particles are perpendicular to a common, typically fixed direction vector (see common_direction), which acts as their local Z axis, and their local Y axis coplanar with common direction and the common up vector (see common_up_vector). The billboard never rotates to face the camera, you might use double-side material to ensure particles never culled by back-facing. Good for aureolas, rings etc where the particles will perpendicular to the ground - this is slightly faster than perpendicular_self (see below). perpendicular_self Particles are perpendicular to their own direction vector, which acts as their local Z axis, and their local Y axis coplanar with their own direction vector and the common up vector (see common_up_vector). The billboard never rotates to face the camera, you might use double-side mater
For further documentation about the properties see http://www.ogre3d.org/docs/manual/manual_35.html#SEC191 http://www.ogre3d.org/docs/manual/manual_36.html#SEC208 and
{
msg:"ParticleSystem",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
mesh:"http://example.com/billboard.dae"//the mesh should be rescaled to be a 1x1 mesh with
particle_size:[20,20],
cull_each:false
quota:10000
billboard:"oriented_self",
sorted:false//defaults to false--whether the particles should be sorted
local:false//defaults to false--if true rotation of the node after the emission of the particle will rotate it
direction: [0,0,1],///the common direction for oriented_common or perpendicular_common
up: [0,0,1],///Only required if billboard_type is set to perpendicular_self or perpendicular_common, this vector is the common up vector used to orient all particles in the system.
accurate_facing:false//if the facing is set to the camera facing or calculated per billboard
iteration_interval:.125//how often the particles are updated--if set to 0, defaults to framerate
invisibility_timeout:10//how many seconds of being outside the frustum before the system stops updating
}
{
msg:"DestroyParticleSystem"
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
once a system is created, particles need to be emitted from it. There should be a global map of default emitters named ParticleEmitters consisting of at least
"Point","Box","Cylinder","Ellipsoid","Shell","Ring" and the extra attributes are specified in http://www.ogre3d.org/docs/manual/manual_38.html
{
msg:"ParticleEmitter",//add or upate a particle emitter
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
name:"Flare"
type:"Ring"
angle 15
emission_rate 75
time_to_live:[2.5,3]//range between 2.5 and 3
direction [0, 1, 0]//3d vector
speed:[250,300]//range between 250 and 300
colour_range:[[1 0 0],[0 0 1]]//random color
position:[0,0,0],
repeat_delay:[2.5,5]
}
{
msg:"RemoveParticleEmitter",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
name:"Flare"
}
There may be forces applied to the emitters and there must be a global map of affectors called ParticleAffector from which the relevent affector is selected consisting of at least
LinearForce, ColourFader, Scaler, Rotator, ColourInterpolator, ColourImage, DeflectorPlane, DirectionRandomiser The detailed definitions are contanied at http://www.ogre3d.org/docs/manual/manual_40.html#SEC234
{
msg:"ParticleAffector",//add or upate a particle emitter
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
name:"TheForce"
type:"LinearForce"
force_vector: [0 -100 0]
force_application: "add"
}
{
msg:"RemoveParticleAffector",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
name:"TheForce"
}
Experimental/Brainstorming ideas for the API
It seems like there should be a manner aside from "embedded iframes" to get art defined in the DOM into the scene graph--perhaps the canvas tag is the way to go here? But maybe that's too webGL specific and won't work for an Ogre port of this
maybe just text rendering belongs here?
Anyhow anything below here is purely speculative
Attaching 3d Text to an Objects
I'm just brainstorming here: it seems like WebGL has facilities to do this efficiently, but I don't have a good use case except buildnig a rendering system inside a canvas tag or something?
Perhaps the canvas tag is the way to go
{
msg:"Text",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
text:"This is a test of the emergency broadcast system",
font:"size=+1"
}
{
msg:"DestroyText",
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479",
}