Difference between revisions of "JavascriptGraphicsAPI"
Line 18: | Line 18: | ||
gfx.send({ msg:"Create", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", time: 2181298451298491284, pos:[1,2,3], orient:[.5,0,0,.5]}) | gfx.send({ msg:"Create", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", time: 2181298451298491284, pos:[1,2,3], orient:[.5,0,0,.5]}) | ||
gfx.destroy(); | gfx.destroy(); | ||
+ | |||
+ | id's can be anything from human readable strings to uuids to integers. They just each must be unique and chosen by the user of the API | ||
=Cross thread communication from Physics and Networking to graphics= | =Cross thread communication from Physics and Networking to graphics= | ||
Line 27: | Line 29: | ||
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" | id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" | ||
time: 2181298451298491284,//milliseconds since 1970 | time: 2181298451298491284,//milliseconds since 1970 | ||
− | pos:[1,2,3], | + | pos:[1,2,3],//absolute position |
vel:[.25,0,0], | vel:[.25,0,0], | ||
− | orient:[.5,0,0,.5] | + | orient:[.5,0,0,.5],//quaternion |
− | + | rotation:[100,0,.25,.25],//normalized quaternion indicating rotational axis with magnitude living in the w term | |
− | |||
scale:[1,1,1],//defaults to 1,1,1 if absent | scale:[1,1,1],//defaults to 1,1,1 if absent | ||
parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479",//<-- optional (defaults to empty--toplevel if absent | parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479",//<-- optional (defaults to empty--toplevel if absent | ||
Line 44: | Line 45: | ||
id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" | id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" | ||
time:39852398592385,//milliseconds since 1970 | time:39852398592385,//milliseconds since 1970 | ||
− | pos:[1, | + | pos:[1,4,3],//absolute position |
vel:[.25,0,0], | vel:[.25,0,0], | ||
orient:[.5,0,0,.5],//defaults to (identity) if absent | orient:[.5,0,0,.5],//defaults to (identity) if absent | ||
− | + | rotation:[100,0,.25,.25],//normalized quaternion indicating rotational axis with magnitude living in the w term | |
− | |||
scale:[1,1,1],//defaults to 1,1,1 if absent | scale:[1,1,1],//defaults to 1,1,1 if absent | ||
interpolate:true,//set to false if the object should snap to new position | interpolate:true,//set to false if the object should snap to new position | ||
parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479"//<-- optional (defaults to previous state if absent, to clear pass empty string) | parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479"//<-- optional (defaults to previous state if absent, to clear pass empty string) | ||
− | + | attachment_point:"Hand"//name of the bone on the parent object that this is attached to. Defaults to previous state if absent, to clear pass empty string | |
} | } | ||
Revision as of 23:46, 4 March 2010
Rationale for a common API to 3d graphics systems
Objects are sent across the thread barrier to alter the current scene graph being displayed--here's why:
In modern engines, graphics framerates should not be tied to physics framerates and networking events and decoding of said events should happen at the correct pace to keep up with the networknig adapter. Graphics, however, is tied to the DOM and therefore must be on the main thread. This forces both networking and physics to be on webworker threads if there is to be any threading.
These threads need to advertise state changes to the main graphics thread so that the scene graph may be altered at the graphics rate. This requires that the individual physics and networking threads send timestamped events to the graphics system which drive changes to it.
Since graphics can run at different rates and the updates from the network may be irregular, the graphics (main) thread needs to have a smooth interpolation scheme interpolating the current position with the timestamped updates sent by the network/physics thread(s).
Below are some example objects that may be sent cross-thread. The objects are listed in JSON format so the type information should be clear from the example.
API To Graphics System
Graphics should provide a constructor method that takes in a callback and a parent DOM element and returns a class that has a send(obj) method that takes in serializable objects from other threads that modify graphics state and an optional "destroy" method which cleans up graphics state in the DOM.
so a sequence of code to construct an graphics system, make an object, and destroy it could look like
gfx= new GLGERenderer(callbackFunction,parentElement) gfx.send({ msg:"Create", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", time: 2181298451298491284, pos:[1,2,3], orient:[.5,0,0,.5]}) gfx.destroy();
id's can be anything from human readable strings to uuids to integers. They just each must be unique and chosen by the user of the API
Cross thread communication from Physics and Networking to graphics
Object Management
Creating a new graphics object
{ msg:"Create" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" time: 2181298451298491284,//milliseconds since 1970 pos:[1,2,3],//absolute position vel:[.25,0,0], orient:[.5,0,0,.5],//quaternion rotation:[100,0,.25,.25],//normalized quaternion indicating rotational axis with magnitude living in the w term scale:[1,1,1],//defaults to 1,1,1 if absent parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479",//<-- optional (defaults to empty--toplevel if absent parentbone:"Hand"//name of the bone on the parent object that this is attached to. Assume root transform otherwise }
Moving a graphics object
//should we define that the graphics system has some sort of interp--otherwise velocity may be useless?
{ msg:"Move" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" time:39852398592385,//milliseconds since 1970 pos:[1,4,3],//absolute position vel:[.25,0,0], orient:[.5,0,0,.5],//defaults to (identity) if absent rotation:[100,0,.25,.25],//normalized quaternion indicating rotational axis with magnitude living in the w term scale:[1,1,1],//defaults to 1,1,1 if absent interpolate:true,//set to false if the object should snap to new position parent:"c46ac00b-58cc-4372-a567-0e02b2c3d479"//<-- optional (defaults to previous state if absent, to clear pass empty string) attachment_point:"Hand"//name of the bone on the parent object that this is attached to. Defaults to previous state if absent, to clear pass empty string }
Destroying a graphics object
{ msg:"Destroy" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" }
Managing object appearance properties
Adding/changing mesh property for an object
{ msg:"Mesh", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", mesh:"http://example.com/test.dae",//the mesh should be rescaled to fit inside a unit sphere centered at 0,0,0 billboard:false//defaults to false, if set to true the object will stay aligned with camera }
note if the billboard property is present and not false, it should be set to one of the following 3 strings This mimics the ogre interface and we introduce a number of billboard types
point This approximates spherical particles and the billboards always fully face the camera. oriented_self Particles are oriented around their own direction vector, which acts as their local Y axis. As the particle changes direction, so the billboard reorients itself to face this way. Good for laser fire, fireworks and other 'streaky' particles that should look like they are traveling in their own direction. perpendicular_self Particles are perpendicular to their own direction vector, which acts as their local Z axis, and their local Y axis coplanar with their own direction vector and the common up vector (see common_up_vector). The billboard never rotates to face the camera, you might use double-side mater
For further documentation about the properties see http://www.ogre3d.org/docs/manual/manual_35.html#SEC191 and http://www.ogre3d.org/docs/manual/manual_36.html#SEC208
Updating shader property (Vertex and Fragment float4) for an object
{ msg:"MeshShaderUniform" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", name:"ColorTint" value:[.24,.24,.25,1.0] }
Removing mesh property for an object
{ msg:"DestroyMesh", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", }
Adding/changing light property for an object
{ msg:"Light" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" diffuse_color:[.25,.5,1], specular_color: [.2,1,.5], power=1.0: //exponent on the light ambient_color: [0,0,0], light_range: 1.0e5 constant_falloff: 0.5, linear_falloff: 0.2, quadratic_falloff: 0.1, cone_inner_radians: 0, cone_outer_radians: 0, cone_falloff: 0.5, type: "POINT",//options include "SPOTLIGHT" or "DIRECTIONAL" casts_shadow: true }
Removing light property for an object
{ msg:"DestroyLight" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" }
Camera Management
Attaching a camera to an object
{ msg:"Camera" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" primary:true//whether the camera should be attached to the canvas rendering surface (may be overridden with future calls to Camera) }
Detaching a camera from an object
{ msg:"DestroyCamera" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" }
Attach a camera to an object's texture"
{ msg:AttachCamera", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", camid:"f47ac10b-58cc-4372-a567-0e02b2c3d479", texname:"example.png"//overwrites this texture }
Attaching UI elements to graphics objects
The UI will naturally need to be in HTML since that's the best established cross platform, sandboxed UI system.
The user may specify a 3d location, orientation and scale for a UI dialog to be. The graphics system should do its best to scale and position the UI in the appropriate place, but the UI may be restricted to always face the camera and always be horizontal compared with the bottom of the screen on many system. The UI should not be displayed if it is completely invisible from the camera angle or smaller than 10 pixels.
Creating/Updating UI Element
{ msg:"IFrame" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", uri: "http://example.com" }
Notification from UI Element
The iframe should have a parent method called "UISend" which takes a serializable object and which will invoke the callback that was passed into the graphics system with an object like
{ msg: "IFrameCallback" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", argument:{name:"Clicked button",location:[4,5]} }
Destroying UI Element
{ msg:"DestroyIFrame" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", }
Skeleton Management
Streaming some joint locations
{ msg:"AnimateBone", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", animation:"uniqueAnimationIdentifier",//so this movement can be associated with one animation and blended with others //if not specified this is a hard constraint (i.e. foot is glued to a wall in order to avoid penetrating it) weight:1.0,//the weight for prospective blending, defaults to 1.0 time:1250120951209510295;//milliseconds since 1970 bone:["ankle","arm"] pos:[[1,2,3],[2,3,4]] vel:[[.25,0,0],[0,0,0]] orient:[[.5,0,0,.5],[1,0,0,0]] rotaxis:[[0,0,1],[0,1,0]] rotvel:[.25,0], interpolate:true//if false then the bone should snap to the location unless smooth is set (in which case it should interpolate as quickly as possible) defaults to true }
Scene Queries
Requesting intersection
{ msg:"RayTrace", id:5 pos:[2,3,4], dir:[.24,.33,.5], multiple:true//if false, only return first hit infinite:false//if false use length of dir to specify ray length }
Intersection callback
{ msg:"Intersections", id:5 pos:[[2,3,4],[2.23,3.32,4.49]] normals:[[0,1,0],[.5,0,.86]] id:["f47ac10b-58cc-4372-a567-0e02b2c3d479","a33ff133-58dd-2272-dd6a-12aadc31d173", }
Particle System
Adding a particle system to an object
This mimics the ogre interface and we introduce a number of billboard types
point The default arrangement, this approximates spherical particles and the billboards always fully face the camera. oriented_common Particles are oriented around a common, typically fixed direction vector (see common_direction), which acts as their local Y axis. The billboard rotates only around this axis, giving the particle some sense of direction. Good for rainstorms, starfields etc where the particles will traveling in one direction - this is slightly faster than oriented_self (see below). oriented_self Particles are oriented around their own direction vector, which acts as their local Y axis. As the particle changes direction, so the billboard reorients itself to face this way. Good for laser fire, fireworks and other 'streaky' particles that should look like they are traveling in their own direction. perpendicular_common Particles are perpendicular to a common, typically fixed direction vector (see common_direction), which acts as their local Z axis, and their local Y axis coplanar with common direction and the common up vector (see common_up_vector). The billboard never rotates to face the camera, you might use double-side material to ensure particles never culled by back-facing. Good for aureolas, rings etc where the particles will perpendicular to the ground - this is slightly faster than perpendicular_self (see below). perpendicular_self Particles are perpendicular to their own direction vector, which acts as their local Z axis, and their local Y axis coplanar with their own direction vector and the common up vector (see common_up_vector). The billboard never rotates to face the camera, you might use double-side mater
For further documentation about the properties see http://www.ogre3d.org/docs/manual/manual_35.html#SEC191 http://www.ogre3d.org/docs/manual/manual_36.html#SEC208 and
{ msg:"ParticleSystem", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" mesh:"http://example.com/billboard.dae"//the mesh should be rescaled to be a 1x1 mesh with particle_size:[20,20], cull_each:false quota:10000 billboard:"oriented_self", sorted:false//defaults to false--whether the particles should be sorted local:false//defaults to false--if true rotation of the node after the emission of the particle will rotate it direction: [0,0,1],///the common direction for oriented_common or perpendicular_common up: [0,0,1],///Only required if billboard_type is set to perpendicular_self or perpendicular_common, this vector is the common up vector used to orient all particles in the system. accurate_facing:false//if the facing is set to the camera facing or calculated per billboard iteration_interval:.125//how often the particles are updated--if set to 0, defaults to framerate invisibility_timeout:10//how many seconds of being outside the frustum before the system stops updating }
{ msg:"DestroyParticleSystem" id:"f47ac10b-58cc-4372-a567-0e02b2c3d479" }
once a system is created, particles need to be emitted from it. There should be a global map of default emitters named ParticleEmitters consisting of at least
"Point","Box","Cylinder","Ellipsoid","Shell","Ring" and the extra attributes are specified in http://www.ogre3d.org/docs/manual/manual_38.html
{ msg:"ParticleEmitter",//add or upate a particle emitter id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", name:"Flare" type:"Ring" angle 15 emission_rate 75 time_to_live:[2.5,3]//range between 2.5 and 3 direction [0, 1, 0]//3d vector speed:[250,300]//range between 250 and 300 colour_range:[[1 0 0],[0 0 1]]//random color position:[0,0,0], repeat_delay:[2.5,5] }
{ msg:"RemoveParticleEmitter", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", name:"Flare" }
There may be forces applied to the emitters and there must be a global map of affectors called ParticleAffector from which the relevent affector is selected consisting of at least
LinearForce, ColourFader, Scaler, Rotator, ColourInterpolator, ColourImage, DeflectorPlane, DirectionRandomiser The detailed definitions are contanied at http://www.ogre3d.org/docs/manual/manual_40.html#SEC234
{ msg:"ParticleAffector",//add or upate a particle emitter id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", name:"TheForce" type:"LinearForce" force_vector: [0 -100 0] force_application: "add" }
{ msg:"RemoveParticleAffector", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", name:"TheForce" }
Experimental/Brainstorming ideas for the API
I decided to reserve a section of the wiki for sort of bleeding edge ideas of cool features that would be nice to have. I could have put that in the "talk" page, but I think it makes more sense here so that it will get wider exposure. These are meant to be things that would help in drawing real scenes and building real VW systems but that we haven't figured out a good API to yet.
It seems like there should be a manner aside from "embedded iframes" to get art defined in the DOM into the scene graph--perhaps the canvas tag is the way to go here? But maybe that's too webGL specific and won't work for an Ogre port of this
maybe just text rendering belongs here?
Anyhow anything below here is purely speculative
Attaching 3d Text to an Objects
I'm just brainstorming here: it seems like WebGL has facilities to do this efficiently, but I don't have a good use case except buildnig a rendering system inside a canvas tag or something?
Perhaps the canvas tag is the way to go
{ msg:"Text", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", text:"This is a test of the emergency broadcast system", font:"size=+1" }
{ msg:"DestroyText", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", }
Specifying a terrain for the world
Would need to be chunked and in some sort of widely readable format---would be nice to be able to tap into google earth for terrain--ideas for how to do this are still very very early
Deprecated API Ideas
Here we put ideas we had but decided to discard so that they don't come up again as new ideas and may be discussed here and evaluated for re-addition if someone feels strongly they should be included
Skeleton file formats
The reason these were removed is that they are too brittle (it's hard to weigh an wave and walk animation and have the steps not be half as wide) and it's difficult to keep the skeletons out of trouble (i.e. feet through the ground) so we think thta the physics system in general should send the bone positions and timestamps since it's the arbiter of what intersects what--and it can always read the skeleton file format.
Animating a skeleton based on a time based animation
{ msg:"Ani", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", time:489192048120984102,///milliseconds since 1970 that the animation should be started from (skip frames if now is later) animation:"http://example.com/animation.dae", loop:false, weight:1.0 ///how strong this animation should compare with other animations that use the same bones fadein:2.3 //how many seconds to fade in }
Note that the animation.dae should have annotations for loop-in point and loop-out point within the .dae so that loop can intelligently function
Stopping a skeleton based on a time based animation
{ msg:"AniStop", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", animation:"http://example.com/animation.dae" fadeout:1.0//how many seconds to fade out }
Should an object just be a sprite
We figured that a collada square file may be a more compat representation for a sprite and can contain the appropriate shader, materials, etc
Making an object a point sprite
This mimics the ogre interface and we introduce a number of billboard types
point The default arrangement, this approximates spherical particles and the billboards always fully face the camera. oriented_self Particles are oriented around their own direction vector, which acts as their local Y axis. As the particle changes direction, so the billboard reorients itself to face this way. Good for laser fire, fireworks and other 'streaky' particles that should look like they are traveling in their own direction. perpendicular_self Particles are perpendicular to their own direction vector, which acts as their local Z axis, and their local Y axis coplanar with their own direction vector and the common up vector (see common_up_vector). The billboard never rotates to face the camera, you might use double-side mater
For further documentation about the properties see http://www.ogre3d.org/docs/manual/manual_35.html#SEC191 http://www.ogre3d.org/docs/manual/manual_36.html#SEC208
{ msg:"Sprite", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", sprite:"http://example.com/test.jpg",//the mesh should be rescaled to fit inside a unit sphere centered at 0,0,0 billboard:"perpendicular_self" sorted:false//defaults to false--whether the particles should be sorted up: [0,0,1],///Only required if billboard_type is set to perpendicular_self, this vector is the common up vector used to orient all particles in the system. }
Removing point sprite property from object
{ msg:"DestroySprite", id:"f47ac10b-58cc-4372-a567-0e02b2c3d479", }