- Generated by
1.8.13
|
3es
0.7
|
3rd Eye Scene debugging is built on top of a core messaging system used to expose 3D data for external viewing. This section details the packet header used for all messages, documents the core messages and details compression.
The primary transport layer is intended to be TCP/IP, however other protocols may be used in future. For consistency, all data elements are in Network Byte Order (Big Endian). Many common platforms are Little Endian and will required byte order swaps before using data values; e.g., Intel based processors are generally Little Endian.
This page documents version 0.1 of the 3rd Eye Scene protocol. Changes to the major version number indicate a breaking change or an introduction of major features in the core protocol. Point version changes indicate minor changes such as the introduction of new flag values.
The protocol version number will be at least 1.0 on the initial release of the 3rd Eye Scene viewer regardless of whether or not the protocol contains actual changes from the previous version.
All messages begin with a standard packet header. This header begins with a common, 4-byte marker identifying the packet start, followed by the remaining header, a message payload and in most cases a 2-byte CRC. The packet header also contains protocol version details, routing and message IDs, payload size details and a small number of packet flags.
The routing and message IDs are used to identify how the packet is to be handled. Conceptually, the routing ID identifies the recipient while the message ID identifies the packet content. Routing IDs must uniquely identify the receiver, while message IDs have different meanings depending on the routing ID.
The packet header is layed out as follows (16-bytes).
| Datum | Byte Size | Description |
|---|---|---|
| Marker | 4 | Identifies the start of a packet. Always the hex value 0x03e55e30. |
| Major version | 2 | The major version of the message protocol. With the minor version, This identifies the packet header and message contents and layout. |
| Minor version | 2 | The minor version of the message protocol. |
| Routing ID | 2 | Identifies the message recipient. See Core Routing IDs. |
| Message ID | 2 | The message ID identifies the payload contents to the handler. |
| Payload size | 2 | Size of the payload after this header. Excludes header size and CRC bytes. |
| Payload offset | 1 | For future use. Identifies a byte offset after this header where the payload begins. Initially always zero, in future this may be used to put additional data into the packet header. |
| Flags | 1 | Packet flags. See below. |
The initial protocol version is 0.1.
The Flags member supports the following flags:
| Flag | Value | Description |
|---|---|---|
| No CRC | 1 | The packet does not include a CRC after the header and payload. |
The header is followed by the message payload separated by the number of bytes specified in Payload offset. The offset should always be zero in the original protocol with the message payload immediately following the packer header.
The payload is followed by a 2-byte CRC, unless the No CRC flag is set. This CRC is calculated over the entire packet header and message content.
The 3rd Eye Scene core supports the following routing IDs and message handlers.
| Name | RoutingID | Description |
|---|---|---|
| Null | 0 | An invalid routing ID. Not used. |
| Server Info | 1 | Handles messages about the server connection. |
| Control | 2 | Used for control messages such as frame refresh. |
| Collated Packet | 3 | Handles collated and and optionally compressed packet data. |
| Mesh | 4 | Handles mesh resource messages. See TODO. |
| Camera | 5 | Handles camera related messages. |
| Category | 6 | Handles category and related messages. |
| Material | 7 | Not implemented. Intended for future handling of material messages. |
| Shapes | 64+ | Shape handlers share core message structures, but identify different 3D shapes and primitives. |
Shape handlers all use a common message structure (see TODO) using the same create, update and destroy messages. Shapes support a data message for sending additional data specific to that shape. Shapes have the following routing IDs.
| Shape | RoutingID | Description |
|---|---|---|
| Sphere | 64 | A sphere primitive. |
| Box | 65 | A box primitive. |
| Cone | 66 | A cone primitive. |
| Cylinder | 67 | A cylinder primitive. |
| Capsule | 68 | A capsule primitive. A capsule is formed from two hemispheres connected by a cylinder. Often used as a physics primitive. |
| Plane | 69 | A 2D quadrilateral positioned in 3D space with a normal component.Used to represent 3D planes at a point. |
| Star | 70 | A star shape. |
| Arrow | 71 | An arrow shape made from a conical head and cylindrical body. |
| Mesh Shape | 72 | A single mesh object made up of vertices and indices with a specified draw type or topology. |
| Mesh Set | 73 | A collection of mesh objects created via Mesh Routing ID. A mesh set supports more complex mesh structures than Mesh Set allowing multiple shared mesh resources. Meshes themselves support vertex colour and vertex normals. See TODO "mesh resources". |
| Point Cloud | 74 | Similar to a mesh set, a point cloud shape supports a single, shared point cloud mesh resource. |
| Text 3D | 75 | Supports text with full 3D positioning and scale, with optional billboarding. |
| Text 2D | 76 | Supports 2D text either located in screen space, or located in 3D and projected into screen space. |
Server info messages provide information about the server. There is currently only one server message, message ID zero. It is detailed below.
| Datum | Byte Size | Description |
|---|---|---|
| Time Unit | 8 | Specifies the time unit used in frame messages. That is, each time unit in other messages is scaled by this value. This value is measured in microseconds and defaults to 1000us (1 milliscond). |
| Default Frame Time | 4 | Specifies the default time to display a frame for when an end frame message does not specify a time value (zero). Generally not relevant for real time visualisation as there should always be a real frame |
| | time in this case. This value is scaled by the Time Unit and defaults to 33(ms). Coordinate Frame | 1 | Specifies the server's coordinate frame. See below. Reserved | 35 | Additionaly bytes reserved for future expansion. These pad the structure to 48-bytes, which pads to 64-bytes when combined with the packet header. All bytes must be zero for correct CRC.
The coordinate frame identifies the server's basis axes. It is set from one of the following values. The coordinate frame name identifies the right, forward and up axes in turn. For example XYZ, specifies the X axis as right, Y as forward and Z as up and is right handed. Similarly XZY specifies X right, Z forward, Y up and is left handed. Some axes are prefixed with a '-' sign. This indicates the axis is flipped.
| Frame Name | Value | Left/Right Handed |
|---|---|---|
| XYZ | 0 | Right |
| XZ-Y | 1 | Right |
| YX-Z | 2 | Right |
| YZX | 3 | Right |
| ZXY | 4 | Right |
| ZY-X | 5 | Right |
| XY-Z | 6 | Left |
| XZY | 7 | Left |
| YXZ | 8 | Left |
| YZ-X | 9 | Left |
| ZX-Y | 10 | Left |
| ZYX | 11 | Left |
Control messages are special commands to the client. The most commonly used control message is the Frame message which identifies an "end of frame" event and causes the client to display expire transient shapes and dispaly new shapes shapes. All control messages have the same message structure, but the semantics of the message content vary.
| Datum | Byte Size | Description |
|---|---|---|
| Flags | 4 | Flags for the control message. Semantics vary. |
| Value 32 | 4 | A 32-bit value. Semantics vary. |
| Value 64 | 8 | A 64-bit value. Semantics vary. |
Valid control messages and their value and flag semantics are:
| Name | MessageID | Description |
|---|---|---|
| Null | 0 | An invalid message ID. Not used. |
| Frame | 1 | An end of frame message causing a frame flush. Uses flags (see TODO). Value 32 stores the frame time in the server time unit (see Server Info Messages) or zero to use the default frame time. |
| Frame Count | 3 | Value 32 specifies the total number of frames. Intended only for recorded file streams to identify the target frame count. |
| Force Frame Flush | 4 | Force a frame flush without advancing the frame number or time. Intended as an internal control message only on the client. Values not used. |
| Reset | 5 | Clear the scene, dropping all existing data. Values not used. |
Frame message flags are:
| Name | Value | Description |
|---|---|---|
| Persist | 1 | Request that transient objects persist and are not flushed this frame. |
Collated packet rounting has one valid message ID (zero), and identifies the packet payload as containing additional packet headers and packet data, optionally GZIP compressed. Collated packets serve two purposes:
The collated packet message has the following payload:
| Datum | Byte Size | Description |
|---|---|---|
| Flags | 2 | See Collated Packet Flags |
| Reserved | 2 | Reserved for future use (padding) |
| Uncompressed Bytes | 4 | Specifies the number of uncompressed bytes in the payload. This is the sum of all the payload's individual packets (excluding the message header). |
It is important to note that collated packet sizes are generally limited to less than 65535 bytes due to the packet payload size restriction with one important exception (see below). When a server creates collated packets, it is free to add as much data as possible to reach the payload limit, but must not contain partial packets. That is, a packet contained in a collated packet must be wholly contained in that packet regardless of compression settings.
There is an exception to the size limitation which caters to how a collated packet is used in file saving. File streams may contain a collated packet which exceeds the packet size limitation. To support this, it cannot have a CRC as the CRC location cannot be calculated based on a 16-bit payload size in the collated packet header. The collated packet can break the size restrictions by expressing that it contains more than 65535 uncompressed bytes.
Collated packet data begins immediately following the collated packet header and message (above) and is expressed as a series of valid packet headers with their corresponding message payloads. These bytes may be GZIP compressed as specified by the collated packet message flags. A collated packet reader knows it has reached the end of the collated data when it has processed a number of uncompressed bytes equal to Uncompressed Bytes as specified in the collated packet message (this includes CRCs if present for the contained packets). The next two bytes will either be the CRC for the collated packet message (if present) or the beginning of the next packet, as identified by the packet header marker bytes.
Collated packet message flags are:
| Name | Value | Description |
|---|---|---|
| Compress | 1 | Collated packet payload is GZIP compressed. Compression begins after the message structure. That is, neither packet header nor the message are compressed. |
Mesh resource messages are used to create and populate mesh resources for viewing by the client. These resources may be referenced in other messages such as those pertaining to the Mesh Set shape type. Mesh resources are identified by an unique mesh resource ID. This ID is only unique among other mesh resources. It is the server's responsibility to ensure unique resource ID assignment.
Valid mesh resource messages are:
| Name | Value | Description |
|---|---|---|
| Invalid | 0 | Not used. |
| Destroy | 1 | Message to destroy a mesh resource. |
| Create | 2 | Message to create a mesh resource. |
| Vertex | 3 | Transfer of 3D vertex data. |
| Index | 4 | Transfer of index data. |
| Vertex Colour | 5 | Transfer of per vertex colour data. |
| Normal | 6 | Transfer of per vertex normal data data. |
| UV | 7 | Transfer of per vertex UV data. Not supported yet. |
| Set Material | 8 | Identifies the material resource(s) for the mesh. Not supported yet. |
| Redefine | 9 | Prepares the mesh for modification. New vertex, per vertex or index data may be incoming. |
| Finalise | 10 | Finalises the mesh resource, making it ready for viewing. |
The vertex, index, vertex colour, normal and UV messages all share a common message header followed by specific data arrays (see below). All other messages are unique.
Instructs the client it may safely release a previously created mesh resource as identified by its unique mesh resource ID. This message may be sent with an invalid resource ID.
| Datum | Byte Size | Description |
|---|---|---|
| Resource ID | 4 | The resource ID of the mesh to destroy. |
Instructs the client to create a new mesh resource with the specified ID. The resource is not valid for use until a finalise message is sent.
| Datum | Byte Size | Description |
|---|---|---|
| Resource ID | 4 | The resource ID of the mesh to create. Zero is a valid ID. |
| Vertex Count | 4 | Specifies the number of vertices in the mesh. |
| Index Count | 4 | Specifies the number of indices in the mesh. |
| Topology | 1 | Identifies the mesh topology. See TODO |
| Tint | 4 | Mesh tint colour. See Colour Encoding . |
| Translation | 12 | 3 32-bit floats identifying the translation component of the overall mesh tranformation. |
| Rotation | 16 | 4 32-bit floats identifying the quaternion rotation of the overall mesh tranformation. |
| Scale | 12 | 3 32-bit floats identifying the scale component of the overall mesh transformation. |
The mesh topology details how the vertices and indices are interpreted. Valid values are:
| Name | Value | Description |
|---|---|---|
| Points | 0 | Vertices represent individual points. Indices are optional and |
| | each index referencing a single point. Lines | 1 | Vertices represent line end points. Indices must come in pairs. Triangles | 2 | Vertices represent triangles. Indices must come in triples. Quads | 3 | Vertices represent quadrilaterals. Indices must come in sets of four. Not supported yet.
This section details the message structure of transmitting mesh elements; vertices, indices, vertex colour, normals and UVs. All messages have the same structure:
| Datum | Byte Size | Description |
|---|---|---|
| Resource ID | 4 | The resource ID of the mesh to which the data belong. |
| Index Offset | 4 | Index offset for the the first element. See below. |
| Reserved | 2 | Reserved. Must be zero. |
| Count | 2 | Number of elements. |
| Elements | N * Count | Count data elements. |
The element byte size depends on what is being sent as follows:
| Message | Element Byte Size | Description |
|---|---|---|
| Vertex | 12 | Triples of 32-bit floats identifying 3D coordinates. |
| Vertex Colour | 4 | 32-bit integer colour. See Colour Encoding . |
| Index | 4 | 32-bit unsigned integer indices. |
| Normal | 12 | Triples of 32-bit floats identifying a vertex normal. |
| UV | 8 | Doubles of 32-bit floats identifying 2D UV coordinates. |
From this we see that each message identifies the target mesh resource, an index offset to start writing the new data, an element count for this message followed by the elements themselves. The message only supports a 2-byte count, which is likely exceeded by any complex mesh, so the offset identifies where to start writing. Consider a mesh with 80000 vertices. Each message can only transmit at most 65535 vertices because of the 2-byte count limit. The limit is actually much lower than this because of the overall packet payload byte limit of 65535. For the sake of this example, let us say that each packet can contain 30000 vertices. This will be split across three vertex messages. Each message identifies the vertex Count and the Index Offset which is the index of the first vertex in the packet. In this case, we have the following three messages:
Camera routing supports a single message (ID zero) which specifies settings for a camera view. This is an optional camera and the client can elect to ignore or override any camera message, or parts of a camera. However, this message can be used to set an initial focus, or to show what the server is viewing. Multiple cameras are supported and the client can elect to lock their view to any valid camera.
| Datum | Byte Size | Description |
|---|---|---|
| Camera ID | 1 | Identifies the camera. ID 255 is reserved for serialising details |
| | of the client's own camera when recording. Reserved | 3 | Three bytes of padding. Must be zero. Position | 12 | XYZ coordinate for the camera position. Each element is a 32-bit float. Facing | 12 | XYZ forward vector (normalised) identifying the camera facing relative to Position. Up | 12 | XYZ up vector (normalised) for the camera. Near | 4 | Near clip plane (32-bit float). Zero implies no change. Far | 4 | Far clip plane (32-bit float). Zero implies no change. FOV | 4 | Horizontal field of view in degrees. Zero implies no change.
Category messages are used to identify and control categories associated with shapes. These messages can be used to identyfing valid categories, give a category a name, build a category hierarchy or set the default state for a category.
There is currently one category message with ID of zero.
| Datum | Byte Size | Description |
|---|---|---|
| Category ID | 2 | Specifies the category ID. |
| Parent ID | 2 | Sets the parent of this category, building category hierarchy. Zero always implies no parent. |
| Default Active | 2 | Default the category to the active state? Must be either 1 (active) or 0 (inactive). |
| Name Length | 2 | Number of bytes in the name string, excluding the null terminator. |
| Name | Varies | The name string in UTF-8 encoding. Contains Name Length bytes. |
Material messages are not yet supported. They are intended for future expansion to detail material resources which may be used with mesh resources.
Shapes for the core of the 3rd Eye Scene message protocol. Shapes are used to visualise 3D geometry on the client. Shapes cover a range of routing IDs all with a common message structure. Shapes may also be extended to user defined routing IDs supporting the same message protocol. Shapes may be simple or complex. A simple shape is fully defined by its create message, while a complex shape requires additional data messages to be completed.
Shapes support four primary messages:
In all cases a shape may append additional data after the main message (essential for secshapemsgdata "data messages"). Details of how individual shapes vary are detailed in later sections.
Shapes may be persistent or transient. Persistent shapes require unique IDs - unique within the context of the shape's routing ID, not across all existing shapes - and persist until explicitly destroy. Transient shapes have no ID and are generally destroyed on the next frame (unless the end frame message includes the persist flag).
| Datum | Byte Size | Description |
|---|---|---|
| Object ID | 4 | Unique identifies the shape within the context of the current shape routing ID. Zero is reserved for transient shapes. |
| Category ID | 2 | Identifies the category to which the shape belongs. Zero is the catch-all default. |
| Flags | 2 | Shape creation flags. See below. |
| Reserved | 2 | Reserved. Must be zero. |
| Attributes | * | See Shape Attributes |
Shapes may support the following set of flags, though not all shapes will support all flags.
| Flag | Value | Description |
|---|---|---|
| Wireframe | 1 | The shape should be visualised using wirefame rendering if supported. |
| Transparent | 2 | The shape should be visualised as transparent. The colour alpha channel specifies the opacity. |
| User | 256 | Shape specific flags start here. |
Shape attributes are used in both shape create and update messages and contain the following data:
| Datum | Byte Size | Description |
|---|---|---|
| Colour | 4 | A four byte encoded colour for the same. See Colour Encoding . |
| Translation | 12 | 3 32-bit floats identifying the translation component of the overall shape transformation. |
| Rotation | 16 | Usage varies, but generally defines 4 32-bit floats identifying a rotation for the shape. |
| Scale | 12 | Usage varies, but generally defines 3 32-bit floats identifying the scale of the shape. |
Note: Individual shapes may treat rotation and scale differently. Spheres, for example, ignore rotation, while planes use rotation to store the plane normal. Spheres may only consider the first scale component, while cylinders interpret the first components as radius, radius (ignored), length.
Shape attributes include a Colour value to apply to the shape. Meshes also support a tint colour. All 3rd Eye Scene colour values are 32-bit, RGBA colour values with 8-bits per colour channel. The channels are ordered in low to high byte as red, green, blue, alpha.
This section lists any how specific shapes interpret the translation, rotation and scale components of their attributes and also any additional data appended to a create message. Any unlisted shapes interpret the translation, rotation and scale simply as is (rotation as a quaternion rotation) and append no data. Unless otherwise specifies, each shape's local pivot or origin is at the centre of the shape.
| Shape | Complex? | Variation |
|---|---|---|
| Sphere | No | Pivot is at the centre, rotation is ignored, scale[0] specifies the radius. |
| Box | No | Scale sets the box edge length. E.g., (1, 1, 1) makes a unit cube. |
| Cone | No | Pivot is at the apex. Default direction is (0, 0, 1) away from the apex. Scale 0 is the radius at the base (also in element 1). Scale 2 is the length away from the apex. |
| Cylinder | No | Default direction is (0, 0, 1). Scale 0 is the radius (also in element 1). Scale 2 is the length. |
| Capsule | No | Default direction is (0, 0, 1). Scale 0 is the cylinder and hemisphere radius (also in element 1). Scale 2 is the cylinder length (total length is length + 2 * radius). |
| Plane | No | Position defines a local pivot point to centre the quad on. Rotation defines the quaternion rotation away from the default 'normal' (0, 0, 1) Scale components 0, 2, define the quad size (both are equal) Scale component 1 specifies the length to render the normal with. |
| Star | No | Pivot is at the centre, rotation is ignored, scale[0] specifies the radius. |
| Arrow | No | Pivot is at the arrow base. Default direction is (0, 0, 1) away from the base. Scale 0 is the radius (of the arrow wall) (also in element 1). Scale 2 is the length. |
| Mesh Shape | Yes | Writes additional data. |
| Mesh Set | No | Writes additional data. |
| Point Cloud | Yes | Writes additional data. |
| Text 3D | No | Supports user flags. Writes additional data. Position may defaults to screen space with (0, 0) the upper left corer and (1, 1) the lower right (Z ignored). |
| Text 2D | No | Supports user flags. Writes additional data. |
The directional shapes (arrow, capsule, cone, cylinder, plane, text 3d) all use rotation as a quaternion rotation away from the default direction (0, 0, 1). It could have been overridden to specify a true direction vector. However, this was not done to better support a common set of get/set rotation methods with consistent semantics. A rotation is either ignored, or it is a quaternion rotation.
| Datum | Byte Size | Description |
|---|---|---|
| Vertex Count | 4 | Number of vertices in the mesh. Must match topology if there are not indices. |
| Index Count | 4 | Number of indices. Must match topology, or zero. Zero implies sequential vertex indexing. |
| Topology | 1 | See topology |
| Datum | Byte Size | Description |
|---|---|---|
| Number of Parts | 2 | Number of mesh resource parts to the mesh set. |
| Mesh Resource ID | 4 * | Mesh Resource ID for the next part. |
| Attributes | N * | Shape Attributes for the part. |
A Part ID and Attributes Are written for each part and appear a number of times equal to the Number of Parts.
| Datum | Byte Size | Description |
|---|---|---|
| Mesh Resource ID | 4 * | Mesh Resource ID for this point cloud. Shared with mesh resources (mind the topology). |
| Index Count | 4 | Optional index count used to index a subset of the vertices in the mesh resource. |
| Point Size | 1 | Optional override for the point rendering size. Zero to use the default. |
Text 2D/3D additional data:
| Datum | Byte Size | Description |
|---|---|---|
| Text Length | 2 | Number of bytes in the following text (excludes null terminator). |
| Text | N | UTF-8 encoded string with Text Length bytes. No null terminator. |
Text 2D shapes also support the following flags:
| Flag | Value | Description |
|---|---|---|
| World Space | 256 | Position is in world space to be projected onto the screen. |
Text 3D Flags:
| Flag | Value | Description |
|---|---|---|
| Screen Facing | 256 | Text should always face the screen (billboarding). |
Shape data messages are only sent for complex shapes. These are shapes which cannot be succinctly defined by their create message, even with additional data appended to the create message. All data messages share the following header, after which the data layout is entirely dependent on the shape type.
| Datum | Byte Size | Description |
|---|---|---|
| Object ID | 4 | ID of the target shape. Transient shapes are ID zero. |
Note: Transient shapes may be complex and require additional data messages. However, the message must have an object ID of zero, which cannot uniquely identify a shape. Clients can only assume that a data message targeting a transient shape pertains to the last transient shape created for that shape routing ID.
Mesh shapes write the following data messages:
| Datum | Byte Size | Description |
|---|---|---|
| Data Type ID | 2 | Identifies the data payload: vertices or indices. |
| Index Offset | 4 | Offset into the overall data array to start indexing at. See TODO |
| Element Count | 4 | Number of elements. |
| Elements | N * Count | Data elements follow. |
The index offset/count values behave in the same was as for meshes.
Vertex elements are 12 byte, single precision XYZ coordinates. Index elements are 4 byte unsigned integers.
Note: packet payload size restrictions may be ignored when serialising to disk.
Point cloud shapes can optionally write a set of indices which restrict what is viewed in the referenced point cloud mesh resource.
| Datum | Byte Size | Description |
|---|---|---|
| Index Offset | 4 | Offset into the overall data array to start indexing at. See TODO |
| Element Count | 4 | Number of elements. |
| Indices | 4 * Count | Index array. |
The index offset/count values behave in the same was as for meshes.
Note: packet payload size restrictions may be ignored when serialising to disk.
Shape udpate messages are used to adjust the core attributes of a shape. Most notably, update messages are used to reposition shapes in 3D space. Updates messages are only for persistent shapes. Update messages are formatted as follows:
| Datum | Byte Size | Description |
|---|---|---|
| Object ID | 4 | ID of the target shape. Transient (ID zero) do not support update messages. |
| Reserved | 2 | Was intended for object flags, but has been removed. Must be zero. |
| Attributes | * | See Shape Attributes |
Update messages generally do not support additional data, shape specific data, though this is not a hard restriction.
1.8.13