Intercognitive Foundation

We’re not going to get anywhere like this.

An off-the-shelf robot can cost six figures and be unable to answer basic questions.

Which way’s up? Forward? Is the camera publishing at 30 Hz or 60? Are your pixels RGB8, BGR8, or something else entirely? Which color space does it use? Is a rotation encoded as a quaternion in xyzw or wxyz order, a matrix, Euler angles, axis-angle vector, or a 6D representation? Which clock produced this timestamp? If an agent sends 0.5 to a gripper… what does that mean?

Before bots share bodies or data, they’ll have to be able to tell other machines what they sense, which frames and clocks they use, and how they encode motion.

People in the lab can excavate these answers from PDFs, driver code, sample applications, launch files, test rigs, and the fraying memories of the last engineer who integrated the robot. But it’s painful.

AI agents cannot reliably infer them.

Indeed, they should not be asked to.

At a recent meeting of the Intercognitive Co-Embodiment Working Group, several teams discovered that they had accumulated the same integration trauma (and accompanying scar tissue) independently.

One beleaguered lab had been forced to brute force test robots just to extract basic metadata. Another valiant crew had been juggling hundreds of bespoke model-environment adapters, a nightmare that grew in complexity with every new pairing.

The frustration was visceral and all-too-shared.

Saying, “I HAVE CAMERA,” is woefully insufficient.

Our plea: if a machine exposes a resource, it should also expose enough machine-readable metadata to interpret that resource easily and correctly.

Declaratory transparency.

That’s the whole game.

The problem is not the absence of standards

Robotics already has valuable conventions and schemas. We are not calling for a crusade to reinvent the wheel. At Intercognitive, we love existing wheels. We are pro wheel.

ROS REP-103 defines standard units and coordinate conventions, including the familiar camera optical frame: X = right, Y = down, Z = forward. The ROS CameraInfo message carries calibration and frame information. URDF and ros2_control describe joints, interfaces, and limits.

Other ecosystems have their own sound answers.

The thing driving us nuts is what happens at the boundary of a new robot, device, stream, model, or SDK.

A lot of the time, the roboticist simply cannot ask the system for a complete declaration of what is available and how to interpret it.

Sure, the answer may exist someplace, but not in a

1. form that is queryable by software,

2. pinned to the data,

3. versioned when it changes, and

4. legible to an agent.

This gap transforms what ought to be a convention into arcane tribal knowledge that requires a robot shaman to access.

This is not the way.

In our ideal world, every robot would use the same coordinate convention, frame rate, image encoding, rotation representation, clock, and actuator vocabulary.

But, alas, this is not our world. It would also be very difficult to achieve.

However, there is a much more modest departure point: a producer simply declares its convention, even when that convention differs from someone else’s.

The roboticist can easily convert known convention A into known convention B. It cannot safely convert unknown into unknown. That alchemy is beyond even our brightest minds. The same principle applies to image channels, units, clock domains, joint ordering, command ranges, and every other place where integrators currently have to guess.

Declare before you convert. That’s it.

This does not eliminate adapters. It makes adapters derivable.

Three verbs that should not be collapsed

Aiming to improve our collective mental health, we have clarified three separate, low-lift requests:

Describe: What is this resource, and what do its values mean?
Advertise: Which described resources can this machine provide right now, and where can they be requested?
Authorize: Who may access or command them, under which permissions and safety constraints?

Machine identity and signed credentials can tell us who made a declaration. Access control can determine who may open a stream or command an actuator. Neither tells a consumer anything useful about whether a four-element rotation is xyzw or wxyz.

Description is not authorization. But authorization without description only gives an agent permission to misunderstand the machine.

What a declaration can look like

Auki’s open-source SDK offers a concrete starting point.

Note: this is not a proposed universal schema. It is evidence that basic declaratory commitment is practical.

Auki’s declaration for an RGB camera:

{
"peer_id": "galbot",
"sensor_id": "head_left_rgb",
"kind": "camera",
"type": "rgb",
"width": 1920,
"height": 1200,
"frame_rate_hz": 30,
"pixel_format": "rgb8",
"color_space": "srgb",
"intrinsics_model": "pinhole",
"distortion_model": "brown_conrady",
"frame": {
"peer_id": "galbot",
"id": "head_left_camera_optical",
"hash": "3055a8ac27eecd57aa37235b05871c01"
}
}

“I HAVE CAMERA” has become: this peer has this specific RGB camera; it produces 1920 × 1200 rgb8 images at 30 Hz in sRGB; its intrinsics and distortion follow named models; and its output is expressed in a particular, versioned frame.

The referenced frame has its own declaration:

{
"peer_id": "galbot",
"frame_id": "head_left_camera_optical",
"axes": {
"x": "right",
"y": "down",
"z": "forward"
},
"handedness": "right",
"units": "meters"
}

Nobody has to infer what “optical” means from a name. No wasting precious lab time guessing coordinates. The convention is clear.

Auki then binds a data product to exact sensor, frame, and clock declarations:

{
"source_peer_id": "galbot",
"writer_peer_id": "galbot",
"app_id": "galbot-control-plane",
"session_id": "01HV-galbot-session",
"sensor": {
"peer_id": "galbot",
"id": "head_left_rgb",
"hash": "8295922307fa2b426453486ba87a59ef"
},
"clock": {
"peer_id": "galbot",
"id": "session/sdk_clock",
"hash": "fb0120a35f4fd1de8a7d46f5e76b7f68"
},
"frame": {
"peer_id": "galbot",
"id": "head_left_camera_optical",
"hash": "3055a8ac27eecd57aa37235b05871c01"
},
"segment_duration_ns": 1000000000,
"retention_ns": 5000000000
}

This composition matters.

Static metadata does not need to ride in every frame. It can be registered once, content-addressed, and fetched by reference. The data stays compact, its meaning remains recoverable.

Auki’s live resource catalog adds the advertisement layer: a peer can answer “what can you provide right now?” with stable resource identifiers and typed pointers back to the relevant declarations (unavailable resources are omitted until they can accept a stream request again).

Rotations demonstrate why guessing is unacceptable

There are many correct ways to encode a rotation. There are equally many ways to get the conversion subtly wrong.

RLMesh, an open-source model-evaluation framework from ArenaX Labs, currently recognizes six explicit rotation encodings in its adapter system:

quat_xyzw, quat_wxyz, axis_angle, rot6d, rot6d_rowmajor, and euler_xyz.

The distinction is operational, not academic. An environment can declare that its end-effector orientation is quat_xyzw and its rotation action is a three-dimensional axis_angle.

For example:

{
"observation": {
"eef_quat": {
"type": "state",
"role": "proprio/eef_rot",
"encoding": "quat_xyzw",
"range": null
}
},
"action": {
"components": [
{
"role": "action/delta_eef_rot",
"dim": 3,
"encoding": "axis_angle",
"range": null,
"binary": false
}
]
}
}

Now, the model can independently declare that it consumes or emits rot6d. RLMesh’s resolver can then derive the conversion. The two sides do not need to import one another or share homemade, bespoke glue.

This is the payoff of declaration: an adapter becomes a result of comparing two contracts instead of an artisanal piece of craftsmanship full of assumptions.

What should be declared?

Different resources need different fields, but to our mind a useful minimum profile should make several categories explicit:

Identity and version: a stable resource ID, the declaring peer or machine, and a version or content hash.
Semantics and shape: sensor or actuator kind, subtype, dimensions, ordering, data type, pixel or sample encoding, and units.
Space: frame ID, axis directions, handedness, length unit, rotation representation, and the transforms needed to relate the resource to a robot or world frame.
Time: expected rate, timestamp clock, clock scope or epoch, and any timing or synchronization guarantees.
Provenance and availability: source, writer, application, session, retention, current availability, and where the resource can be requested.
Actuation contract: command role, control mode, dimensionality, frame, units, valid range, update rate, limits, clipping behavior, and the safety or session boundary under which commands are accepted.

Not every field belongs in every declaration. Unknown values should be representable as unknown. Extensions should be possible without breaking older readers.

But a useful standard will explicitly define minimal profiles, richer optional profiles, and explicit failure behavior instead of necessitating guesswork.

Agent-readable means more than “JSON-shaped”

The list above describes what a declaration should contain. But content alone is not enough. A blob of JSON is not automatically a contract.

For a declaration to be useful to an agent, it should be:

semantically explicit rather than dependent on suggestive names;
versioned or content-addressed;
discoverable through a documented interface;
linked to the data or command surface it describes;
validated against a schema and test vectors;
extensible without making typos indistinguishable from new fields; and
attributable to the machine, vendor, or software that made the claim.

Declarations are still claims. They do not prove that a camera is calibrated correctly, that a frame is truthful, or that an actuator is safe. Identity, signatures, certification, runtime checks, and safety systems remain necessary.

But those systems need a claim to evaluate. Opaqueness is not assurance.

The ask

We are not asking the robotics ecosystem to adopt Auki’s SDK, RLMesh, ROS, or any single vendor’s ontology wholesale.

We are asking robot makers, sensor vendors, model builders, middleware developers, and physical-AI platforms to make a smaller commitment:

“We will declare the metadata required to interpret and use the resources we expose.”

The first community deliverable can be modest:

camera, general sensor, frame, rotation, actuator, and data-product examples;
minimal and recommended fields for each;
JSON Schemas with valid and invalid fixtures;
versioning and extension rules;
conformance tests across at least two independent implementations; and
mappings to working systems rather than replacements for them.

This is a shared, public trauma so the Co-Embodiment Working Group is opening this conversation in public. If you have built an integration, tell us which undeclared assumption cost you in time. If your system already exposes a rigorous capability manifest, tell us about it.

Better yet, show us.

If a field in the examples above is wrong or incomplete, propose a better one.

Reject Kings, Presidents, and Voting. Bring us the rough consensus and running code.

Co-embodiment is the ambitious destination: multiple intelligences securely sharing sensors, context, compute, and actuators in the physical world. Declaratory transparency is the much smaller step and the giant leap for roboticists everywhere.

Making “the physical world accessible to AI” begins here: making the physical world legible to AI.

If we’re going to get anywhere worth going, this has to be part of it.

‍

A Plea for Declaratory Transparency in Physical AI

The problem is not the absence of standards

Three verbs that should not be collapsed

What a declaration can look like

Rotations demonstrate why guessing is unacceptable

What should be declared?

Agent-readable means more than “JSON-shaped”

The ask

Further Reading

Members

Principles

Useful Links

Follow us