What an agent is

An agent is not the robot. The physical system stays what it always is in DimOS: modules exchanging streams, composed by a blueprint. An agent is one more module in that blueprint - an LLM loop that listens for human text, picks tools, and calls them. The tools are ordinary module methods marked with @skill. “Agent native” means robot capabilities are toolized for an LLM, not that the whole OS is an agent. Skills are the product surface; the agent is one client of them. You are another: every skill can be called from the CLI with no LLM involved.

The control loop

You ──→ human_input          (dimos agent-send / web chat / humancli)
          │
          ▼
       McpClient             (LLM loop, default gpt-4o)
          │  tools/list, tools/call over HTTP
          ▼
       McpServer             (MCP tool bus, port 9990)
          │  RPC
          ▼
       Skill containers      (navigation, sport commands, TTS, ...)
          │
          ▼
       str | image result ──→ back to the LLM ──→ next step or reply

The agent does not continuously subscribe to camera, LiDAR, or odometry streams. Perception reaches the LLM pull-based, through skills: when the agent wants to see, it calls a skill like observe() and gets an image back as a tool result. This keeps the LLM loop cheap and makes every observation explicit in the conversation history. McpClient (dimos/agents/mcp/mcp_client.py) is a Module with exactly three streams:

human_input: In[str] - text from dimos agent-send, the web chat, or humancli
agent: Out[BaseMessage] - the agent’s responses (text, tool calls, images)
agent_idle: Out[bool] - signals when the agent is waiting for input

At startup it connects to McpServer, lists the available tools, and exposes them to the LLM. McpServer in turn discovers every @skill-annotated method across all deployed modules via RPC.

Vocabulary

Term	What it is
Module	Autonomous process with typed `In`/`Out` streams and RPC methods
Blueprint	Composition of modules; what `dimos run` starts
Skill	Module method decorated `@skill`; its docstring and type hints become the tool schema
Agent (`McpClient`)	LLM module: conversation history + tools + system prompt
MCP (`McpServer`)	HTTP tool bus (port 9990) listing and calling all skills
System prompt	Robot-specific policy; the Go2 and G1 ship different ones

A real agentic stack

dimos run unitree-go2-agentic composes (from dimos/robot/unitree/go2/blueprints/agentic/):

unitree-go2-agentic
├── unitree-go2-spatial          # robot + SpatialMemory + perceive loop
├── McpServer                    # HTTP tool bus on :9990
├── McpClient                    # the LLM loop
├── UnitreeSkillContainer        # relative_move, sport commands, wait
├── NavigationSkillContainer     # navigate_with_text, tag_location, stop_navigation
├── PersonFollowSkillContainer   # follow_person, stop_following
├── SpeakSkill                   # text-to-speech
└── WebInput                     # web chat + voice at localhost:5555

The system prompt gives the Go2 agent its persona (“Daneel”, dimos/agents/system_prompt.py) and, more importantly, policy matched to the skills that actually exist on this robot. A G1 stack ships a different prompt because it ships different skills.

Talking to the agent

Method	How it works
`dimos agent-send "text"`	One-shot CLI message
Web chat	`http://localhost:5555`, with optional Whisper voice input
`humancli`	Interactive terminal session

Models

Config	Model	Notes
Default	`gpt-4o`	Requires `OPENAI_API_KEY`
Ollama	local models	`unitree-go2-agentic-ollama` blueprint; `ollama serve` must be running
Custom	any LangChain-compatible model string	`McpClient.blueprint(model="...")`

Where to go next

Skills & MCP - the @skill contract and driving skills without an LLM
Tutorial: drive the Go2 with language - tag places, navigate by description
Tutorial: add your own skill - from a Python method to a tool the agent calls

Getting Started

Build with DimOS

Agents

Capabilities

Platforms

Going deeper

Contribute

The control loop

Vocabulary

A real agentic stack

Talking to the agent

Models

Where to go next

​The control loop

​Vocabulary

​A real agentic stack

​Talking to the agent

​Models

​Where to go next

The control loop

Vocabulary

A real agentic stack

Talking to the agent

Models

Where to go next