Skip to main content
This guide explains how dynamic User Interface (UI) tools work in MeshAgent. These elements are client-side tools that run on a user’s browser, desktop, or mobile app. Dynamic UIs enable agents to interact directly with users by presenting responsive, on-demand interfaces as needed. They allow agents to do things like:
  • Collect a document from the user in the middle of a task
  • Pop up a one-off approval dialog
  • Fan out a survey to multiple participants in the room

​
Why Dynamic UI Tools Matter

Problem in multi-user agent appsHow a Dynamic UI Tool solves it
Mid-task input: An agent needs extra input during a task (e.g. “Which PDF should I summarise?”).Pop a file-picker on only the requester’s screen, collect the answer, continue the run.
Room wide polls/surveys: You want a one-off survey of everyone currently in the Room.Allow the agent to fan-out an ask_user dialog to each participant and gather the responses asynchronously.
Targeted notifications: Agents must present notifications, approvals, or error messages only to the relevant user—not the whole Room.Each UI toolkit is automatically scoped to the participant who registered it, so dialogs can’t leak across screens.
Security / Phishing: Users should not be able to show dialogs on another user’s screen just because a tool to show dialogs is exposed.Toolkits are scoped to the participant that registers them. An agent must pass participant_id when it calls invoke_tool, so dialogs can’t leak to other users.

​
How MeshAgent Safely Routes UI Tool calls

Behind the scenes MeshAgent supports private tool registrations. This ensures that dialogues only appear on the intended user’s screen even if several clients register toolkits with the same name. For example, you might have multiple ui toolkits: one with an ask_user tool that shows dialogues on a mobile app, and another with an ask_user tool that shows a dialogue in the browser. When a user interface registers a private tool, that tool is accessible only to the registering user. An agent can then invoke this private tool by including the unique identifier of the user (the participant ID) in its call. This ensures that the interface is displayed solely to the intended recipient.

​
Example: Survey Room Participants

Let’s create a tool that surveys participants in the Room. The tool will conduct a survey of the participants, summarize the results, and store both the raw results and the summary to the Room storage. The survey fields are dynamically generated so that we can gather a variety of information using the same UI tools. This means we can use the same tool to conduct a survey where participants respond yes/no to a question, or to provide more detailed feedback on their experience, etc. 
import asyncio
import json
import logging

from meshagent.api.messaging import JsonContent, TextContent
from meshagent.api.room_server_client import RoomClient
from meshagent.agents import SingleRoomAgent
from meshagent.agents.llmrunner import LLMTaskRunner
from meshagent.otel import otel_config
from meshagent.openai import OpenAIResponsesAdapter
from meshagent.tools import LocalRoomTool, ToolContext, Toolkit

otel_config(service_name="my-service")
log = logging.getLogger("my-service")


async def save_to_storage(room: RoomClient, path: str, data: bytes):
    await room.storage.upload(path=path, data=data)


class Survey(LocalRoomTool):
    def __init__(self, *, room: RoomClient):
        super().__init__(
            room=room,
            name="survey",
            title="survey",
            description="a tool that conducts a survey of the participants",
            input_schema={
                "type": "object",
                "additionalProperties": False,
                "required": ["subject", "description", "name"],
                "properties": {
                    "subject": {
                        "type": "string",
                        "description": "The subject of the form",
                    },
                    "description": {
                        "type": "string",
                        "description": "The content to fill in (e.g. feedback, poll result)",
                    },
                    "name": {
                        "type": "string",
                        "description": "A short name to be used on the form",
                    },
                },
            },
        )

    async def execute(
        self, context: ToolContext, subject: str, description: str, name: str
    ):
        room = self.room
        participants = await self._wait_for_user_participants(room)
        log.info("Starting survey for %d participant(s)", len(participants))
        if not participants:
            return TextContent(
                text=(
                    "No user participants are available to survey. Open a MeshAgent UI "
                    "client in this room, confirm it is registered as a user participant, "
                    "and invoke the survey again."
                )
            )

        MAX_ATTEMPTS = 2

        async def ask_participant(p):
            errors = []
            for attempt in range(1, MAX_ATTEMPTS + 1):
                try:
                    log.info("→ ask_user attempt %d --> %s", attempt, p.id)
                    resp = await room.agents.invoke_tool(
                        toolkit="ui",
                        tool="ask_user",
                        participant_id=p.id,
                        input={
                            "subject": subject,
                            "help": description,
                            "form": [
                                {
                                    "input": {
                                        "multiline": False,
                                        "name": name,
                                        "description": description,
                                        "default_value": "",
                                    },
                                },
                            ],
                        },
                    )
                    answer = resp.json.get(name)
                    if answer:
                        log.info("participant_id", p.id, "response", answer)
                        return {"participant_id": p.id, "response": answer}
                    raise RuntimeError("empty or timed-out response")

                except Exception as exc:
                    errors.append(f"attempt {attempt}: {exc}")
                    if attempt < MAX_ATTEMPTS:
                        log.info("Retrying %s after: %s", p.id, exc)
                        await asyncio.sleep(1)  # brief back-off

            # All attempts failed – return aggregated error list
            return {"participant_id": p.id, "errors": errors}

        log.info("Surveying participants")
        tasks = [asyncio.create_task(ask_participant(p)) for p in participants]
        results = await asyncio.gather(*tasks)

        summary = {
            "meta": {  # save the prompt generated for the survey
                "subject": subject,
                "description": description,
                "name": name,
            },
            "success": {},
            "failed": {},
        }
        for item in results:
            pid = item["participant_id"]
            if "response" in item:
                summary["success"][pid] = item["response"]
            else:
                summary["failed"][pid] = item["errors"]

        # write survey results to the room
        log.info("Survey completed, writing raw results to Room storage")
        await save_to_storage(
            room=room,
            path=f"survey/{room.room_name}-{name}.json",
            data=json.dumps({"summary": summary}, indent=2).encode("utf-8"),
        )

        # summarize results
        log.info("Summarizing survey results")
        summary_schema = {
            "type": "object",
            "properties": {"summary": {"type": "string"}},
            "required": ["summary"],
            "additionalProperties": False,
        }
        runner = LLMTaskRunner(
            llm_adapter=OpenAIResponsesAdapter(),
            output_schema=summary_schema,
        )
        summary_resp = await runner.run(
            room=room,
            arguments={
                "prompt": f"Summarize these survey results:\n{json.dumps(summary)}",
                "model": None,
            },
            caller=context.caller,
        )
        if isinstance(summary_resp, JsonContent):
            summary_text = summary_resp.json.get("summary") or summary_resp.json.get(
                "result", ""
            )
        elif isinstance(summary_resp, TextContent):
            summary_text = summary_resp.text
        else:
            summary_text = str(summary_resp)
        log.info("Saving survey result summary")
        await save_to_storage(
            room=room,
            path=f"survey/{room.room_name}-{name}-summary.doc",
            data=summary_text.encode("utf-8"),
        )

        return TextContent(text=summary_text)

    async def _wait_for_user_participants(self, room: RoomClient, timeout: float = 5.0):
        deadline = asyncio.get_running_loop().time() + timeout
        while True:
            participants = [
                p for p in room.messaging.remote_participants if p.role == "user"
            ]
            if participants:
                return participants
            if asyncio.get_running_loop().time() >= deadline:
                return []
            await asyncio.sleep(0.25)


class SurveyToolkit(Toolkit):
    def __init__(self, *, room: RoomClient):
        super().__init__(
            name="survey-toolkit",
            title="survey-toolkit",
            description="a toolkit for conducting a survey",
            tools=[Survey(room=room)],
        )


class SurveyAgent(SingleRoomAgent):
    async def start(self, *, room: RoomClient) -> None:
        await room.messaging.enable()
        await super().start(room=room)

    async def get_exposed_toolkits(self) -> list[Toolkit]:
        return [SurveyToolkit(room=self.room)]


async def main() -> None:
    agent = SurveyAgent(title="survey-toolkit-host")
    await agent.run()


if __name__ == "__main__":
    asyncio.run(main())
Copy this code, activate your virtual environment, and run the toolkit in the gettingstarted room. Leave this room connect process running; it is the process that hosts survey-toolkit and watches for messaging-enabled user participants.
bash
meshagent setup # this will prompt you to authenticate to MeshAgent, select your project and API keys
meshagent rooms create gettingstarted --if-not-exists
meshagent room connect --room=gettingstarted --identity=survey-toolkit -- python3 private-tool-call.py
Next go to studio.meshagent.com and click into the gettingstarted room. You can also use a Powerboards room page. The survey toolkit looks for messaging-enabled room participants with the user role and then invokes that participant’s private ui.ask_user tool. Before invoking the survey, confirm the room sees a user participant. This command joins the room as a temporary CLI participant, enables messaging for that CLI participant, and lists the other messaging-enabled participants it discovers:
bash
meshagent room messaging list --room=gettingstarted
Copy the id for the user participant from that output. Then confirm the room sees the public survey-toolkit and, for that user participant, a private ui toolkit:
bash
meshagent room agents list-toolkits --room=gettingstarted
meshagent room agents list-toolkits --room=gettingstarted --participant-id <participant-id>
If meshagent room messaging list returns only agents or [], the toolkit host does not currently see a user participant it can survey. Make sure the Studio or Powerboards room page is open in the same MeshAgent project and room. In that state no browser UI will appear, even if the survey tool itself is reachable. Invoke the survey tool from a separate terminal:
bash
meshagent room agents invoke-tool \
  --room gettingstarted \
  --toolkit survey-toolkit \
  --tool survey \
  --arguments '{"subject":"Hiring feedback","description":"What did you think of the candidate?","name":"feedback"}'
Connected user participants will be prompted to fill out the survey in their UI client. Once the results are captured, the tool summarizes the responses and saves both the raw results and summary to Room storage.

​
Next Steps: Using Dynamic Tools in Your Own App

MeshAgent UI client libraries include a private ui toolkit pattern with tools such as ask_user and ask_user_for_file. A connected UI client must register that toolkit for its participant before another agent can invoke it. If you want custom dialogs in your own web or mobile client, register a private toolkit named ui with the same tool names and schemas. When an agent invokes the tool with that participant’s ID, MeshAgent routes the call to your application’s client-side UI implementation.