Chat Tools - Docs

Overview

Chat Tools allow your app to provide custom functions that become available in Omi chat when users install your app. These tools enable the AI assistant to interact with your service directly through natural language conversations.

Custom Actions

Define tools that perform actions on external services

Natural Language

Users interact with tools through normal conversation

Automatic Discovery

Tools become available when users install your app

Example: A Slack integration app might provide tools to:

Send messages to Slack channels
List available channels
Search messages in Slack

When a user installs your app, these tools automatically become available in their Omi chat, allowing them to say things like “Send a message to #general saying hello” or “What Slack channels do I have?”

Chat Tools require your app to have the external_integration capability enabled. They are not a separate capability, but rather a feature that works with external integration apps.

How Chat Tools Work

User Chats with Omi

User makes a request in natural language

AI Decides to Use Tool

Omi’s AI determines your tool is the best way to fulfill the request

Omi Calls Your Endpoint

POST request with uid, app_id, tool_name, plus extracted parameters

Your Server Processes

Execute the action and return a result

User Sees Result

Omi displays your response to the user

Part 1: Implementing Chat Tools

Step 1: Create Tool Endpoints

Your backend server needs to expose endpoints that Omi will call when tools are invoked.

Endpoint Requirements

Your endpoints should:

Accept POST requests with JSON payload (or GET with query parameters)
Receive these standard fields:
- uid: User ID
- app_id: Your app ID
- tool_name: Name of the tool being called
- geolocation (optional): User’s cached location, see Geolocation below
- Plus any tool-specific parameters
Return JSON with either:
- result: Success message (string)
- error: Error message (string)

Example: Send Message Tool

from flask import Flask, request, jsonify
import requests

app = Flask(__name__)

@app.route('/api/send_message', methods=['POST'])
def send_message():
    """
    Tool endpoint: Send a message to a Slack channel

    Expected payload:
    {
        "uid": "user_id",
        "app_id": "slack_app_id",
        "tool_name": "send_slack_message",
        "channel": "#general",
        "message": "Hello from Omi!"
    }
    """
    data = request.json

    # Validate required parameters
    if not data:
        return jsonify({'error': 'Missing request body'}), 400

    uid = data.get('uid')
    channel = data.get('channel')
    message = data.get('message')

    if not uid:
        return jsonify({'error': 'Missing uid parameter'}), 400
    if not channel:
        return jsonify({'error': 'Missing required parameter: channel'}), 400
    if not message:
        return jsonify({'error': 'Missing required parameter: message'}), 400

    # Get user's authentication token (from your database)
    slack_token = get_user_token(uid)
    if not slack_token:
        return jsonify({
            'error': 'Slack not connected. Please connect your Slack account.'
        }), 401

    # Call external API (e.g., Slack API)
    slack_response = requests.post(
        'https://slack.com/api/chat.postMessage',
        headers={'Authorization': f'Bearer {slack_token}'},
        json={'channel': channel, 'text': message}
    )

    if slack_response.json().get('ok'):
        return jsonify({
            'result': f'Successfully sent message to {channel}'
        })
    else:
        return jsonify({
            'error': f"Failed to send message: {slack_response.json().get('error')}"
        }), 400

Example: Search Tool

@app.route('/api/search_messages', methods=['POST'])
def search_messages():
    """
    Tool endpoint: Search for messages

    Expected payload:
    {
        "uid": "user_id",
        "app_id": "slack_app_id",
        "tool_name": "search_slack_messages",
        "query": "meeting notes",
        "channel": "#general"  # optional
    }
    """
    data = request.json

    uid = data.get('uid')
    query = data.get('query')
    channel = data.get('channel')

    if not uid:
        return jsonify({'error': 'Missing uid parameter'}), 400
    if not query:
        return jsonify({'error': 'Missing required parameter: query'}), 400

    # Get user's authentication token
    slack_token = get_user_token(uid)
    if not slack_token:
        return jsonify({
            'error': 'Slack not connected. Please connect your Slack account.'
        }), 401

    # Build search query
    search_query = query
    if channel:
        search_query = f'in:{channel} {query}'

    # Call external API
    slack_response = requests.get(
        'https://slack.com/api/search.messages',
        headers={'Authorization': f'Bearer {slack_token}'},
        params={'query': search_query}
    )

    if slack_response.json().get('ok'):
        messages = slack_response.json().get('messages', {}).get('matches', [])
        if not messages:
            return jsonify({
                'result': f'No messages found for "{query}"'
            })

        results = []
        for msg in messages[:5]:
            results.append(f"- {msg.get('text', '')[:100]} (in #{msg.get('channel', {}).get('name', 'unknown')})")

        return jsonify({
            'result': f'Found {len(messages)} messages:\n' + '\n'.join(results)
        })
    else:
        return jsonify({
            'error': f"Failed to search messages: {slack_response.json().get('error')}"
        }), 400

Step 2: Handle Authentication

If your tools require user authentication:

Store Tokens Securely

Save OAuth tokens in a secure database associated with uid

Validate Authentication

Check if user has connected their account before processing

Return Helpful Errors

If auth is missing, return a clear error message

def get_user_token(uid: str) -> Optional[str]:
    """Get user's authentication token from database"""
    # In production, fetch from secure database
    return user_tokens.get(uid)

@app.route('/api/send_message', methods=['POST'])
def send_message():
    data = request.json
    uid = data.get('uid')

    # Check authentication
    token = get_user_token(uid)
    if not token:
        return jsonify({
            'error': 'Account not connected. Please connect your account in app settings.'
        }), 401

    # Proceed with tool execution
    # ...

Step 3: Error Handling Best Practices

Always return helpful error messages that guide users on how to fix the issue.

# Good error messages
return jsonify({'error': 'Slack not connected. Please connect your Slack account.'}), 401
return jsonify({'error': 'Missing required parameter: channel'}), 400
return jsonify({'error': 'Channel not found. Please check the channel name.'}), 404

# Avoid exposing sensitive information
# ❌ Bad: return jsonify({'error': f'Database connection failed: {db_password}'}), 500
# ✅ Good: return jsonify({'error': 'Service temporarily unavailable. Please try again later.'}), 500

Part 2: Adding Chat Tools in Omi App Store

Chat tools are defined via a manifest endpoint hosted on your server. When you create or update your app in the Omi App Store, Omi will automatically fetch the tool definitions from your manifest URL.

Single Source of Truth

Tool definitions live on your server

Easy Updates

Modify your manifest and re-save the app to refresh

Full Control

Define parameters with JSON Schema

Version Control

Track changes in your codebase

Step 1: Create the Manifest File

Create a JSON file at /.well-known/omi-tools.json on your server:

{
  "tools": [
    {
      "name": "send_slack_message",
      "description": "Send a message to a Slack channel. Use this when the user wants to send a message, post an update, or notify a channel in Slack.",
      "endpoint": "/api/send_message",
      "method": "POST",
      "parameters": {
        "properties": {
          "channel": {
            "type": "string",
            "description": "The Slack channel to send to (e.g., '#general')"
          },
          "message": {
            "type": "string",
            "description": "The message text to send"
          }
        },
        "required": ["channel", "message"]
      },
      "auth_required": true,
      "status_message": "Sending message to Slack..."
    },
    {
      "name": "search_slack_messages",
      "description": "Search for messages in Slack. Use this when the user wants to find specific messages or look up past conversations.",
      "endpoint": "/api/search_messages",
      "method": "POST",
      "parameters": {
        "properties": {
          "query": {
            "type": "string",
            "description": "Search query"
          },
          "channel": {
            "type": "string",
            "description": "Optional channel to search in"
          }
        },
        "required": ["query"]
      },
      "auth_required": true,
      "status_message": "Searching Slack..."
    }
  ]
}

Step 2: Host the Manifest

Make sure your manifest is accessible via HTTPS. Common locations:

https://your-app.com/.well-known/omi-tools.json (recommended)
https://your-app.com/omi/manifest.json
https://your-app.com/api/tools-manifest

Step 3: Add Manifest URL in Omi App Store

Open Omi App

Navigate to Apps → Create App (or edit an existing app)

Select Capability

Choose External Integration capability

Enter Manifest URL

In the Chat Tools Manifest URL field, enter:

https://your-app.com/.well-known/omi-tools.json

Submit

Omi will automatically fetch your tool definitions

Endpoint URLs in Manifest: You can use relative paths (e.g., /api/send_message) in your manifest. Omi will automatically resolve them using your App Home URL as the base URL.

Manifest Schema Reference

Field	Type	Required	Description
`name`	string	Yes	Unique tool identifier (e.g., `send_slack_message`)
`description`	string	Yes	Detailed description for the AI to understand when/how to use the tool
`endpoint`	string	Yes	URL endpoint (can be relative or absolute)
`method`	string	No	HTTP method (default: `POST`)
`parameters`	object	No	JSON Schema defining tool parameters
`auth_required`	boolean	No	Whether user auth is required (default: `true`)
`status_message`	string	No	Message shown to user when tool is called

Parameters Schema

{
  "parameters": {
    "properties": {
      "param_name": {
        "type": "string | integer | boolean | array | object",
        "description": "Description of what this parameter does"
      }
    },
    "required": ["param_name"]
  }
}

Example: FastAPI Manifest Endpoint

from fastapi import FastAPI

app = FastAPI()

@app.get("/.well-known/omi-tools.json")
async def get_omi_tools_manifest():
    return {
        "tools": [
            {
                "name": "search_songs",
                "description": "Search for songs on Spotify",
                "endpoint": "/tools/search_songs",
                "method": "POST",
                "parameters": {
                    "properties": {
                        "query": {"type": "string", "description": "Search query"},
                        "limit": {"type": "integer", "description": "Max results"}
                    },
                    "required": ["query"]
                },
                "auth_required": True,
                "status_message": "Searching Spotify..."
            },
            {
                "name": "add_to_playlist",
                "description": "Add a song to a playlist",
                "endpoint": "/tools/add_to_playlist",
                "method": "POST",
                "parameters": {
                    "properties": {
                        "song_name": {"type": "string", "description": "Song name"},
                        "artist_name": {"type": "string", "description": "Artist name"},
                        "playlist_name": {"type": "string", "description": "Playlist name"}
                    },
                    "required": ["song_name"]
                },
                "auth_required": True,
                "status_message": "Adding to playlist..."
            }
        ],
        "chat_messages": {
            "enabled": True,
            "target": "app",
            "notify": False
        }
    }

Sending Chat Messages Proactively

Your app can send messages directly to a user’s chat without waiting for them to invoke a tool. This is useful for:

Sending results from long-running background tasks
Proactive notifications based on external events
Asynchronous updates from your service

Enabling Chat Messages

Add the chat_messages configuration to your manifest:

{
  "tools": [...],
  "chat_messages": {
    "enabled": true,
    "target": "app",
    "notify": false
  }
}

Field	Type	Default	Description
`enabled`	boolean	`false`	Whether your app can send chat messages
`target`	string	`"app"`	Where messages appear: `"app"` (app-specific chat) or `"main"` (main chat)
`notify`	boolean	`false`	Whether to send a push notification with the message

Sending Messages via API

Once enabled, use the notification API to send messages:

curl -X POST https://api.omi.me/v1/integrations/notification \
  -H "Authorization: Bearer YOUR_APP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "uid": "user_firebase_id",
    "aid": "your_app_id",
    "message": "Your task has completed! Here are the results..."
  }'

API Request Fields

Field	Type	Required	Description
`uid`	string	Yes	The user’s Firebase UID
`aid`	string	Yes	Your app’s ID
`message`	string	Yes	The message content to send

Example: Background Task Completion

import httpx

async def send_task_result(uid: str, result: str):
    """Send task completion message to user's chat."""
    async with httpx.AsyncClient() as client:
        response = await client.post(
            "https://api.omi.me/v1/integrations/notification",
            headers={
                "Authorization": f"Bearer {APP_API_KEY}",
                "Content-Type": "application/json",
            },
            json={
                "uid": uid,
                "aid": APP_ID,
                "message": result,
            },
        )
        return response.status_code == 200

Rate Limits

Chat messages are rate-limited to prevent notification fatigue:

Maximum: 10 messages per hour per app per user
Response: HTTP 429 with Retry-After header when exceeded

Abuse of chat messages (spam, excessive notifications) may result in your app being removed from the store.

Tool Definition Best Practices

Tool Names

Use descriptive, action-oriented names:

Good	Bad
`send_slack_message`	`slack1`
`list_slack_channels`	`do_stuff`
`search_slack_messages`	`msg`

Descriptions

Write detailed descriptions that help the AI understand:

When to use: “Use this when the user wants to…”
Required parameters: List what’s needed
What it does: Clear explanation of the action

Example:

Send a message to a Slack channel. Use this when the user wants to
send a message, post an update, or notify a channel in Slack.
Required parameters: channel (e.g., '#general') and message (the text to send).

HTTP Methods

Choose the appropriate method:

Method	Use For
`POST`	Creating resources or sending data (most common)
`GET`	Retrieving data (parameters as query params)
`PUT/PATCH`	Updating resources
`DELETE`	Deleting resources

Status Messages

Provide custom messages shown during tool execution:

“Searching Slack…”
“Sending message…”
“Creating calendar event…”

If not provided, Omi generates a default based on the tool name.

Request and Response Format

Request Format

POST Request:

{
  "uid": "user_firebase_id",
  "app_id": "your_app_id",
  "tool_name": "send_slack_message",
  "channel": "#general",
  "message": "Hello from Omi!",
  "geolocation": {
    "latitude": 30.2672,
    "longitude": -97.7431,
    "address": "123 Main St, Austin, TX 78701",
    "google_place_id": "ChIJLwPMoJm1RIYRetVp1EtGm10",
    "location_type": "approximate"
  }
}

GET Request:

GET /api/search?uid=user_id&app_id=app_id&tool_name=search_slack_messages&query=meeting

Geolocation

Tool call payloads include an optional geolocation field when the user has shared their location with Omi. Use it to make your tool location-aware (e.g., tag entries with where the user was, find nearby stores, localize search results).

Field	Type	Description
`latitude`	float	User’s latitude in decimal degrees
`longitude`	float	User’s longitude in decimal degrees
`address`	string \| null	Reverse-geocoded human-readable address
`google_place_id`	string \| null	Google Maps place identifier
`location_type`	string \| null	How the location was determined (e.g., `approximate`)

Backward compatible: The geolocation field is only present when location data is available. If the user has never granted location permission or their cached location has expired, the field is omitted entirely from the payload — it’s not set to null. Always treat it as optional when parsing.

@app.route('/api/add_product_entry', methods=['POST'])
def add_product_entry():
    data = request.json
    geolocation = data.get('geolocation')  # may be None

    store_location = None
    if geolocation:
        store_location = {
            'lat': geolocation['latitude'],
            'lng': geolocation['longitude'],
            'address': geolocation.get('address'),
        }

    # ... save entry with store_location ...

Location data is cached server-side for 30 minutes. If the user moves significantly, your tool will receive updated coordinates on the next call.

Response Format

Success
Error

{
  "result": "Successfully sent message to #general"
}

HTTP Status: 200

{
  "error": "Slack not connected. Please connect your Slack account."
}

HTTP Status: 400, 401, or 500

Best Practices

Clear Responses

Keep responses concise but informative

✅ “Successfully sent message to #general”
❌ “OK”

Helpful Errors

Guide users on how to fix issues

✅ “Slack not connected. Please connect your account.”
❌ “Error 401”

Secure Auth

Store tokens encrypted and implement token refresh for long sessions

Rate Limiting

Protect your endpoints from abuse

Rate Limiting Example

from functools import wraps
from flask import request, jsonify
import time

request_counts = {}

def rate_limit(max_requests=100, window=60):
    def decorator(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            uid = request.json.get('uid')
            key = f"{uid}_{f.__name__}"
            now = time.time()

            if key in request_counts:
                requests, first_request = request_counts[key]
                if now - first_request < window:
                    if requests >= max_requests:
                        return jsonify({'error': 'Rate limit exceeded'}), 429
                    request_counts[key] = (requests + 1, first_request)
                else:
                    request_counts[key] = (1, now)
            else:
                request_counts[key] = (1, now)

            return f(*args, **kwargs)
        return wrapper
    return decorator

@app.route('/api/send_message', methods=['POST'])
@rate_limit(max_requests=100, window=60)
def send_message():
    # ...

Testing Your Tools

Test Endpoints Directly

Use curl or Postman before adding to your app:

curl -X POST https://your-server.com/api/send_message \
  -H "Content-Type: application/json" \
  -d '{
    "uid": "test_user_id",
    "app_id": "slack-integration",
    "tool_name": "send_slack_message",
    "channel": "#general",
    "message": "Test message"
  }'

Create the App

Add your tool definitions in the Omi App Store

Install the App

Install it in your test account

Connect Your Account

If auth is required, click the connect button in app settings

Test in Chat

Try using the tools naturally:

“Send a message to #general saying hello”
“What Slack channels do I have?”
“Search Slack for messages about the project”

Troubleshooting

Tool Not Appearing in Chat

Verify the app is installed and enabled (enabled: true)
Check that chat_tools array is properly formatted
Ensure endpoints are accessible and return proper responses
Verify the app has external_integration capability

Tool Calls Failing

Check endpoint logs for errors
Verify authentication is working
Ensure response format matches specification
Test endpoints directly with curl/Postman
Check that required parameters are being sent

AI Not Using Your Tool

Improve tool description to be more specific about when to use it
Add examples in the description
Ensure tool name is descriptive
Make sure the description clearly states when the tool should be used

Complete Example

Slack Integration Example

See a complete working example that demonstrates OAuth authentication, multiple chat tools, handling invocations, and error handling

Developer API

Access your own personal Omi data programmatically

Data Import APIs

Create conversations and memories via REST API

Integration Apps

Learn about external integrations

OAuth Guide

Set up authentication for your app

Submit Your App

Publish your app to the store

Notifications

Send push notifications to users

​Overview

Custom Actions

Natural Language

Automatic Discovery

​How Chat Tools Work

​Part 1: Implementing Chat Tools

​Step 1: Create Tool Endpoints

​Example: Send Message Tool

​Example: Search Tool

​Step 2: Handle Authentication

Store Tokens Securely

Validate Authentication

Return Helpful Errors

​Step 3: Error Handling Best Practices

​Part 2: Adding Chat Tools in Omi App Store

Single Source of Truth

Easy Updates

Full Control

Version Control

​Step 1: Create the Manifest File

​Step 2: Host the Manifest

​Step 3: Add Manifest URL in Omi App Store

​Manifest Schema Reference

​Parameters Schema

​Example: FastAPI Manifest Endpoint

​Sending Chat Messages Proactively

​Enabling Chat Messages

​Sending Messages via API

​API Request Fields

​Example: Background Task Completion

​Rate Limits

​Tool Definition Best Practices

​Request and Response Format

​Request Format

​Geolocation

​Response Format

​Best Practices

Clear Responses

Helpful Errors

Secure Auth

Rate Limiting

​Rate Limiting Example

​Testing Your Tools

​Troubleshooting

​Complete Example

Slack Integration Example

​Related Documentation

Developer API

Data Import APIs

Integration Apps

OAuth Guide

Submit Your App

Notifications

Overview

How Chat Tools Work

Part 1: Implementing Chat Tools

Step 1: Create Tool Endpoints

Example: Send Message Tool

Example: Search Tool

Step 2: Handle Authentication

Step 3: Error Handling Best Practices

Part 2: Adding Chat Tools in Omi App Store

Step 1: Create the Manifest File

Step 2: Host the Manifest

Step 3: Add Manifest URL in Omi App Store

Manifest Schema Reference

Parameters Schema

Example: FastAPI Manifest Endpoint

Sending Chat Messages Proactively

Enabling Chat Messages

Sending Messages via API

API Request Fields

Example: Background Task Completion

Rate Limits

Tool Definition Best Practices

Request and Response Format

Request Format

Geolocation

Response Format

Best Practices

Rate Limiting Example

Testing Your Tools

Troubleshooting

Complete Example

Related Documentation