Latchkey Bot

A Matrix bot plugin for interacting with local LLM models through llama-swap.

Features

• Chat: Interactive conversations with configurable AI models, streaming responses
• Image Generation: Text-to-image with multiple model support (Stable Diffusion, Flux, etc.)
• Image-to-Image: Transform existing images with a prompt via img2img
• Tools: Web search, calculator, weather, Wikipedia, memory, reminders, dice, DNS/ping, and more
• Conversation History: Per-room/user context with automatic consolidation
• Vision: Analyze images sent in chat (captioned with !lk)

Requirements

• Maubot 0.1.0+
• llama-swap running locally or remotely
• Optional: SearXNG instance for web search functionality

Installation

Via Maubot Dashboard

1. Click "Install from URL" in your Maubot dashboard
2. Enter the URL to this repository or download the .mbp file
3. Configure the bot with your settings

Manual Installation

1. Build the plugin:

   pip install maubot
   maubot build
   

2. Upload com.devvul.latchkeybot-v*.mbp via Maubot dashboard or CLI

Configuration

Edit the config.yaml generated by Maubot or use base-config.yaml as reference:

# Only users on this list are allowed to use the bot
allowlist:
  - "@username:matrix.org"

# Users with admin privileges (must also be in allowlist)
admin_allowlist:
  - "@username:matrix.org"

# llama-swap backend URL
llama_swap_url: "http://localhost:8080"

# API key for llama-swap (leave empty for no auth)
api_key: ""

# Model to use for chat completions
chat_model: "qwen3.5"

# Image generation models (key is model name, value is backend model ID)
# use --model <key> to select; "default" is used when no --model specified
image_models:
  default: "z-image"
  flux: "flux"

# img2img models (same format as image_models)
img2img_models:
  default: "stable-diffusion-xl"

# Default denoising strength for img2img (0.0 = no change, 1.0 = fully regenerated)
img2img_default_denoise: 0.75

# System prompt for chat
system_prompt: "You are Latchkey, a helpful and concise chat assistant..."

# Token threshold for conversation consolidation (estimated as len/3)
consolidation_token_threshold: 12000

# Max tokens for chat completion responses
max_tokens: 2048

# Image generation size (e.g., "1024x1024")
image_size: "1024x1024"

# Enable tool use in chat
tools_enabled: true

# SearXNG instance URL for web search tool (e.g., "http://localhost:8888")
searxng_url: ""

# Max tool call iterations per chat message before forcing a final response
max_tool_iterations: 10

# Timeout in seconds for LLM backend requests
llm_timeout: 120

# Minimum interval in seconds between message edits during streaming
stream_edit_interval: 1.0

Commands

Chat

!lk <message>            Chat with Latchkey
!latchkey <message>      Alias for !lk

Image Generation

!lk image <prompt>       Generate an image (--model <name> for alternate models)
!image <prompt>          Shortcut for !lk image
!lk img2img <prompt>     Reply to an image to transform it (--model, --denoise)

Reminders & Memory

!lk remind <text>        Set a reminder (natural language)
!lk reminders            List pending reminders
!lk reminders cancel <id>  Cancel a reminder
!lk memories             List your saved memories
!lk forget <key>         Delete a saved memory
!lk timezone [tz]        View or set your timezone

Other

!lk history [n]          Browse conversation summaries
!lk system [prompt|reset]  View/set/reset room system prompt (admin)
!lk clear                Clear room conversation history (admin)
!lk version              Show bot version
!lk help                 Show help message

Image Captioning

Send an image with a caption starting with !lk to have the bot analyze it in conversation context.

How It Works

Chat Mode

1. User sends a message via command
2. Bot retrieves conversation history for the room/user pair
3. Messages are sent to llama-swap with optional system prompt
4. Response is streamed back and displayed to the user
5. Conversation history is updated and consolidated when it exceeds the token threshold

Tool-Enabled Mode

When tools_enabled is true and searxng_url is configured, the bot has access to: web search, multi-search, web fetch, search & read, summarize, calculator, unit conversion, weather, date/time, Wikipedia, memory, reminders, describe image, dice roller, reactions, DNS/ping, and geocode.

1. Bot sends messages with tool definitions
2. Model can request tool calls in its response
3. Bot executes requested tools and appends results to context
4. Loop continues until model responds without tool calls or max iterations reached

Image Generation

1. User sends image generation command with prompt
2. Request sent to llama-swap /v1/images/generations endpoint
3. Base64-encoded image returned and decoded
4. Image uploaded to Matrix home server and sent to room

Image-to-Image

1. User replies to an existing image with !lk img2img <prompt>
2. The original image is downloaded and sent along with the prompt
3. Optional --denoise controls how much the image changes (0.0–1.0)
4. Transformed image is uploaded and sent to room