Build a Custom AI Terminal Assistant with Python: A Complete Step-by-Step Developer Tutorial

You know the drill. You’re deep in a debugging session, and you need a quick answer. So you alt-tab to Chrome, open ChatGPT, type your question, copy the response, and paste it back. That’s three context switches for one answer. Do that twenty times a day, and you’ve wasted an hour just on overhead.

I got sick of it. So I built my own AI terminal assistant.

Stop Dreading Legacy Code: How AI Assisted Debugging and Refactoring Saves Your Sanity

This article explores how AI assisted debugging and refactoring tools reduce production bugs by 40% and cut development… ...

Read more

It’s not a toy. It’s a production-grade tool that lives in my shell, understands my project context, and can even execute commands when I ask nicely. And here’s the kicker: you can build one too, in under two hours.

Let’s build it.

Why Vietnam Outsourcing Is Winning Southeast Asia’s Tech Talent War

TL;DR Vietnam has quietly become Southeast Asia’s most potent tech hub, offering a unique blend of competitive costs,… ...

Read more

What We’re Building

A CLI tool called `ai` that you can invoke like this:

bash
ai "explain this tail -f log file command"
ai --ask "why is my docker container exiting with code 137"
ai --exec "find all files modified in the last 24 hours"

It’ll stream responses to your terminal, maintain conversation history, and optionally execute shell commands after you approve them.

Prerequisites

• Python 3.10+
• An OpenAI API key (or any LLM provider, but we’ll use OpenAI for simplicity)
• Basic familiarity with `argparse`, `asyncio`, and `subprocess`

Step 1: Project Setup

Create a new directory and set up a virtual environment:

bash
mkdir ai-terminal-assistant
cd ai-terminal-assistant
python3 -m venv venv
source venv/bin/activate
pip install openai rich pyyaml

We’re using Rich for beautiful terminal output and PyYAML for configuration. Nothing fancy.

Step 2: The Core Assistant Class

Create `assistant.py`. This is the brain of our tool.

python
import os
import json
import subprocess
from typing import Optional
from openai import OpenAI

class TerminalAssistant:
    def __init__(self, config: dict):
        self.client = OpenAI(api_key=config.get("openai_api_key"))
        self.model = config.get("model", "gpt-4o")
        self.history: list = []
        self.system_prompt = """You are a senior Linux/macOS engineer helping a developer in their terminal.
Keep responses concise. When explaining commands, show the exact syntax.
When asked to execute commands, use the execute_command function.
Never run destructive commands without explicit user confirmation.
If you're unsure about something, say so."""
        
    def add_message(self, role: str, content: str):
        self.history.append({"role": role, "content": content})
        
    def get_response(self, user_input: str) -> str:
        self.add_message("user", user_input)
        
        messages = [{"role": "system", "content": self.system_prompt}]
        messages.extend(self.history[-10:])  # Keep last 10 messages for context
        
        response = self.client.chat.completions.create(
            model=self.model,
            messages=messages,
            stream=True
        )
        
        full_response = ""
        for chunk in response:
            if chunk.choices[0].delta.content:
                content = chunk.choices[0].delta.content
                full_response += content
                print(content, end="", flush=True)
        
        print()  # Newline after streaming
        self.add_message("assistant", full_response)
        return full_response

Notice the streaming. You want to see the answer appear character by character, not wait for the whole thing. That’s the difference between a tool that feels responsive and one that feels like a chore.

Step 3: Adding Function Calling for Command Execution

Here’s where it gets interesting. We’ll let the AI suggest and execute shell commands.

Add this method to your `TerminalAssistant` class:

python
    def get_available_functions(self):
        return [
            {
                "type": "function",
                "function": {
                    "name": "execute_command",
                    "description": "Execute a shell command and return its output",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "command": {
                                "type": "string",
                                "description": "The shell command to execute"
                            },
                            "description": {
                                "type": "string",
                                "description": "What this command does, for user review"
                            },
                            "requires_approval": {
                                "type": "boolean",
                                "description": "Whether this command needs user approval (true for destructive operations)"
                            }
                        },
                        "required": ["command", "description", "requires_approval"]
                    }
                }
            }
        ]

And a method to handle the function call:

python
    def handle_function_call(self, function_name: str, arguments: dict) -> str:
        if function_name == "execute_command":
            cmd = arguments["command"]
            desc = arguments.get("description", "")
            needs_approval = arguments.get("requires_approval", True)
            
            if needs_approval:
                print(f"\n[AI wants to run: {desc}]")
                print(f"Command: {cmd}")
                confirm = input("Run this command? (y/N): ")
                if confirm.lower() != 'y':
                    return "Command execution cancelled by user."
            
            try:
                result = subprocess.run(
                    cmd, 
                    shell=True, 
                    capture_output=True, 
                    text=True, 
                    timeout=30
                )
                output = result.stdout + result.stderr
                return output[:2000]  # Limit response size
            except subprocess.TimeoutExpired:
                return "Command timed out after 30 seconds."
            except Exception as e:
                return f"Error executing command: {str(e)}"

Step 4: The CLI Entry Point

Create `cli.py`:

python
import argparse
import yaml
import os
from assistant import TerminalAssistant

def load_config():
    config_path = os.path.expanduser("~/.ai-assistant/config.yaml")
    default_config = {
        "openai_api_key": os.getenv("OPENAI_API_KEY"),
        "model": "gpt-4o"
    }
    
    if os.path.exists(config_path):
        with open(config_path) as f:
            user_config = yaml.safe_load(f)
            default_config.update(user_config)
    
    return default_config

def main():
    parser = argparse.ArgumentParser(description="AI Terminal Assistant")
    parser.add_argument("query", nargs="*", help="Your question or command")
    parser.add_argument("--ask", type=str, help="Ask a question about a command")
    parser.add_argument("--exec", type=str, help="Execute a command described in natural language")
    
    args = parser.parse_args()
    
    config = load_config()
    if not config.get("openai_api_key"):
        print("Error: OPENAI_API_KEY not set. Set it in your environment or ~/.ai-assistant/config.yaml")
        return 1
    
    assistant = TerminalAssistant(config)
    
    if args.ask:
        assistant.get_response(f"Explain this command in detail: {args.ask}")
    elif args.exec:
        assistant.get_response(f"Execute this task: {args.exec}. Use the execute_command function when ready.")
    elif args.query:
        assistant.get_response(" ".join(args.query))
    else:
        # Interactive mode
        print("AI Terminal Assistant. Type 'exit' to quit.")
        while True:
            try:
                user_input = input("\n> ")
                if user_input.lower() in ['exit', 'quit']:
                    break
                assistant.get_response(user_input)
            except KeyboardInterrupt:
                print("\nGoodbye!")
                break

if __name__ == "__main__":
    main()

Step 5: Make It a Shell Command

Add this alias to your `.bashrc` or `.zshrc`:

bash
alias ai='python3 /path/to/your/project/cli.py'

Or better yet, make it a proper script. Create `ai` in your PATH:

bash
#!/usr/bin/env python3
import sys
sys.path.insert(0, '/path/to/your/project')
from cli import main
main()

Real-World Usage

Here’s what it looks like in practice:

bash
$ ai "what's the difference between docker kill and docker stop"

Docker stop sends SIGTERM first, giving the process 10 seconds to 
gracefully shut down. Docker kill sends SIGKILL immediately.

Use `docker stop` for normal shutdowns. Use `docker kill` when 
a container is stuck or ignoring the stop signal.

$ ai --ask "grep -r 'TODO' --include='*.py' ."

This command recursively searches all Python files in the current 
directory for the string "TODO". The breakdown:

- `-r`: recursive search through directories
- `--include='*.py'`: only search .py files
- `.`: start from current directory

$ ai --exec "find the top 5 largest files in this directory"
[AI wants to run: Find largest files recursively]
Command: du -ah . | sort -rh | head -5
Run this command? (y/N): y

Output:
2.3G    ./node_modules
450M    ./dist/bundle.js
120M    ./data/backup.sql
85M     ./.git/objects
12M     ./venv/lib

Honestly, the `–exec` flag is where this tool shines. It’s like having a junior sysadmin who never sleeps.

Configuration File

Create `~/.ai-assistant/config.yaml`:

yaml
openai_api_key: sk-your-key-here
model: gpt-4o
max_history: 20
safe_mode: true

Set `safe_mode: false` if you trust the AI to run commands without confirmation. I don’t recommend it for production use.

Performance Considerations

• Streaming is mandatory. Without it, responses feel sluggish and the tool feels broken.
• History management matters. We keep the last 10 messages to stay within token limits. For longer sessions, consider summarizing old history.
• Command timeout. We set a 30-second timeout on shell commands. Adjust based on your needs.

Extending the Tool

This is a foundation, not the final product. Here’s what I’d add next:

• Multi-model support. Swap between GPT-4o, Claude, or local models via Ollama.
• Project context injection. Read your `package.json`, `requirements.txt`, or `Dockerfile` and include it in the system prompt.
• File editing capabilities. Let the AI read and write files with your approval.
• Persistent conversation history. Save sessions to disk so you can resume later.

I’ve been using this for three months now. It’s replaced my ChatGPT browser tab entirely. And the best part? I can share it with my team at ECOA AI, and everyone customizes it for their own workflow.

Frequently Asked Questions

Q: Can I use this with local LLMs instead of OpenAI?

Yes. Swap the OpenAI client for Ollama or llama.cpp. The function calling API differs slightly, but the architecture stays the same. For a local setup, use `ollama run llama3.1` and replace the streaming logic with Ollama’s Python client.

Q: How do I handle API rate limits?

Implement exponential backoff. The `openai` library has built-in retry logic if you set `max_retries=3` on the client. For heavy usage, cache common responses based on a hash of the user input and system prompt.

Q: Is it safe to let the AI execute commands?

Only if you keep the approval flow. Never set `safe_mode: false` on a production machine. The function call parameters include `requires_approval` — always set it to `true` for destructive operations like `rm`, `dd`, or `chmod`.

Q: Can I add custom commands or integrations?

Absolutely. Extend the `get_available_functions()` method. Add functions for `read_file`, `write_file`, `git_status`, or `docker_ps`. Each function gets its own JSON schema definition and handler method.

Related reading: Outsourcing Software in 2025: The Hard Truths and Hidden Wins

Related reading: Hire Vietnamese Developers: The Strategic Edge for Scalable Tech Teams