Have you ever wanted to create a sophisticated AI sales system that combines multiple specialized AI agents while allowing human team members to monitor and step in when needed? In this DIY guide, I’ll walk you through how we built our own system using OpenAI’s Assistants API, WebSockets, PHP, and React.

What You’ll Need

• A web server with PHP support (we used WAMP)
• MySQL database
• OpenAI API key
• Basic knowledge of PHP, JavaScript, and React
• Familiarity with WebSockets

Step 1: Set Up Your Project Structure

Start by creating your project directory structure:

CopyInsert/your-project-name/
├── admin/
│   ├── index.php
│   └── login.php
├── vendor/
├── config.php
├── index.html
├── style.css
├── websocket_server.php
└── composer.json

Step 2: Configure Your Environment

Install required dependencies using Composer:

{ "require": { "cboden/ratchet": "^0.4.4", "openai/client": "^1.0.0", "phpmailer/phpmailer": "^6.8" } }

Create a config.php file with your database and API credentials:

<?php define(‘DB_HOST’, ‘localhost’); define(‘DB_USER’, ‘your_username’); define(‘DB_PASS’, ‘your_password’); define(‘DB_NAME’, ‘your_database’); define(‘OPENAI_API_KEY’, ‘your_openai_api_key’); // Define your OpenAI assistant IDs define(‘AISALESMANAGER’, ‘your_manager_assistant_id’); define(‘NEWSALESAGENT’, ‘your_new_sales_assistant_id’); define(‘EXISTINGSALESAGENT’, ‘your_existing_sales_assistant_id’); define(‘ESTIMATESAGENT’, ‘your_estimates_assistant_id’); // For email notifications define(‘SMTPUSER’, ‘your_smtp_username’); define(‘SMTPPASS’, ‘your_smtp_password’); ?>

Step 3: Set Up Your Database

Create your MySQL database with the necessary tables:

CREATE DATABASE your_database;
USE your_database;

CREATE TABLE conversation_summaries (
    id INT AUTO_INCREMENT PRIMARY KEY,
    thread_id VARCHAR(255) NOT NULL,
    user_ip VARCHAR(45) NOT NULL,
    summary TEXT,
    current_agent VARCHAR(255),
    status ENUM('active', 'completed') DEFAULT 'active',
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

CREATE TABLE messages (
    id INT AUTO_INCREMENT PRIMARY KEY,
    thread_id VARCHAR(255) NOT NULL,
    message_id VARCHAR(255) NOT NULL,
    role ENUM('user', 'assistant', 'system', 'admin') NOT NULL,
    content TEXT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    INDEX (thread_id)
);

Step 4: Create Your WebSocket Server

The heart of our system is the WebSocket server that handles real-time communication. Here’s how to build it:

Create websocket_server.php:

<?php require 'vendor/autoload.php'; require_once 'config.php'; use Ratchet\Server\IoServer; use Ratchet\Http\HttpServer; use Ratchet\WebSocket\WsServer; use React\Socket\Server as SocketServer; use React\EventLoop\Factory; use Ratchet\ConnectionInterface; use Ratchet\MessageComponentInterface; class ConversationServer implements MessageComponentInterface { protected $clients; protected $clientThreadMap = []; // Map clients to thread IDs protected $userClients = []; // Track user clients private $conn; private $openAIClient; public function __construct() { $this->clients = new \SplObjectStorage; $this->userClients = new \SplObjectStorage; // Initialize database connection $this->conn = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME); // Initialize OpenAI client $this->openAIClient = \OpenAI::client(OPENAI_API_KEY); } // Implement the required methods: onOpen, onMessage, onClose, onError // [Add implementation details here] } // Create and start the server $loop = Factory::create(); $socket = new SocketServer('0.0.0.0:8080', $loop); $server = new IoServer( new HttpServer( new WsServer( new ConversationServer() ) ), $socket, $loop ); echo "WebSocket server running at 0.0.0.0:8080\n"; $loop->run();

Step 5: Build Your User Interface

Create a user-friendly chat interface using React:

1. In your index.html, set up the basic structure:<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>AI Sales Assistant</title> <script src="https://cdn.tailwindcss.com"></script> <link rel="stylesheet" href="style.css"> </head> <body> <div id="app"></div> <script src="https://unpkg.com/react@17/umd/react.development.js"></script> <script src="https://unpkg.com/react-dom@17/umd/react-dom.development.js"></script> <script src="https://unpkg.com/babel-standalone@6/babel.min.js"></script> <script type="text/babel"> // Your React application code goes here </script> </body> </html>
2. Create your React chat component with WebSocket connection:
3. function App() { const [messages, setMessages] = React.useState([]); const [threadId, setThreadId] = React.useState(null); const [inputValue, setInputValue] = React.useState(”); const [isProcessing, setIsProcessing] = React.useState(false); const socket = React.useRef(null); // Initialize WebSocket connection React.useEffect(() => { const wsProtocol = window.location.protocol === ‘https:’ ? ‘wss:’ : ‘ws:’; const wsHost = ‘localhost:8080’; const wsUrl = `${wsProtocol}//${wsHost}`; const newSocket = new WebSocket(wsUrl); // Handle WebSocket events (onopen, onmessage, onclose, onerror) // [Add implementation details here] socket.current = newSocket; return () => { if (newSocket) { newSocket.close(); } }; }, []); // Implement message handling and UI components // [Add implementation details here] return ( // Your chat UI JSX goes here ); } ReactDOM.render(<App />, document.getElementById(‘app’));

Step 6: Create the Admin Interface

Build an admin panel to monitor and manage conversations:

1. Create admin/index.php:phpCopyInsert<?php session_start(); require_once '../config.php'; // Implement authentication if (!isset($_SESSION['admin_authenticated'])) { // Show login form or authenticate // [Add implementation details] } // Fetch active conversations from database $conn = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME); $query = "SELECT * FROM conversation_summaries ORDER BY created_at DESC"; $result = $conn->query($query); ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Admin Panel</title> <script src="https://cdn.tailwindcss.com"></script> </head> <body> <!-- Implement your admin UI here --> <!-- Left panel for conversation list --> <!-- Right panel for viewing and participating in conversations --> <script> // JavaScript for WebSocket connection and admin functions // [Add implementation details] </script> </body> </html>

Step 7: Configure Your OpenAI Assistants

You’ll need to create four different assistants in the OpenAI platform:

1. AI Sales Manager: Configure with instructions to identify user needs and direct to the appropriate specialized agent
2. New Sales Agent: Specialize in handling new customer inquiries
3. Existing Sales Agent: Optimize for returning customer interactions
4. Estimates Agent: Focus on gathering requirements and generating estimates

For each assistant, use the OpenAI web interface to:

• Create a new assistant with appropriate name and description
• Set up the model (we recommend GPT-4)
• Add detailed instructions for the assistant’s role
• Enable structured output for agent handoffs

Step 8: Implementing Agent Handoffs

The key to our multi-agent system is the handoff mechanism. In your WebSocket server, implement handling for structured outputs:

private function processAssistantResponse($threadId, $runId) {
    // Retrieve the run and check for structured output
    $run = $this->openAIClient->threads()->runs()->retrieve(
        threadId: $threadId,
        runId: $runId
    );
    
    // Check for structured outputs indicating agent handoffs
    if (property_exists($run, 'structured_outputs') && !empty($run->structured_outputs)) {
        foreach ($run->structured_outputs as $output) {
            if (property_exists($output, 'next_agent')) {
                $nextAgent = $output->next_agent;
                
                // Update the agent in the database
                $this->updateConversationAgent($threadId, $nextAgent);
                
                // Create a new run with the next agent
                $this->openAIClient->threads()->runs()->create(
                    threadId: $threadId,
                    parameters: [
                        'assistant_id' => $nextAgent
                    ]
                );
                
                return true; // Handoff initiated
            }
        }
    }
    
    return false; // No handoff
}

Step 9: Start Your WebSocket Server

Create a simple script to start your WebSocket server:

# start_websocket_server.bat
php websocket_server.php

Step 10: Testing Your System

1. Start your WebSocket server
2. Open your main site in one browser window
3. Open the admin panel in another browser window
4. Test the entire flow:
   • Start a conversation as a user
   • Observe the conversation in the admin panel
   • Test agent handoffs by triggering different scenarios
   • Test human takeover from the admin panel

Troubleshooting Common Issues

• WebSocket Connection Failures: Ensure your server allows WebSocket connections and that ports are properly configured
• OpenAI API Errors: Verify your API key and check OpenAI’s status page for service issues
• Database Connection Problems: Confirm your database credentials and that your tables are properly set up
• Agent Handoff Issues: Review your structured output format and ensure your assistants are properly configured