Building AI Applications

Building Conversational Chatbots

Learn how to build intelligent chatbots with conversation memory, persona management, and real-time streaming.

Building Conversational Chatbots

Chatbots are the most common Generative AI application. This topic covers building production-quality conversational interfaces with LLMs.

Chatbot Architecture

A modern AI chatbot consists of:

User Input → Preprocessing → LLM API Call → Post-processing → Response
                  ↑                                    ↓
            Conversation Memory ←──────────────────────┘

Basic Chatbot (Python)

python
from openai import OpenAI

client = OpenAI()

def create_chatbot():
    messages = [
        {
            "role": "system",
            "content": "You are a friendly customer support agent for a "
                       "software company. Be helpful, concise, and empathetic."
        }
    ]

    print("Chatbot: Hi! How can I help you today?")

    while True:
        user_input = input("You: ").strip()
        if user_input.lower() in ["exit", "quit", "bye"]:
            print("Chatbot: Goodbye! Have a great day!")
            break

        messages.append({"role": "user", "content": user_input})

        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            temperature=0.7,
            max_tokens=500
        )

        assistant_message = response.choices[0].message.content
        messages.append({"role": "assistant", "content": assistant_message})

        print(f"Chatbot: {assistant_message}")

create_chatbot()

Conversation Memory Management

The Problem

LLMs have finite context windows. Long conversations eventually exceed the limit.

Strategy 1: Sliding Window

Keep only the last N messages:

python
MAX_MESSAGES = 20

def add_message(messages, role, content):
    messages.append({"role": role, "content": content})
    # Keep system prompt + last N messages
    if len(messages) > MAX_MESSAGES + 1:
        system = messages[0]
        messages = [system] + messages[-(MAX_MESSAGES):]
    return messages

Strategy 2: Summarization

Periodically summarize older messages:

python
def summarize_history(client, messages):
    history_text = "\n".join(
        f"{m['role']}: {m['content']}" for m in messages[1:-5]
    )

    summary = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{
            "role": "user",
            "content": f"Summarize this conversation in 2-3 sentences:\n{history_text}"
        }],
        max_tokens=200
    )

    # Replace old messages with summary
    system = messages[0]
    recent = messages[-5:]
    return [
        system,
        {"role": "system", "content": f"Previous conversation summary: {summary.choices[0].message.content}"},
        *recent
    ]

Strategy 3: Key-Value Memory

Store important facts extracted from the conversation:

python
user_memory = {}

def extract_facts(client, message):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{
            "role": "user",
            "content": f"""Extract key facts from this message as JSON.
            Only include definitive facts (name, preferences, issues).
            Return {{}}} if no facts found.

            Message: {message}"""
        }],
        response_format={"type": "json_object"},
        max_tokens=200
    )
    import json
    return json.loads(response.choices[0].message.content)

System Prompt Design for Chatbots

Customer Support Bot

python
system_prompt = """You are a support agent for BIGXSTAR Technologies.

PERSONALITY:
- Friendly, professional, and empathetic
- Patient with frustrated customers
- Proactive in offering solutions

CAPABILITIES:
- Answer questions about products (POS App, Smart Billing)
- Help with account and billing issues
- Provide technical troubleshooting steps
- Escalate complex issues to human agents

RULES:
- Never make up pricing or feature information
- If unsure, say "Let me check that for you" and suggest contacting support
- Always ask if there's anything else you can help with
- Keep responses under 150 words unless the user needs detailed steps"""

Persona Bot

python
system_prompt = """You are CodeMentor, an AI programming tutor.

TEACHING STYLE:
- Start with why (motivation before mechanics)
- Use real-world analogies
- Provide working code examples
- Ask follow-up questions to check understanding

RULES:
- Don't give full solutions for homework — guide instead
- Adjust complexity based on the student's level
- Celebrate when they get something right
- Point out common pitfalls and best practices"""

Building a Web Chatbot (Next.js)

API Route

typescript
// app/api/chat/route.ts
import { NextRequest, NextResponse } from 'next/server';
import OpenAI from 'openai';

const openai = new OpenAI();

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant for BIGXSTAR Technologies.'
      },
      ...messages
    ],
    temperature: 0.7,
    max_tokens: 500,
    stream: true
  });

  // Return streaming response
  const stream = new ReadableStream({
    async start(controller) {
      for await (const chunk of response) {
        const text = chunk.choices[0]?.delta?.content || '';
        controller.enqueue(new TextEncoder().encode(text));
      }
      controller.close();
    }
  });

  return new NextResponse(stream, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' }
  });
}

Chat Component

tsx
// components/ChatWidget.tsx
'use client';
import { useState, useRef } from 'react';

interface Message {
  role: 'user' | 'assistant';
  content: string;
}

export default function ChatWidget() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState('');
  const [loading, setLoading] = useState(false);

  async function handleSubmit(e: React.FormEvent) {
    e.preventDefault();
    if (!input.trim() || loading) return;

    const userMessage: Message = { role: 'user', content: input };
    setMessages(prev => [...prev, userMessage]);
    setInput('');
    setLoading(true);

    const response = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ messages: [...messages, userMessage] })
    });

    const reader = response.body?.getReader();
    const decoder = new TextDecoder();
    let assistantMessage = '';

    setMessages(prev => [...prev, { role: 'assistant', content: '' }]);

    while (reader) {
      const { done, value } = await reader.read();
      if (done) break;
      assistantMessage += decoder.decode(value);
      setMessages(prev => [
        ...prev.slice(0, -1),
        { role: 'assistant', content: assistantMessage }
      ]);
    }

    setLoading(false);
  }

  return (
    <div className="chat-widget">
      <div className="messages">
        {messages.map((msg, i) => (
          <div key={i} className={`message ${msg.role}`}>
            {msg.content}
          </div>
        ))}
      </div>
      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          placeholder="Type a message..."
          disabled={loading}
        />
        <button type="submit" disabled={loading}>Send</button>
      </form>
    </div>
  );
}

Chatbot Best Practices

1. Set Clear Boundaries

"I can help with product questions, technical support, and billing.
For legal or compliance matters, please contact legal@company.com."

2. Handle Edge Cases

  • Empty messages
  • Very long messages (truncate or reject)
  • Repeated messages (detect loops)
  • Off-topic requests (politely redirect)

3. Add Typing Indicators

Show the bot is "thinking" during API calls to improve UX.

4. Log Conversations

Store conversations for quality monitoring and improvement (with user consent).

5. Implement Fallback to Human

python
if user_frustration_detected or complex_issue:
    return "Let me connect you with a human agent who can help better."

Key Takeaways

  1. Conversation memory is the biggest challenge — plan your strategy early
  2. System prompts define your bot's personality and capabilities
  3. Streaming dramatically improves perceived performance
  4. Error handling and fallbacks are essential for production
  5. Test with real users early — they'll break your bot in unexpected ways

Next, we'll explore text generation and summarization — building applications beyond chat.