Building Conversational Chatbots
Chatbots are the most common Generative AI application. This topic covers building production-quality conversational interfaces with LLMs.
Chatbot Architecture
A modern AI chatbot consists of:
User Input → Preprocessing → LLM API Call → Post-processing → Response
↑ ↓
Conversation Memory ←──────────────────────┘Basic Chatbot (Python)
python
from openai import OpenAI
client = OpenAI()
def create_chatbot():
messages = [
{
"role": "system",
"content": "You are a friendly customer support agent for a "
"software company. Be helpful, concise, and empathetic."
}
]
print("Chatbot: Hi! How can I help you today?")
while True:
user_input = input("You: ").strip()
if user_input.lower() in ["exit", "quit", "bye"]:
print("Chatbot: Goodbye! Have a great day!")
break
messages.append({"role": "user", "content": user_input})
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
temperature=0.7,
max_tokens=500
)
assistant_message = response.choices[0].message.content
messages.append({"role": "assistant", "content": assistant_message})
print(f"Chatbot: {assistant_message}")
create_chatbot()Conversation Memory Management
The Problem
LLMs have finite context windows. Long conversations eventually exceed the limit.
Strategy 1: Sliding Window
Keep only the last N messages:
python
MAX_MESSAGES = 20
def add_message(messages, role, content):
messages.append({"role": role, "content": content})
# Keep system prompt + last N messages
if len(messages) > MAX_MESSAGES + 1:
system = messages[0]
messages = [system] + messages[-(MAX_MESSAGES):]
return messagesStrategy 2: Summarization
Periodically summarize older messages:
python
def summarize_history(client, messages):
history_text = "\n".join(
f"{m['role']}: {m['content']}" for m in messages[1:-5]
)
summary = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"Summarize this conversation in 2-3 sentences:\n{history_text}"
}],
max_tokens=200
)
# Replace old messages with summary
system = messages[0]
recent = messages[-5:]
return [
system,
{"role": "system", "content": f"Previous conversation summary: {summary.choices[0].message.content}"},
*recent
]Strategy 3: Key-Value Memory
Store important facts extracted from the conversation:
python
user_memory = {}
def extract_facts(client, message):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"""Extract key facts from this message as JSON.
Only include definitive facts (name, preferences, issues).
Return {{}}} if no facts found.
Message: {message}"""
}],
response_format={"type": "json_object"},
max_tokens=200
)
import json
return json.loads(response.choices[0].message.content)System Prompt Design for Chatbots
Customer Support Bot
python
system_prompt = """You are a support agent for BIGXSTAR Technologies.
PERSONALITY:
- Friendly, professional, and empathetic
- Patient with frustrated customers
- Proactive in offering solutions
CAPABILITIES:
- Answer questions about products (POS App, Smart Billing)
- Help with account and billing issues
- Provide technical troubleshooting steps
- Escalate complex issues to human agents
RULES:
- Never make up pricing or feature information
- If unsure, say "Let me check that for you" and suggest contacting support
- Always ask if there's anything else you can help with
- Keep responses under 150 words unless the user needs detailed steps"""Persona Bot
python
system_prompt = """You are CodeMentor, an AI programming tutor.
TEACHING STYLE:
- Start with why (motivation before mechanics)
- Use real-world analogies
- Provide working code examples
- Ask follow-up questions to check understanding
RULES:
- Don't give full solutions for homework — guide instead
- Adjust complexity based on the student's level
- Celebrate when they get something right
- Point out common pitfalls and best practices"""Building a Web Chatbot (Next.js)
API Route
typescript
// app/api/chat/route.ts
import { NextRequest, NextResponse } from 'next/server';
import OpenAI from 'openai';
const openai = new OpenAI();
export async function POST(req: NextRequest) {
const { messages } = await req.json();
const response = await openai.chat.completions.create({
model: 'gpt-4o-mini',
messages: [
{
role: 'system',
content: 'You are a helpful assistant for BIGXSTAR Technologies.'
},
...messages
],
temperature: 0.7,
max_tokens: 500,
stream: true
});
// Return streaming response
const stream = new ReadableStream({
async start(controller) {
for await (const chunk of response) {
const text = chunk.choices[0]?.delta?.content || '';
controller.enqueue(new TextEncoder().encode(text));
}
controller.close();
}
});
return new NextResponse(stream, {
headers: { 'Content-Type': 'text/plain; charset=utf-8' }
});
}Chat Component
tsx
// components/ChatWidget.tsx
'use client';
import { useState, useRef } from 'react';
interface Message {
role: 'user' | 'assistant';
content: string;
}
export default function ChatWidget() {
const [messages, setMessages] = useState<Message[]>([]);
const [input, setInput] = useState('');
const [loading, setLoading] = useState(false);
async function handleSubmit(e: React.FormEvent) {
e.preventDefault();
if (!input.trim() || loading) return;
const userMessage: Message = { role: 'user', content: input };
setMessages(prev => [...prev, userMessage]);
setInput('');
setLoading(true);
const response = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ messages: [...messages, userMessage] })
});
const reader = response.body?.getReader();
const decoder = new TextDecoder();
let assistantMessage = '';
setMessages(prev => [...prev, { role: 'assistant', content: '' }]);
while (reader) {
const { done, value } = await reader.read();
if (done) break;
assistantMessage += decoder.decode(value);
setMessages(prev => [
...prev.slice(0, -1),
{ role: 'assistant', content: assistantMessage }
]);
}
setLoading(false);
}
return (
<div className="chat-widget">
<div className="messages">
{messages.map((msg, i) => (
<div key={i} className={`message ${msg.role}`}>
{msg.content}
</div>
))}
</div>
<form onSubmit={handleSubmit}>
<input
value={input}
onChange={e => setInput(e.target.value)}
placeholder="Type a message..."
disabled={loading}
/>
<button type="submit" disabled={loading}>Send</button>
</form>
</div>
);
}Chatbot Best Practices
1. Set Clear Boundaries
"I can help with product questions, technical support, and billing.
For legal or compliance matters, please contact legal@company.com."2. Handle Edge Cases
- Empty messages
- Very long messages (truncate or reject)
- Repeated messages (detect loops)
- Off-topic requests (politely redirect)
3. Add Typing Indicators
Show the bot is "thinking" during API calls to improve UX.
4. Log Conversations
Store conversations for quality monitoring and improvement (with user consent).
5. Implement Fallback to Human
python
if user_frustration_detected or complex_issue:
return "Let me connect you with a human agent who can help better."Key Takeaways
- Conversation memory is the biggest challenge — plan your strategy early
- System prompts define your bot's personality and capabilities
- Streaming dramatically improves perceived performance
- Error handling and fallbacks are essential for production
- Test with real users early — they'll break your bot in unexpected ways
Next, we'll explore text generation and summarization — building applications beyond chat.