General

Chatbot

Build AI-powered chatbots with streaming responses and conversation history.

The starter kit ships a complete organization chatbot with text streaming, conversation history, model selection and credit accounting. The repository uses AI SDK 7. Custom examples on this page are labeled so they are not confused with shipped files.

Overview

The chatbot uses:

Vercel AI SDK - For streaming responses and state management
tRPC - For type-safe chat CRUD operations
OpenAI - For the LLM backend (configurable)

Streaming Endpoint

The shipped app/api/ai/chat/route.ts authenticates and validates each request, checks organization ownership and credit balance, normalizes UI message parts, calls OpenAI, deducts the actual credit cost and saves the response. This reduced example keeps the same UI message protocol and normalization but omits model selection and credit accounting.

app/api/ai/chat/route.ts

import { openai } from '@ai-sdk/openai';
import { streamText, type ModelMessage } from 'ai';
import { and, eq } from 'drizzle-orm';

import { assertUserIsOrgMember, getSession } from '@/lib/auth/server';
import { db } from '@/lib/db';
import { aiChatTable } from '@/lib/db/schema';

type ChatRequest = {
  messages: Array<{
    role: 'user' | 'assistant' | 'system';
    content?: string;
    parts?: Array<{ type: string; text?: string }>;
  }>;
  chatId: string;
  organizationId: string;
};

function toModelMessages(messages: ChatRequest['messages']): ModelMessage[] {
  return messages.map((message) => {
    const content =
      message.content ??
      message.parts?.find((part) => part.type === 'text')?.text ??
      '';

    switch (message.role) {
      case 'system':
        return { role: 'system', content };
      case 'assistant':
        return { role: 'assistant', content };
      default:
        return { role: 'user', content };
    }
  });
}

export async function POST(req: Request) {
  const session = await getSession();
  if (!session) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 });
  }

  const {
    messages: uiMessages,
    chatId,
    organizationId
  }: ChatRequest = await req.json();
  await assertUserIsOrgMember(organizationId, session.user.id);

  const messages = toModelMessages(uiMessages);

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
    async onFinish({ text }) {
      // Save assistant's response to the database
      const updatedMessages = [
        ...messages,
        { role: 'assistant' as const, content: text }
      ];

      await db
        .update(aiChatTable)
        .set({ messages: JSON.stringify(updatedMessages) })
        .where(
          and(
            eq(aiChatTable.id, chatId),
            eq(aiChatTable.organizationId, organizationId)
          )
        );
    }
  });

  return result.toUIMessageStreamResponse({
    onError: () => 'AI is temporarily unavailable. Please try again later.'
  });
}

DefaultChatTransport on the client must stay paired with toUIMessageStreamResponse() on the server. Do not pass the hook's raw UIMessage[] to streamText; normalize its parts to model content first, as the shipped route does.

UI Components

Main Chat Component

The AiChat component provides a full conversation interface with a history sidebar.

app/(saas)/dashboard/(sidebar)/organization/chatbot/page.tsx

import { AiChat } from '@/components/ai/ai-chat';
import { getSession } from '@/lib/auth/server';

export default async function AiPage() {
  const session = await getSession();
  const organizationId = session?.session.activeOrganizationId;

  if (!organizationId) {
    return <div>No active organization</div>;
  }

  return <AiChat organizationId={organizationId} />;
}

Custom Component

This optional component shows the AI SDK 7 transport and input APIs in a self-contained example. The shipped AiChat component has additional history, billing and error handling behavior.

components/my-custom-ai.tsx

'use client';

import { useState, type FormEvent } from 'react';
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

import { MessageResponse } from '@/components/ai/message';

type MyCustomAIProps = {
  chatId: string;
  organizationId: string;
};

export function MyCustomAI({ chatId, organizationId }: MyCustomAIProps) {
  const [input, setInput] = useState('');
  const { messages, sendMessage, status } = useChat({
    id: chatId,
    transport: new DefaultChatTransport({
      api: '/api/ai/chat',
      body: { chatId, organizationId }
    })
  });
  const isSending = status === 'submitted' || status === 'streaming';

  function handleSubmit(event: FormEvent<HTMLFormElement>) {
    event.preventDefault();
    const text = input.trim();
    if (!text) return;

    setInput('');
    sendMessage({
      role: 'user',
      parts: [{ type: 'text', text }]
    });
  }

  return (
    <div>
      {messages.map((message) => {
        const text = message.parts
          .filter((part) => part.type === 'text')
          .map((part) => part.text)
          .join('');

        return (
          <div key={message.id}>
            <strong>{message.role}:</strong>{' '}
            {message.role === 'assistant' ? (
              <MessageResponse>{text}</MessageResponse>
            ) : (
              <span>{text}</span>
            )}
          </div>
        );
      })}
      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={(event) => setInput(event.target.value)}
          placeholder="Type a message..."
          disabled={isSending}
        />
        <button
          type="submit"
          disabled={isSending}
        >
          Send
        </button>
      </form>
    </div>
  );
}

Conversation History

Chats are stored in the database and can be retrieved via tRPC:

trpc/routers/organization/organization-ai-router.ts

import { createTRPCRouter, protectedOrganizationProcedure } from '@/trpc/init';
import { TRPCError } from '@trpc/server';
import { and, desc, eq, sql } from 'drizzle-orm';
import { z } from 'zod';

import { appConfig } from '@/config/app.config';
import { db } from '@/lib/db';
import { aiChatTable } from '@/lib/db/schema';

export const organizationAiRouter = createTRPCRouter({
  listChats: protectedOrganizationProcedure
    .input(
      z
        .object({
          limit: z
            .number()
            .min(1)
            .max(appConfig.pagination.maxLimit)
            .optional()
            .default(appConfig.pagination.defaultLimit),
          offset: z.number().min(0).optional().default(0)
        })
        .optional()
    )
    .query(async ({ ctx, input }) => {
      // Use SQL builder to select only needed columns and extract first message
      const chats = await db
        .select({
          id: aiChatTable.id,
          title: aiChatTable.title,
          pinned: aiChatTable.pinned,
          createdAt: aiChatTable.createdAt,
          firstMessageContent: sql<string | null>`
            CASE 
              WHEN ${aiChatTable.messages} IS NOT NULL 
                AND ${aiChatTable.messages}::jsonb != '[]'::jsonb 
              THEN (${aiChatTable.messages}::jsonb->0->>'content')
              ELSE NULL 
            END
          `.as('first_message_content')
        })
        .from(aiChatTable)
        .where(eq(aiChatTable.organizationId, ctx.organization.id))
        .orderBy(desc(aiChatTable.pinned), desc(aiChatTable.createdAt))
        .limit(input?.limit ?? 20)
        .offset(input?.offset ?? 0);

      return { chats };
    }),

  getChat: protectedOrganizationProcedure
    .input(z.object({ id: z.string().uuid() }))
    .query(async ({ ctx, input }) => {
      const chat = await db.query.aiChatTable.findFirst({
        where: and(
          eq(aiChatTable.id, input.id),
          eq(aiChatTable.organizationId, ctx.organization.id)
        )
      });

      if (!chat) {
        throw new TRPCError({
          code: 'NOT_FOUND',
          message: 'Chat not found'
        });
      }

      return {
        chat: {
          ...chat,
          messages: chat.messages ? JSON.parse(chat.messages) : []
        }
      };
    }),

  createChat: protectedOrganizationProcedure
    .input(z.object({ title: z.string().optional() }).optional())
    .mutation(async ({ ctx, input }) => {
      const [chat] = await db
        .insert(aiChatTable)
        .values({
          organizationId: ctx.organization.id,
          title: input?.title || 'New Chat',
          messages: JSON.stringify([])
        })
        .returning();

      return { chat };
    }),

  deleteChat: protectedOrganizationProcedure
    .input(z.object({ id: z.string().uuid() }))
    .mutation(async ({ input, ctx }) => {
      await db
        .delete(aiChatTable)
        .where(
          and(
            eq(aiChatTable.id, input.id),
            eq(aiChatTable.organizationId, ctx.organization.id)
          )
        );
    })
});

Custom Example: Tool Calling

The shipped chat route does not register tools. This custom route expects ModelMessage[], not the UIMessage[] returned by useChat. Protect it with the same authentication and organization checks as the shipped route before using it in production.

app/api/ai/chat/route.ts

import { openai } from '@ai-sdk/openai';
import { streamText, type ModelMessage } from 'ai';
import { ilike } from 'drizzle-orm';
import { z } from 'zod/v4';

import { db } from '@/lib/db';
import { leadTable } from '@/lib/db/schema';

export async function POST(req: Request) {
  const { messages }: { messages: ModelMessage[] } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
    tools: {
      findLeads: {
        description: 'Find leads in the database by name',
        inputSchema: z.object({
          query: z.string().describe('The search query')
        }),
        execute: async ({ query }) => {
          const leads = await db.query.leadTable.findMany({
            where: ilike(leadTable.name, `%${query}%`),
            limit: 10
          });

          return leads;
        }
      }
    }
  });

  return result.toTextStreamResponse();
}

Custom Example: Generation Settings

The shipped route only accepts model IDs from chatModels in config/billing.config.ts. For a separate fixed-model helper, use the AI SDK 7 maxOutputTokens setting:

lib/ai/generate-short-reply.ts

import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

export async function generateShortReply(prompt: string) {
  const { text } = await generateText({
    model: openai('gpt-4o-mini'),
    prompt,
    temperature: 0.7,
    maxOutputTokens: 1000
  });

  return text;
}

Custom Example: Error Handling

This custom route also accepts ModelMessage[]. A client using useChat must normalize its message parts first or switch to the UI message protocol.

app/api/ai/chat/route.ts

import { openai } from '@ai-sdk/openai';
import { streamText, type ModelMessage } from 'ai';

export async function POST(req: Request) {
  try {
    const { messages }: { messages: ModelMessage[] } = await req.json();

    const result = streamText({
      model: openai('gpt-4o-mini'),
      messages
    });

    return result.toTextStreamResponse();
  } catch (error) {
    console.error('AI chat error:', error);
    return Response.json(
      { error: 'Failed to process chat request' },
      { status: 500 }
    );
  }
}

Custom Rate Limiting

The starter kit does not ship a generic @/lib/rate-limit module. It checks organization credit balance before generation and deducts actual usage after generation. If you need request-frequency limits too, add a durable rate-limit provider and enforce it after authentication.

Best Practices

Stream responses - Always use streaming for better UX
Save conversations - Store chat history in the database
Implement rate limiting - Control API costs
Handle errors - Provide user-friendly error messages
Use tools wisely - Add tools for database queries and external APIs
Monitor usage - Track token usage and costs