Streaming AI Chat Response with History to Livewire

While working on Skipjack AI, I needed to be able to stream a response to the browser for a better user experience. Livewire has good documentation on using `wire:stream` with a chat-bot, but I wanted to take it a step further and keep chat history visible.

Decoupling the Stream

I like how the stream method call is passed in a callback so the AI system doesn’t need to know any of the stream details, it just calls the callback when there’s new data. For my purposes I want to support different LLM services and I have some additional data I need to include with the chat request so I’m using a Service Provider and resolving the interface.

$streamed_chat_service = resolve(StreamedChatServiceInterface::class);
$chat_response_data = $streamed_chat_service(
    $chat_request_data,
    fn($delta_content) => $this->stream(to: 'answer', content: $delta_content)
);

Setting the Stage

Now, I want to show the history of the chat. For a while I struggled with keeping the old content in the stream and adding new content, then I realized that really only the latest answer was being streamed.

Component (Simplified)

<?php

namespace App\Livewire;

use App\Data\ChatRequestData;
use App\Interfaces\StreamedChatServiceInterface;
use App\Models\Chat as ModelsChat;
use App\Services\ChatSaveService;
use Livewire\Component;

class Chat extends Component
{
    public string $prompt = '';
    public string $question = '';
    public string $answer = '';
    public ModelsChat $chat;

    public function mount()
    {
        $this->chat = new ModelsChat([
            'title' => 'New Chat',
        ]);
    }

    public function submitPrompt()
    {
        $this->question = $this->prompt;
        $this->prompt = '';
        $this->js('$wire.ask()');
    }

    public function ask()
    {
        $chat_request_data = ChatRequestData::from($this->chat, $this->question);

        $streamed_chat_service = resolve(StreamedChatServiceInterface::class);
        $chat_response_data = $streamed_chat_service(
            $chat_request_data,
            fn($delta_content) => $this->stream(to: 'answer', content: $delta_content)
        );

        $chat_save_service = new ChatSaveService($this->chat);
        $chat_save_service($chat_request_data, $chat_response_data);

        $this->answer = '';
        $this->question = '';
        $this->chat->refresh();
    }
}

View (Simplified)

<h1
	class="text-3xl font-bold tracking-tight text-gray-900 dark:text-gray-100 sm:text-4xl"
>{{ $chat->title }}</h1>
<div class="flex flex-col grow">
	<div class="grow overflow-scroll my-6 border border-gray-300 rounded p-4 space-y-4">
		@foreach ($chat->entries as $entry)
			<div
				@class([
					'p-4 rounded prose dark:prose-invert',
					'bg-green-50 dark:bg-green-900 mr-8' => $entry->role == 'user',
					'bg-blue-50 dark:bg-blue-900 ml-8' => $entry->role == 'assistant',
				])
				wire:key="{{ $entry->id }}"
			>{!! $entry->displayText() !!}</div>
		@endforeach
		@if ( $question )
			<div 
				class="p-4 rounded prose dark:prose-invert bg-green-50 dark:bg-green-900 mr-8"
			>{{ $question }}</div>
			<div 
				class="p-4 rounded prose dark:prose-invert bg-blue-50 dark:bg-blue-900 ml-8" wire:stream="answer"
			>{{ $answer }}</div>
		@endif
	</div>
	<div class="flex justify-between gap-4">
		<textarea
			wire:model="prompt"
			wire:keydown.cmd.enter="submitPrompt"
			rows="3"
			class="block w-full rounded-md border-0 py-1.5 text-gray-900 dark:text-gray-100 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-indigo-600 sm:text-sm sm:leading-6 dark:bg-gray-700"
		></textarea>
		<button
			type="button"
			wire:click="submitPrompt"
		>Send</button>
	</div>
</div>

Explanation

The interesting bits are lines 42-47 in the Component and line 6-23 in the View.

In the Component, the answer is done being streamed in and (behind the scenes) I have all of the chat entry info between the $chat_request_data and $chat_response_data variables. I use this data to save the details to the chat (using a Service to encapsulate the functionality to use in multiple places) and then I can clear out the answer and question. We only use the answer and question when the current answer is being streamed in, after that the data is part of the Chat history and when we refresh the $chat property Livewire will update the interface and show the chat history without any streaming.

If a user types in a new question, we just go through the process again to get an answer and the question/answer data just gets added to the $chat as more entries and the process repeats itself.