λ ollama-haskell v0.4.1
Hackage GitHub
Feature Tutorials

Chat & Conduit Streaming

Learn how to execute multi-turn conversations and stream tokens in real-time with Conduit.

Overview

The ollama-haskell library provides two primary ways to interact with chat models: 1. Synchronous Chat (chat): Blocks until the full response is generated and returned. 2. Streaming Chat (chatStream): Streams chunks token-by-token using conduit for interactive user interfaces and terminal CLI apps.

1. Basic Multi-Turn Conversation

Construct chat requests using NonEmpty Message:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Data.List.NonEmpty (NonEmpty ((:|)))
import Data.Text.IO qualified as TIO
import Ollama

main :: IO ()
main = do
  client <- defaultClient

  -- Build a multi-turn message history
  let history =
        systemMessage "You are a concise Haskell expert."
        :| [ userMessage "What is a Functor?"
           , assistantMessage "A Functor is a type class that can be mapped over using fmap."
           , userMessage "Can you give an example?"
           ]
      req = chatRequest "qwen3.5:2b" history

  res <- chat client req
  case res of
    Left err   -> putStrLn $ "Error: " <> show err
    Right resp -> case crMessage resp of
      Just msg -> TIO.putStrLn (messageContent msg)
      Nothing  -> putStrLn "No response message"

2. Real-Time Streaming with Conduit

Stream response tokens to the terminal as they are generated by the model:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Conduit
import Data.List.NonEmpty (NonEmpty ((:|)))
import Data.Text.IO qualified as TIO
import Ollama

main :: IO ()
main = do
  client <- defaultClient

  let req = chatRequest "qwen3.5:2b" (userMessage "Write a poem about functional programming." :| [])

  -- Stream tokens directly to stdout
  runConduit $
    chatStream client req
    .| mapM_C (\chunk -> do
        case crMessage chunk of
          Just msg -> liftIO $ do
            TIO.putStr (messageContent msg)
            hFlush stdout
          Nothing  -> pure ()
      )

  putStrLn "\n--- Stream complete ---"

3. Stream Combinators (collectStream & foldStream)

If you want to consume stream results into a list or accumulate text without setting up custom conduits:

-- Collect all chunks into a list
chunks <- collectStream (chatStream client req)

-- Or fold text tokens into a single Text string
fullText <- foldStream (\acc chunk ->
  acc <> maybe "" messageContent (crMessage chunk)) "" (chatStream client req)

4. Multi-Turn Session State with STM (InMemoryStore)

For interactive chatbots, Ollama.Conversation provides a thread-safe transactional store:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Data.List.NonEmpty (NonEmpty ((:|)))
import Data.Time (getCurrentTime)
import Ollama
import Ollama.Conversation

main :: IO ()
main = do
  client <- defaultClient
  store  <- initInMemoryStore
  now    <- getCurrentTime

  let sessionId = "101"
      initialConv = Conversation sessionId [userMessage "My favorite language is Haskell."] "qwen3.5:9b" now now

  -- Save initial conversation
  saveConversationInMemory store initialConv

  -- Later in another request, load existing history:
  prevConv <- loadConversationInMemory store sessionId
  case prevConv of
    Nothing -> putStrLn "No conversation found"
    Just prev -> do
      let newMsg  = userMessage "What is my favorite language?"
          allMsgs = messages prev <> [newMsg]

      case allMsgs of
        (firstMsg : rest) -> do
          let req = chatRequest "qwen3.5:9b" (firstMsg :| rest)
          res <- chat client req
          case res of
            Left err -> print err
            Right resp -> case crMessage resp of
              Nothing -> pure ()
              Just botMsg -> do
                updatedTime <- getCurrentTime
                let updatedConvo = prev
                      { messages    = allMsgs <> [assistantMessage (messageContent botMsg)]
                      , lastUpdated = updatedTime
                      }
                saveConversationInMemory store updatedConvo
                putStrLn "History Saved!"
        [] -> pure ()