Exploring Claude: Streaming Messages Using Claude 3 Haiku

I work on a music app, and I have been using the normal message endpoint to fetch the JSON, decode the data, and display it on the screen.

I had this idea of streaming the data with animation instead of waiting for all of it to display. I thought it would be a cool feature to have, and it would make the app more engaging for the users. I have been loving Claude 3 Haiku for how fast and cheap it is, and I thought it would be perfect for streaming messages.

Here is my journey of exploring Claude 3 Haiku to stream messages in the music app.

Streaming Messages

According to the documentation, I just needed to add the parameter "stream": true, and it would send different updates. I was nervous about building a direct API integration as I have not worked with streaming data before, but the documentation made it sound easy enough.

So, I started by requesting the API. Here is what it looked like:

curl https://api.anthropic.com/v1/messages \
--header "anthropic-version: 2023-06-01" \
--header "anthropic-beta: messages-2023-12-15" \
--header "content-type: application/json" \
--header "x-api-key: $ANTHROPIC_API_KEY"" \
--data \
'{
"model": "claude-3-haiku-20240307",
"messages": [{"role": "user", "content": "Somebody Else by The 1975"}],
"system": "You are a knowledgeable music enthusiast with a deep understanding of various genres, artists, and songs. Your task is to recommend songs that are similar in style, genre, or mood to a given song. When providing recommendations, consider factors such as the artist'''s sound, lyrical themes, and overall vibe.\n\nFor each recommendation, provide the artist name and song title in the following format:\n\nArtist: artistName\nSong: songName\n---\nGenerate 20 high-quality song recommendations that are closely related to the given song. Separate each recommendation with "<>". Ensure that the recommendations are diverse yet coherent, showcasing your expertise in finding songs that capture the essence of the original song.\n\nFocus on providing accurate and relevant recommendations without any additional commentary or explanations.",
"max_tokens": 1024,
"stream": true
}'

And here is a part of what the response looks like:

event: message_start
data: {"type":"message_start","message":{"id":"msg_01FBtgoNaB3RsRB6nx8p31dX","type":"message","role":"assistant","content":[],"model":"claude-3-haiku-20240307","stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":181,"output_tokens":1}}          }

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}  }

event: ping
data: {"type": "ping"}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"<Artist"}               }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":":"}         }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" The"}            }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" Week"}        }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"nd"}             }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\nSong"}             }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":":"}    }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" Earned"}              }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" It"}          }

event: content_block_stop
data: {"type":"content_block_stop","index":0              }

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":303}       }

event: message_stop
data: {"type":"message_stop" }

As the data is streamed, I can see how the artist's name is on different lines and the same for the song. I have to figure out a way to fix that with Swift. There are no official client-side SDKs for Swift, so I need to handle these events myself.

Creating Structures to Decode Data

Now that we have our curl request, we need to translate it into Swift code. We'll start by defining some structs that will help us decode the JSON data that we receive from the API:

enum EventType: String, Codable {
  case contentBlockDelta = "content_block_delta"
  case messageStop = "message_stop"
  case contentBlockStart = "content_block_start"
  case messageStart = "message_start"
  case ping = "ping"
  case contentBlockStop = "content_block_stop"
  case messageDelta = "message_delta"
}

struct StreamData: Codable {
  let type: EventType
  let index: Int?
  let delta: Delta?
}

struct Delta: Codable {
  let text: String?
}

First, we have an enum called EventType, that represents the different types of events that we can receive in the stream. Each event type is represented by a string, and we conform to the Codable protocol so that we can easily decode the JSON data.

Next, we have a struct called StreamData, which will represent the data that we receive for each event. It contains an EventType, an optional index, and an optional Delta. The Delta struct contains an optional text property, which will hold the text data for content block delta events.

Creating the Configuration

We also have a struct called APIConfig, which contains some configuration data for our API requests. It includes a maxTokens property, which sets the maximum number of tokens that we want to receive in the response, and a key property, which holds the API key:

struct APIConfig {
  static let maxTokens = 1024
  static let key = "YOUR-KEY-ONLY-FOR-TESTING"
  static let systemPrompt = """
  You are a knowledgeable music enthusiast with a deep understanding of various genres, artists, and songs. Your task is to recommend songs that are similar in style, genre, or mood to a given song. When providing recommendations, consider factors such as the artist's sound, lyrical themes, and overall vibe.

  For each recommendation, provide the artist name and song title in the following format:

  Artist: artistName
  Song: songName
  ---

  Generate 20 high-quality song recommendations that are closely related to the given song. Separate each recommendation with "<>". Ensure that the recommendations are diverse yet coherent, showcasing your expertise in finding songs that capture the essence of the original song.

  Focus on providing accurate and relevant recommendations without any additional commentary or explanations.
  """
}

The APIConfig struct also includes a systemPrompt property, which is a string that defines the system prompt for our model. I should not have put it here, but I did anyway to avoid populating the request code.

Creating the Request

Next, we have the APIRequest struct, where the initialiser sets the url property to the Anthropic API endpoint, sets the request property to a URLRequest object with the url property, and sets the requestBody property to a dictionary with the necessary parameters for our request.

struct APIRequest {
  let userPrompt: String
  let url: URL
  var request: URLRequest
  let requestBody: [String: Any]
  
  init(userPrompt: String) {
    self.userPrompt = userPrompt
    self.url = URL(string: "https://api.anthropic.com/v1/messages")!
    self.request = URLRequest(url: url)
    self.requestBody = [
      "model": "claude-3-haiku-20240307",
      "messages": [["role": "user", "content": userPrompt]],
      "system": APIConfig.systemPrompt,
      "max_tokens": APIConfig.maxTokens,
      "stream": true
    ]
    
    configureRequest()
  }
  
  private func configureRequest() {
    request.httpMethod = "POST"
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")
    request.setValue("2023-06-01", forHTTPHeaderField: "anthropic-version")
    request.setValue("messages-2023-12-15", forHTTPHeaderField: "anthropic-beta")
    request.setValue(APIConfig.key, forHTTPHeaderField: "x-api-key")
    request.httpBody = try? JSONSerialization.data(withJSONObject: requestBody)
  }
}

Making the request:

Finally, we have the ContentViewModel class, which contains a currentSongData property that will hold the current song data as it is streamed in. The streamSongsData method creates an APIRequest object with a user prompt of an example prompt of searching songs similar to "Somebody Else by The 1975", and then creates a stream of bytes using the URLSession.shared.bytes(for:) method.

@MainActor
class ContentViewModel: ObservableObject {
  private var currentSongData = ""
  
  func streamSongsData() async throws {
    let userPrompt = "Somebody Else by The 1975"
    let apiRequest = APIRequest(userPrompt: userPrompt)
    
    let (stream, _) = try await URLSession.shared.bytes(for: apiRequest.request)
}

Streaming the Data

Here is the code for streaming the data. It is written using Claude 3 Opus, and modified by me to match the requirements:

@MainActor
class ContentViewModel: ObservableObject {
  private var currentSongData = ""

  func streamSongsData() async throws {
    let userPrompt = "Somebody Else by The 1975"
    let apiRequest = APIRequest(userPrompt: userPrompt)

    let (stream, _) = try await URLSession.shared.bytes(for: apiRequest.request)

    for try await line in stream.lines {
      if line.hasPrefix("data: ") {
        let data = line.dropFirst(6)
        if let streamData = data.data(using: .utf8),
           let streamDataObject = try? JSONDecoder().decode(StreamData.self, from: streamData) {
          switch streamDataObject.type {
            case .contentBlockDelta:
              handleContentBlockDelta(streamDataObject)
            case .messageStop:
              break
            default:
              break
          }
        }
      }
    }
  }
}

First, we have a for-await-in loop that reads the stream line by line. If a line starts with "data: ", it means that the line contains actual data, so we extract the data and decode it into a StreamData object.

Next, we handle the` StreamData` object in the handleContentBlockDelta function:

private func handleContentBlockDelta(_ streamDataObject: StreamData) {
  guard let index = streamDataObject.index, index == 0,
        let textDelta = streamDataObject.delta?.text else {
    return
  }
  
  currentSongData += textDelta
  
  if currentSongData.contains("<>") {
    let songDataArray = currentSongData.components(separatedBy: "<>")
    processSongData(songDataArray)
    currentSongData = ""
  }
}

We check if the index is 0 and if the text delta is not nil. If both conditions are met, we append the text delta to the currentSongData string.

Once we have the entire song data, we split it into an array of strings using the "><" separator. We then loop through the array and process each song data:

private func processSongData(_ songDataArray: [String]) {
  for songData in songDataArray {
    let lines = songData.trimmingCharacters(in: .whitespacesAndNewlines).split(separator: "\n")
    guard lines.count == 2 else {
      continue
    }

    let artistName = String(lines[0].dropFirst("Artist: ".count))
    let songName = String(lines[1].dropFirst("Song: ".count))
    let searchQuery = artistName == songName ? songName : "\(songName) \(artistName)"

    Task {
      await searchAndAppendSong(searchQuery, songName: songName, artistName: artistName)
    }
  }
}

For each song data, we split it into lines and extract the artist and song name. We then create a search query and call the searchAndAppendSong function to search for the song in Apple Music using MusicKit:

private func searchAndAppendSong(_ searchQuery: String, songName: String, artistName: String) async {
  do {
    let response = try await MCatalog.searchSongs(for: searchQuery)

    if let matchedSong = response.first(where: { $0.title.localizedCaseInsensitiveContains(songName) }) {
      print("MATCH FOUND FOR \(songName) by \(artistName)")
      print(matchedSong)

      await appendSong(matchedSong)
    } else {
      print("MATCH NOT FOUND FOR \(songName) by \(artistName)")
    }
  } catch {
    print("Error searching for song: \(error)")
  }
}

In the searchAndAppendSong function, we search for the song using the MCatalog.searchSongs function.

This is a method from my MusadoraKit framework, the ultimate companion for MusicKit.

If we find a match, we print a message and call the appendSong function to add the song to the songs array. If we find no match, we print a message indicating that no match was found.

private func appendSong(_ song: Song) {
  withAnimation {
    songs += MusicItemCollection(arrayLiteral: song)
  }
}

And with that, here is the code on the UI side:

struct ContentView: View {
  @StateObject private var model = ContentViewModel()

  var body: some View {
    ScrollView {
      ForEach(model.songs, content: { song in
        HStack {
          if let artwork = song.artwork {
            ArtworkImage(artwork, height: 60)
              .cornerRadius(8)

            VStack(alignment: .leading) {
              Text(song.title).bold()

              Text(song.artistName)
            }

            Spacer()
          }
        }
        .transition(.opacity)
      })
    }
    .padding()
    .task {
      do {
        if await MusicAuthorization.request() == .authorized {
          try await model.streamSongsData()
        }
      } catch {
        print(error)
      }
    }
  }
}

Conclusion

And that is it for streaming data using Claude 3 Haiku and some Foundation code.

One of the key benefits of using Claude 3 Haiku for streaming data is its ability to handle large amounts of data in real-time for dirt-cheap pricing! Happy coding!