Data on the Go: Effortless Streaming

For one of my AI projects, I needed to make the app more interactive/responsive when generating travel plans. Initially, I was considering using WebSockets, but then I remembered that this is achievable with a basic HTTP GET call

Wait, what?

Yes, you heard it correctly. We can enhance our GET request so that we can stream data to the client.

But how can we do that?

To be honest, we can achieve this with the technology you already use in your day-to-day work. For example, did you know that the fetch api supports readable streams? Yes, you heard that correctly. With some minor changes, we can turn our Fetch API into a readable stream.

Instead of using the response.json or response.text methods to retrieve the data, we can use the response.body.getReader.

The getReady stream is still a promise. But now we get an object back which gives use the value of the server and a done flag which tells us that the backend has send all data. With that information we can create a recursive promise which is listening to new data until we get a done confirmation. And that is everything related to the frontend.

Basic example of the frontend

const decoder = new TextDecoder(); // Used for converting Uint8Array to UTF8
const response = await fetch("http://localhost:3000/api");

if (!response.body) {
  return "";
}

const reader = response.body.getReader();

const readStream = () => {
  reader.read().then(({ done, value, ...props }) => {
    // Stream is finished
    if (done) {
      return;
    }

    // Display the streamed data
    const payload = decoder.decode(value);
    callback(payload, count);

    count++;

    // Continue reading the stream
    readStream();
  });
};

// Start reading the stream
readStream();

Now that we have implemented the frontend site, let’s make our backend response a stream, for the backend side, we will go with Fastify. To create this functionality, we have two ways to do it. One is with the help of reply.raw, which is not recommended, and the second one is with the help of streams.

For more info about streams, check out my previous blog.

To make this work, we will create a basic readable stream, which will write to the reply.send. The send method supports streams by default, so the only thing we really have to do is define the logic for when to send the data. In our case, we do this when requesting the endpoint and every 2 seconds until we exceed a count of 10:

const fastify = require("fastify")();
const path = require("path");
const { Readable } = require("stream");

fastify.get("/api", (req, reply) => {
  const contentStream = new Readable({ read() {} });

  // when sending streams we have to change the content
  // type to this otherwise it will not work
  reply.header("Content-Type", "text/event-stream");

  contentStream.push("Hello word example"); // Sending the Hello world example
  let count = 0;

  setInterval(() => {
    contentStream.push(`The count is ${count}`); // Send message every 2 seconds
    count++;

    if (count > 10) {
      contentStream.push(null); // close the stream
    }
  }, 2000);

  reply.send(contentStream);
});

And that’s it here we have a simple implementation for streaming multiple chunks without having to use websockets. Seems quite simple what do you thing?

Data on the Go: Effortless Streaming

Resources