Building a PWA Outbox Feature

I needed my PWA to gracefully handle network errors for users’ submissions so they would be stored in an outbox to send later. This would be kind of like the experience of using Gmail or your choice of messanging app while offline. You can still click send but the message won’t actually deliver until you regain internet access.

To do this we need to break the problem down into a few parts. First, a queue to store the failed requests. Second, a hook so failed requests can be intercepted and placed into the queue. Lastly, a couple UI elements to interact with the queue. The process will look something like the following.

Why Not Use Workbox’s Background Sync Plugin?

I think this belongs in a post of its own.

Building the Queue

I used IndexedDB for the queue. I wrote a few helper functions for adding, removing and fetching items from the queue becuase later we need to import these functions into both the service worker and the frontend.

retry-queue.ts

import { openDB } from 'idb';
import { RETRY_QUEUE } from '../constants';
import type { Movement } from '../services/directus';
 
export const initQueue = async () => {
  console.log('[Queue] Init');
  await openDB(RETRY_QUEUE, 1, {
    upgrade(db) {
      if (!db.objectStoreNames.contains('queue')) {
        console.log('[Queue] Creating store')
        db.createObjectStore('queue', { keyPath: 'id' });
      }
    },
  });
};
export const addToQueue = async (payload: Movement) => {
  console.log('[Queue] Add one', payload);
  const db = await openDB(RETRY_QUEUE, 1);
  const tx = db.transaction('queue', 'readwrite');
  const id = Date.now()
  await tx.store.add({
    'id': id,
    payload,
  });
  await tx.done;
};
 
export const deleteFromQueue = async (id: number) => {
  console.log('[Queue] Delete one', id)
  const db = await openDB(RETRY_QUEUE, 1);
  const tx = db.transaction('queue', 'readwrite');
  await tx.store.delete(id);
  await tx.done;
}
 
export const getQueue = async () => {
  console.log('[Queue] Get all')
  const db = await openDB(RETRY_QUEUE, 1);
  const tx = db.transaction('queue', 'readonly');
  const queue = await tx.store.getAll();
  // add logic for maxRetention if you want that here
  await tx.done;
  return queue.sort((a, b) => {
    return a.date_reported > b.date_reported ? 1 : -1;
  });
};

Later be sure to call initQueue in your entrypoint of your frontend.

Intercepting Failed Requests

In our service worker we create a hook to handle failed requests and place them into the queue. For my usecase I only need to filter for POST requests to a single /items/Movements endpoint so the logic is quite straightforward.

sw.ts

self.addEventListener('fetch', (event: FetchEvent) => {
  if (event.request.method !== 'POST') {
    return;
  }
 
  if (!event.request.url.includes('/items/Movements')) {
    return;
  }
 
  const bgSyncLogic = async (event: FetchEvent) => {
    try {
      const response = await fetch(event.request.clone());
      if (!response.ok) {
        throw new Error('Server error was not ok');
      }
      return response;
    } catch (error) {
      console.error('[SW] Failed to POST, adding to retry queue', error);
      const clonedrequest = event.request.clone();
      const payload = await clonedrequest.json();
      try {
        await addToQueue(payload);
      } catch (queueError) {
        console.error('[Queue] Failed to save to queue', queueError);
        throw queueError
      }
      throw error;
    }
  };
 
  event.respondWith(bgSyncLogic(event));
});

Frontend UI

On the frontend I first wrap the queue’s helper functions in a Pinia store. At the moment it looks like it adds very little value over the helper functions itself but later I will expand upon it by adding calls to the user notification library, user authentication store and the service worker.

queueStore.ts

import {
  addToQueue,
  deleteFromQueue,
  getQueue,
  initQueue,
} from '../service-worker/retry-queue';
 
interface QueueItem {
  id: number;
  payload: any;
};
 
interface State {
  items: QueueItem[];
  isSyncing: boolean;
  isSyncingTimer: any | null;
};
 
export const useRetryQueueStore = defineStore('queue', {
  state: (): State => ({
    items: [],
    isSyncing: false,
    isSyncingTimer: null,
  }),
  getters: {
    count(): number {
      return this.items.length;
    },
  },
  actions: {
    async init() {
      await initQueue();
      await this.getItems();
    },
    async getItems() {
      const items = await getQueue();
      this.items = items;
    },
    async addItem(payload: Movement) {
      await addToQueue(payload);
    },
    async deleteItem(id: number) {
      await deleteFromQueue(id);
    },
    async syncQueue() {
      ...
    },
    async setQueueComplete() {
      ...
    },
  },
});

Then I built out a view for the user to interact with the queue.

QueueView.vue

<template>
<div>
  <div v-if="isLoggedIn" role="alert">
    <IconThumbsUp v-if="count === 0" />
    <IconLoading v-else-if="isSyncing" />
    <IconInfo v-else />
    <div>
      <span v-if="count === 0">Queue is empty.</span>
      <span v-else-if="!isSyncing">{{ count }} items are waiting in the queue.</span>
      <span v-else>Attempting to submit queue items.</span>
    </div>
    <button :disabled="count === 0 || isSyncing" @click="syncPendingItems">Sync</button>
  </div>
  <div v-else>
    <IconInfo />
    <span>You must login again. Then you may submit items in the queue.</span>
    <router-link :to="{ name: 'login' }">Login</router-link>
  </div>
 
  <div class="divider"></div>
 
  <div v-if="count > 0">
    <table>
      <tbody>
        <tr v-for="item in items" :key="item.id" @click="setSelected">
          <!-- queue item payload content and button to open modal to delete item -->
        </tr>
      </tbody>
    </table>
  </div>  
</template>
 
<script setup lang="ts">
import { onMounted, ref, watch } from 'vue';
import { storeToRefs } from 'pinia';
 
const queueStore = useQueueStore();
const { count, isSyncing, items } = storeToRefs(queueStore);
 
const selected = ref();
const setSelected = (item) => {
  selected.value = item;
};
 
const deleteSelectedItem = async () => {
  if (!selected.value) return;
  await queueStore.deleteItem(selected.value.id);
  selected.value = undefined;
  await queueStore.getItems();
};
 
const syncPendingItems = async () => {
  await queueStore.syncQueue();
};
 
watch(isSyncing, async (newValue: boolean) => {
  if (!newValue) {
    await queueStore.getItems();
  }
});
 
onMounted(async () => {
  await queueStore.getItems();
});
<script>

Elewhere you may use the count value to display a badge or other UI element to indicate the queue is not empty.

Message Passing

Lastly, I need the means to pass messages from the frontend to the service worker and vice versa. This will allow the user to retry the queue manually and for the service worker to send back the results after retrying the queue for the purpose of UI notifications.

To begin I store a constant RETRY_MESSAGE_KEY for the key for my event messages.

constants.ts

export const RETRY_MESSAGE_KEY = 'RETRY_QUEUE';

This key is used in my auth store to tell the service worker I want to resubmit the items in the queue. I also must point out that my particular authentication pattern requires I submit the user’s authentication token in the message payload. This is because the service worker does not have access to the browser’s local storage and hence can not read it for the purpsoe of updating the request headers for the queue items so we manually pass it to the service worker. A different authentication pattern may not require this.

Another alternative would be to store a refresh token with the original payload then including logic in your service worker to fetch its own auth token before attempting to submit an item from the queue. I found that to be a bit too complicated for my needs so I opted for this pattern. But, if you wanted true background sync without requiring the user to be actively online to submit the queue items this would be a good approach.

authStore.ts

actions: {
  async syncRetryQueue() {
    console.log('[Retry] Sending message to retry queue');
    const authStore = useAuthStore();
    const notifyStore = useNotifyStore();
    const authToken = authStore.authToken;
    if ('serviceWorker' in navigator && navigator.serviceWorker.controller) {
      if (!authToken) {
        notifyStore.notify('Failed to retry queue; you are not logged in.', NotificationType.Error);
        return;
      }
      navigator.serviceWorker.controller.postMessage({
        type: RETRY_MESSAGE_KEY,
        payload: {
          authToken,
        },
      });
      this.isSyncing = true;
      // A failsafe to stop isSyncing in case of unexpected error from SW
      this.isSyncingTimer = setTimeout(async () => {
        await this.setRetryComplete();
      }, 10000);
    }
  },
  async setRetryComplete() {
    console.log('[Retry] Updating retry state as complete');
    clearTimeout(this.isSyncingTimer);
    this.isSyncing = false;
    await this.getItems();
  },
},

The service worker will receive the message and replay the queue with the provided authToken. Along the way we count the success rate of the retries and return a message back to the frontend via messageClientsRetryResults. This is like the authStore before sending a message to the service worker but in reverse and I reuse the RETRY_MESSAGE_KEY constant.

sw.ts

self.addEventListener('message', (event: ExtendableMessageEvent) => {
  console.log('SW received message', event.data);
  if (event.data && event.data.type === RETRY_MESSAGE_KEY) {
    const authToken = event.data.payload.authToken;
    replayQueue(authToken);
  }
});
 
const replayQueue = async (authToken: string) => {
  console.log('[Queue] Attempting retry queue');
  let successCount = 0;
  let failureCount = 0;
  const queue = await getQueue();
  try {
    for (const item of queue) {
      try {
        console.log('retrying item', item);
        await fetch(`${DIRECTUS_URL}/items/Movements`, {
          method: 'POST',
          headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${authToken}`
          },
          body: JSON.stringify(item.payload)
        });
        console.log('[Queue] Retry Success; will delete from queue', item.id);
        await deleteFromQueue(item.id);
        successCount += 1;
      } catch (error) {
        // Request fails, leave it in the queue
        console.error('[Queue] Retry Failed', error);
        failureCount += 1;
      }
    }
  } finally {
    messageClientsRetryResults(successCount, failureCount);
  }
};
 
const messageClientsRetryResults = (success: number = 0, failure: number = 0) => {
  self.clients.matchAll().then((clients) => {
    clients.forEach((client) =>
      client.postMessage({
        type: RETRY_MESSAGE_KEY,
        success: success,
        failure: failure,
      })
    );
  });
});

Back on the frontend I have an event listener which listens for the same RETRY_MESSAGE_KEY value and then uses the UI notification library to inform the user of the results.

useRetryQueueEventListenee.ts

import { onMounted, onUnmounted } from 'vue';
import { RETRY_MESSAGE_KEY } from '../constants';
import { NotificationType, useNotifyStore } from '../stores/notifyStore';
import { useQueueStore } from '../stores/queueStore';
 
export default function useRetryQueueEventListener() {
  const queueStore = useQueueStore();
  const notifyStore = useNotifyStore();
 
  const notifyRetryQueueEvent = (event: any) => {
    console.log('retry queue event', event);
    if (event.data && event.data.type === RETRY_MESSAGE_KEY) {
      // Update `isSyncing` status to mark end of SW side of work
      queueStore.setRetryComplete();
 
      // Notify user of results
      const successCount = event.data?.success || 0;
      if (successCount > 0) {
        notifyStore.notify(
          `${successCount} container movements in the retry queue have been successfully submitted.`,
          NotificationType.Success
        );
      }
      const failureCount = event.data?.failure || 0;
      if (failureCount > 0) {
        notifyStore.notify(
          `${failureCount} container movements in the retry queue have failed to submit. Try again later.`,
          NotificationType.Error
        );
      }
    }
  };
 
  onMounted(() => {
    navigator.serviceWorker.addEventListener('message', notifyRetryQueueEvent);
  });
 
  onUnmounted(() => {
    navigator.serviceWorker.removeEventListener('message', notifyRetryQueueEvent);
  });
}

Finally, as mentioned way back in the beginning of this article, the queue store and the event listener on our service worker need to be initalized at the start of the application.

App.vue

<script setup lang="ts">
import { useQueueStore } from './stores/queueStore';
import useRetryQueueEventListener from './composables/useRetryQueueEventListener';
 
const queueStore = useQueueStore();
queueStore.init();
 
useRetryQueueEventListener();
</script>

Conclusion

Again, my requirements are quite straightforward and therefore I could take some shortcuts in the design of this outbox. For an outbox for multiple endpoints you may need to store the endpoint with the payload for your queue or some other solution. No matter I hope this can be a good starting point for your PWA’s outbox feature.