Uppy and Tus

As part of my Bilolok project’s goal to be friendly to poor network conditions I decided to use the Tus protocol for resumeable uploads that would mean users would not lose their upload progress when the network had a hiccup.

Tus is an open protocol for resumable file uploads which pairs well with Uppy on the frontend. On the backend, I didn’t need to implement the protocol myself and instead I used the official tusd server by including it in the docker-compose.yml config. To actually integrate the file uploads into Bilolok I added a custom hook in the backend API to handle the uploaded files.

Frontend

upload.vue

<template>
  <div>
    <DashboardModal
      :uppy="uppy"
      :open="open"
      :plugins="[]"
      :props="{theme: 'light'}"
    />
  </div>
</template>
 
<script>
import { mapGetters } from 'vuex';
import { DashboardModal } from '@uppy/vue';
import { uploadDomain } from '@/env';
 
import '@uppy/core/dist/style.css';
import '@uppy/dashboard/dist/style.css';
 
import { Uppy } from '@uppy/core';
import Tus from '@uppy/tus';
 
export default {
  name: 'NakamalImageUpload',
  props: ['nakamal', 'open'],
  components: {
    DashboardModal,
  },
  computed: {
    ...mapGetters({
      user: 'auth/user',
      token: 'auth/token',
    }),
    uppy() {
      return new Uppy({
        meta: {
          NakamalID: this.nakamal.id,
        },
      })
        .use(Tus, {
          endpoint: `${uploadDomain}/files/`,
          chunkSize: 2_000_000,
          headers: {
            authorization: `Bearer ${this.token}`,
          },
        })
        .on('complete', () => {
          this.$emit('close-modal');
        })
        .on('upload-error', (_, error) => {
          this.$emit('close-modal');
          this.$store.dispatch('notify/add', {
            title: 'Upload Error',
            text: 'Upload failed due to some unknown issues. Try again later.',
            type: 'warning',
          });
        });
    },
  },
  beforeDestroy() {
    this.uppy.close();
  },
};
</script>

Backend

The tusd instance will handle the incoming uploads and communicate with our API via the -hooks-http value, including forwarding the Authorization header from the client so our API can authenticate the user uploading the file.

docker-compose.yml

tusd:
  image: tusproject/tusd
  ports:
    - "8070:8070"
  volumes:
    - "${IMAGES_LOCAL_DIR}/uploads/:/data/"
  env_file:
    - .env
  command: -port 8070 -upload-dir /data/ -behind-proxy -hooks-http https://example.com/tus-hook -hooks-http-forward-headers authorization -hooks-enabled-events pre-create,post-finish

Lastly, our backend built with FastAPI handles the incoming webhook requests from the tusd instance to authenticate uploads and when the upload completes it handles the file and adds it to our database.

Do take note this example assumes the tusd instance is saving the files to the same locally accessible filesystem as the backend API.

tus.py

@router.post("/tus-hook", include_in_schema=False)
async def tus_hook(
    db: AsyncSession = Depends(get_db),
    user_manager: UserManager = Depends(get_user_manager),
    *,
    hook_name: str = Header(...),
    tusdIn: Any = Body(...),
) -> Any:
    """
    Hook for tusd.
    """
    scheme, _, token = tusdIn.get("HTTPRequest").get("Header").get("Authorization")[0].partition(" ")
    try:
        user_id = get_user_id_from_jwt(token)  # psuedo code
        if user_id is None:
            raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST)
        # Check user is verified and active or superuser
        user = await user_manager.get(user_id)
        status_code = status.HTTP_401_UNAUTHORIZED
        if user:
            status_code = status.HTTP_403_FORBIDDEN
            if not user.is_active:
                status_code = status.HTTP_401_UNAUTHORIZED
                user = None
            if not user.is_verified or not user.is_superuser:
                user = None
        if not user:
            raise HTTPException(status_code=status_code)
    except jwt.PyJWTError:
        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED)
 
    # Check Nakamal exists
    nakamal_id = tusdIn.get("Upload").get("MetaData").get("NakamalID")
    nakamal = get_nakamal(nakamal_id)  # psuedo code
    if not nakamal:
        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Nakamal not found.")
    if hook_name == "post-finish":
        file_id = tusdIn.get("Upload").get("ID")
        filename = tusdIn.get("Upload").get("MetaData").get("filename")
        filetype = tusdIn.get("Upload").get("MetaData").get("filetype")
        try:
            tusUpload = Path(settings.IMAGES_LOCAL_DIR) / "uploads" / file_id
            assert tusUpload.exists()
            save_file(tusUpload, nakamal_id=nakamal_id, file_id=file_id, filename=filename)  # psuedo code
            # Remove Tus `.info` file
            tusInfo = Path(str(tusUpload) + ".info")
            tusInfo.unlink()
        except Exception as exc:
            raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR)
        try:
            image = create_image(
                file_id=file_id,
                filename=filename,
                filetype=filetype,
                user_id=user_id,
                nakamal_id=nakamal_id,
            )  # psuedo code
        except Exception as exc:
            raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR)

Conclusion

All-in-all this was a relatively easy setup for the goal of making user uploads resilient to poor network conditions.