Thumbor

In past projects with user uploaded images I used a task queue to process images after they were uploaded into various resolutions and to include watermarks. This method is simple but if down the road you find the image resolutions are not ideal or the watermark needs to change then you have to write some one-off tasks to redo all of the images (hope you kept the original) and depending on your file serving implemenation and/or filename convention you may need to update your frontend service too. For my Bilolok project I wanted to have on-demand image resizing and watermark capabilities.

Thumbor is an on-demand image resizing, cropping and filtering service which fits my needs. In fact, I was able to add into Bilolok by including just two services into our docker-compose.yml config and by adding a short function into the backend API.

The two services added are the nginx-proxy service which acts as an image caching layer infront of the thumbor service. So when an image is not found in the cache then the request goes all the way to Thumbor and only then would image processing take place and not on every single request.

services:
  thumbor:
    image: minimalcompact/thumbor
    volumes:
      - "${DATA_LOCAL_DIR}/:/data/uploads/"
    env_file:
      - .env
    environment:
      - VIRTUAL_HOST=image.bilolok.com
      - THUMBOR_NUM_PROCESSES=1
      - AUTO_WEBP=True
      - ALLOW_UNSAFE_URL=False
      - SECURITY_KEY=${THUMBOR_SECURITY_KEY}
      - LOADER=thumbor.loaders.file_loader
      - FILE_LOADER_ROOT_PATH=/data/uploads/
      # nginx-proxy does caching automatically, so no need to store the result storage cache
      # (this greatly speeds up and saves on CPU)
      - RESULT_STORAGE=thumbor.result_storages.no_storage
      - RESULT_STORAGE_STORES_UNSAFE=True
      - STORAGE=thumbor.storages.file_storage
    restart: always
 
  nginx-proxy:
    image: minimalcompact/thumbor-nginx-proxy-cache
    environment:
      - DEFAULT_HOST=image.bilolok.com
      - PROXY_CACHE_SIZE=10g
      - PROXY_CACHE_MEMORY_SIZE=500m
      - PROXY_CACHE_INACTIVE=168h
    volumes:
      # this is essential for nginx-proxy to detect docker containers, scaling etc
      # see https://github.com/jwilder/nginx-proxy
      - /var/run/docker.sock:/tmp/docker.sock:ro
      # mapping cache folder, to persist it independently of the container
      - bilolok-img-cache-data-prod:/var/cache/nginx
    ports:
      - "8888:80"
    restart: always
  
volumes:
  bilolok-img-cache-data-prod:

In the API codebase I keep a helper file which has configured libthumbor ready to use Thumbor’s secret key. We use this because otherwise a malicious user could spam our backend for millions of variations of the same image.

from libthumbor import CryptoURL
from app.core.config import settings
 
img_crypto_url = CryptoURL(key=settings.THUMBOR_SECURITY_KEY)

And lastly, I use the img_crypto_url instance to generate safe URLs that Thumbor will accept on the client side.

def make_src_url(self, image: Image, width: int, height: int, **kwargs) -> str:
    uri = img_crypto_url.generate(
        width=width,
        height=height,
        smart=True,
        image_url=str(
            Image.build_filepath(image.nakamal.id, image.file_id, image.filename)
        ),
        **kwargs,
    )
    return "{}{}".format(settings.THUMBOR_SERVER, uri)
 
def make_src_urls(self, image: ImageSchema) -> ImageSchema:
    image.src = self.make_src_url(
        image,
        height=720,
        width=1280,
        full_fit_in=True,
        filters=[
            "watermark(/images/watermark.png,20,-20,20,30)",
        ],
    )
    image.msrc = self.make_src_url(image, height=32, width=32, full_fit_in=True)
    image.thumbnail = self.make_src_url(image, height=200, width=200)
    return image

So when the user requests an image object from the API they are returned a response like the one below.

{
  "id": "7bebc655-ee33-4f44-bea9-3244c57d1d22",
  "created_at": "2021-12-02T06:28:02.669341+00:00",
  "src": "https://image.bilolok.com/ls3NgsypowbLm8YQvK79UOINcNk=/full-fit-in/1280x720/smart/nakamals/f9/f9b1994d-bfc7-4d2b-8225-47e9b5b9dd16/0ee90ed94acc56f000579f2d13f3866f.jpg",
  "msrc": "https://image.bilolok.com/e7wazqFPJMko1obtAFoCNoDN0rw=/full-fit-in/32x32/smart/nakamals/f9/f9b1994d-bfc7-4d2b-8225-47e9b5b9dd16/0ee90ed94acc56f000579f2d13f3866f.jpg",
  "thumbnail": "https://image.bilolok.com/RDkUYpi5ZM8hxwH_6iCWn8yCD8M=/200x200/smart/nakamals/f9/f9b1994d-bfc7-4d2b-8225-47e9b5b9dd16/0ee90ed94acc56f000579f2d13f3866f.jpg",
  "user": {
    "id": "d509f517-3042-4595-89bc-4f93be36acf7",
    "avatar": "https://image.bilolok.com/F5cOi-sYJg5KAdL81_3mmmeRCgY=/100x100/smart/users/d5/d509f517-3042-4595-89bc-4f93be36acf7/default.png"
  }
}

Conclusion

If at a later point I want to offer larger images or change the watermark I only need to update the backend API and new images will be generated.