Skip to main content

Running in Docker

Running headless browsers in Docker requires a lot of setup to do it right. But there's no need to worry about that, because we already created base images that can now be freely used. We use them every day on the Apify Platform.

All images can be found in their GitHub repo and in our DockerHub.

Overview​

Browsers are pretty big, so we try to provide a wide variety of images to suit the specific needs. Here's a full list of our Docker images.

Versioning​

Each image is tagged with up to 2 version tags, depending on the type of the image. One for Node.js version and second for pre-installed web automation library version. If we use the image name without a version tag, we'll always get the latest available version.

We recommend always using at least the Node.js version tag in the production Dockerfiles. It will ensure that a future update of Node.js will not break our automations.

Node.js versioning​

Our images are built with multiple Node.js versions to ensure backwards compatibility. Currently, Node.js versions 14, 15 and 16 are supported. To select the preferred version, the appropriate number as the image tag should be used.

# Use Node.js 14
FROM apify/actor-node:14
# Use Node.js 16
FROM apify/actor-node-playwright:16

Automation library versioning​

Images that include a pre-installed automation library, which means all images that include puppeteer or playwright in their name, are also tagged with the pre-installed version of the library. For example, apify/actor-node-puppeteer-chrome:16-13.7.0 comes with Node.js 16 and Puppeteer v13.7.0. If we try to install a different version of Puppeteer into this image, we may run into compatibility issues, because the Chromium version bundled with puppeteer will not match the version of Chrome we pre-installed.

Similarly apify/actor-node-playwright-firefox:14-1.21.1 runs on Node.js 14 and is pre-installed with the Firefox version that comes with v1.21.1.

Installing apify/actor-node-puppeteer-chrome (without a tag) will install the latest available version of Node.js and puppeteer.

Pre-release tags​

We also build pre-release versions of the images to test the changes we make. Those are typically denoted by a beta suffix, but it can vary depending on our needs. If we need to try a pre-release version, we can do it like this:

# Without library version.
FROM apify/actor-node:16-beta
# With library version.
FROM apify/actor-node-playwright-chrome:16-1.10.0-beta

Best practices​

  • Node.js version tag should always be used.
  • The automation library version tag should be used for added security.
  • Asterisk * should be used as the automation library version in our package.json files.

It makes sure the pre-installed version of Puppeteer or Playwright is not re-installed on build. This is important, because those libraries are only guaranteed to work with specific versions of browsers, and those browsers come pre-installed in the image.

FROM apify/actor-node-playwright-chrome:16
{
    "dependencies": {
        "apify": "^3.0.0",
        "@crawlee/playwright": "^3.0.0",
        "playwright": "*"
    }
}

Warning about image size​

Browsers are huge. If we don't need them all in our image, it's better to use a smaller image with only the one browser we need.

We should also be careful when installing new dependencies. Nothing prevents us from installing Playwright into theactor-node-puppeteer-chrome image, but the resulting image will be about 3 times larger and extremely slow to download and build.

When we use only what we need, we'll be rewarded with reasonable build and start times.

Apify Docker Images​

actor-node​

This is the smallest image we have based on Alpine Linux. It does not include any browsers, and it's therefore best used with CheerioCrawler. It benefits from lightning fast builds and container startups.

​PuppeteerCrawler, PlaywrightCrawler and other browser based features will NOT work with this image.

FROM apify/actor-node:16

actor-node-puppeteer-chrome​

This image includes Puppeteer (Chromium) and the Chrome browser. It can be used with CheerioCrawler and PuppeteerCrawler, but NOT with PlaywrightCrawler.

The image supports XVFB by default, so we can run both headless and headful browsers with it.

FROM apify/actor-node-puppeteer-chrome:16

actor-node-playwright​

A very large and slow image that can run all Playwright browsers: Chromium, Chrome, Firefox, WebKit. Everything is installed. If we need to develop or test with multiple browsers, this is the image to choose, but in most cases, it's better to use the specialized images below.

FROM apify/actor-node-playwright:16

actor-node-playwright-chrome​

Similar to actor-node-puppeteer-chrome, but for Playwright. We can run CheerioCrawler and PlaywrightCrawler, but NOT PuppeteerCrawler.

It uses the PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD environment variable to block installation of more browsers into our images (to keep them small). If we want more browsers, either the actor-node-playwright image should be chosen or this env var should be overridden.

The image supports XVFB by default, so we can run both headless and headful browsers with it.

FROM apify/actor-node-playwright-chrome:16

actor-node-playwright-firefox​

Same idea as actor-node-playwright-chrome, but with Firefox pre-installed.

FROM apify/actor-node-playwright-firefox:16

actor-node-playwright-webkit​

Same idea as actor-node-playwright-chrome, but with WebKit pre-installed.

FROM apify/actor-node-playwright-webkit:16

Example Dockerfile​

To use the above images, it's necessary to have a Dockerfile. We can either use this example, or bootstrap our projects with the Crawlee CLI which automatically adds the correct Dockerfile into our project folder.

# First, specify the base Docker image. You can read more about
# the available images at https://crawlee.dev/docs/guides/docker-images
# You can also use any other image from Docker Hub.
# The 16 represents the version of Node.js you want to use.
FROM apify/actor-node:16

# Second, copy just package.json and package-lock.json since it should be
# the only file that affects "npm install" in the next step, to speed up the build
COPY package*.json ./

# Install NPM packages, skip optional and development dependencies to
# keep the image small. Avoid logging too much and print the dependency
# tree for debugging
RUN npm --quiet set progress=false \
 && npm install --only=prod --no-optional \
 && echo "Installed NPM packages:" \
 && (npm list --only=prod --no-optional --all || true) \
 && echo "Node.js version:" \
 && node --version \
 && echo "NPM version:" \
 && npm --version

# Next, copy the remaining files and directories with the source code.
# Since we do this after NPM install, quick build will be really fast
# for most source file changes.
COPY . ./

# Optionally, specify how to launch the source code of your crawler.
# By default, Apify's base Docker images define the CMD instruction
# that runs the Node.js source code using the command specified
# in the "scripts.start" section of the package.json file.
# In short, the instruction looks something like this:
#
# CMD npm start
Last updated on by Andrey Bykov