Introduction

Source image for body model
Image with rendered body model
A source image and the corresponding body model made using the OpenPose and SMPLify-X containers

This is a more detailed post on how to leverage the containers for SMPLify-X and OpenPose that I provide in my public repo.

SMPLify-X is a computer vision algorithm that creates an expressive body model of individuals using a single image. However, downloading and running the code is slightly difficult, and requires building a few different packages as dependencies. OpenPose is a 2D human pose model that extracts the joints and other keypoints of an individual from a single image. SMPLify-X requires that you have OpenPose setup and functional which in itself can be tough. Further, even if you take all the correct steps to get SMPLify-X working, doing it with a conda environment the torch-mesh-isect package can throw a segmentation fault without any information as to why. The repo author suggests that SMPLify-X should be used with either the system's installation of python3, or to use venv.

I personally do not like working with the system installation of python on my native OS; venv could be a potential solution but SMPLify-X and OpenPose require building quite a few packages so I'd rather containerize them to ensure it's portable and simple to get running on a new system. I figured I'd write a more detailed how-to guide here than what is provided by my repo's README just to help those who are new to Docker for machine learning.

Table of Contents

1 - Prerequisites

There are four main requirements to keep in mind when using these containers.

  1. A Linux based OS
  2. A CUDA enabled GPU that supports CUDA 10
  3. Sudo access
  4. Docker set up such that you can passthrough GPUs to docker containers

The first three are based on your software and hardware configuration, but the last one will require to install the latest version of Docker (19.03.12 as of this writing) and the NVIDIA container toolkit. Note, the host system does not need to have CUDA itself installed, but it must have the NVIDIA drivers. I'll highlight the steps to install Docker and the NVIDIA Container Toolkit next.

1.1 - Set Up Docker for GPU Passthrough

If you are using an older version of Docker (< 19.03.12) and already enabled GPU support using nvidia-docker2 then you simply need to update your Docker installation and everything should work. However nvidia-docker2 will soon be deprecated so I would suggest uninstalling it, and installing the nvidia-container-toolkit. If you have neither Docker, or the nvidia packages to enable GPU passthrough follow the instructions below. If you do, you may skip to section 2.

1) Install Docker 19.03.12

Note these instructions are for Ubuntu 16.04 or later, and Debian Jessie or later. They have been retrieved from Docker's own website. Please refer to their instructions if you're using an ARM based system, or installing on a linux distribution that isn't Ubuntu or Debian.

First we must set up our system to enable support for adding new repositories over HTTPS

# Update package manager repositories
$ sudo apt-get update

# Install dependencies for adding a new repository to package manager
$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

After the previous step, we must add the Docker repository relevant to our system

# Ubuntu
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

# Debian
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/debian \
   $(lsb_release -cs) \
   stable"

Make sure you have the right key.

$ sudo apt-key fingerprint 0EBFCD88

The output of the previous command will look similar to the code block below and must match this fingerprint: 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88

pub   4096R/0EBFCD88 2017-02-22
      Key fingerprint = 9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
uid                  Docker Release (CE deb) <docker@docker.com>
sub   4096R/F273FCD8 2017-02-22

Finally we can install Docker

$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io

2) Install the NVIDIA Container Toolkit

The NVIDIA Container Toolkit provides utilities that make it easier to use NVIDIA GPUs within a Docker container. These instructions are again relevant to Ubuntu and Debian systems. Check the Nvidia Container Toolkit repo for instructions to install on other distributions.

First we must add the repository to our package manager.

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee \
    /etc/apt/sources.list.d/nvidia-docker.list

Then we update our package manager's sources, and install the NVIDIA Container Toolkit

$ sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
$ sudo systemctl restart docker

At this point if everything installed successfully, then you should have both Docker and NVIDIA Container Toolkit on your system.

2 - Running SMPLify-X on Your Own Images

Once you have both Docker and the NVIDIA Container Toolkit setup, using the containerized versions of OpenPose and the SMPLify-X is pretty straightforward. Clone my repo and place the images for which you want to generate 3D body models into the data/images directory. Then simply run the run_both.sh script and both the OpenPose and SMPLify-X containers will be built and used. First the images will be run through OpenPose to get 2D keypoints (they can be found under data/keypoints after OpenPose is run), followed by both the keypoints and images being used by SMPLify-X to generate the body models. The outputs of SMPLify-X including images with the body models rendered on top will be in the outputs directory.

To use the SMPLify-X container, you will need to download the necessary pretrained weights from the project website. You will need to register for an account, and place them in the smplify-x/models directory. The models are available here. You will need to download the SMPL-X model, the VPoser model, and the Homogenus model. You will also need the smplx_parts_segm.pkl model from here. The structure of the directory should look as follows:

smplify-x/
    -> models/
      -> homogenus_v1_0/*
      -> smplx/*        
      -> vposer_v1_0/*        
      -> smplx_parts_segm.pkl

When running the containers, you will be asked for your password to perform root level tasks. This includes changing the ownership of files produced within containers, and to bind your host's X-Session to the Docker Container. This is required because SMPLify-X requires access to an OpenGL environment to visualize the rendering, and this will require a screen to display.

Please keep in mind that the first time you run these containers, the images will need to be built. This means that within the container the dependencies will be built, and based on your system this may take up to 20-30 minutes if not more. Also, if you only wish to run one of the OpenPose or SMPLify-X containers, then the relevant script can be used independently. You can use the containers as follows:

$ git clone https://github.com/kidkych/smplifyx-and-openpose-containers

$ cd smplifyx-and-openpose-containers

# Make sure the necessary files are downloaded and placed in smplify-x/models
# and place images of interest into data/images now

$ ./run_both.sh

3 - Using the SMPLify-X Container for Development

If you wish to use SMPLify-X as a part of your own python code, this is possible using bind-mounts in Docker. Simply add the directory containing your source code as a bind mount and execute your code's entrypoint when starting the container as such:

docker run --rm -it --gpus all \
  -v "path_to_source_code:/home/src \
  smplifyx_container:latest \
  python3 /home/src/main.py

you will also need to modify the sys.path variable in the file you first import the smplifyx packages. This would be done as follows:

# path_to_source_code/main.py
sys.path.append("/home/repos/smplify-x/smplifyx")
from fit_single_frame import fit_single_frame

# do something

If you wish to use a debugger while developing your code base, I would suggest using the extensions for VS Code, or PyCharm both of which greatly simplify the process. I personally use PyCharm when developing within a container but either will work fine.