How to Use Multiple Versions of Python Using Pyenv and Virtualenv

I was recently involved in a python 2 to 3 upgrade project at work and had a decent experience using pyenv and virtualenv together during development to deal with multiple pythons in a relatively sane way.

If you’re unfamiliar with these projects, here’s what they do in a nutshell:

  • pyenv helps you install multiple versions of python and switch between them in your regular shell environment
  • virtualenv helps you create isolated library installations (via pip) and manage them in a new (virtual) shell environment

Sounds good … why do I need to use them both again?

If you need virtualenv to create environments based on some arbitrary python version, you need to use the --python or -p option and provide it with the name (or path to the interpreter if the interpreter name is not already in your PATH).

virtualenv does not try to install that interpreter for you. It expects the python interpreter to already exist so it’s up to you how to install it, and that’s where pyenv comes in to help simplify that process.

Setup

Make sure you have both pyenv and virtualenv installed. They have great documentation - here are the links to their installation steps:

I’m currently using MacOSX to write this article, so I’ll walk through the entire process (with code) of setting up multiple projects using pyenv and virtualenv and explain stuff along the way.

First, here are the versions of pyenv and virtualenv I’m working with:

alin in ~/src
λ which pyenv && pyenv --version && which virtualenv && virtualenv --version
/usr/local/bin/pyenv
pyenv 1.2.19
/usr/local/bin/virtualenv
16.7.

I used Homebrew to install pyenv and a really old pip to install virtualenv :D

Create two projects: old-python-stuff and new-python-stuff.

alin in ~/src
λ mkdir old-python-stuff new-python-stuff

Since my virtualenv was installed using python 2.7.16, I’ll create an environment using virtualenv in old-python-stuff using its normal defaults and then create a new environment in new-python-stuff by using python 3.8.3.

Creating a virtual environment in old-python-stuff, activating it, and printing out my python and pip versions:

alin in ~/src/old-python-stuff
λ virtualenv env --prompt="(old-python-env)"
New python executable in /Users/alin/src/old-python-stuff/env/bin/python
Installing setuptools, pip, wheel...
done.
alin in ~/src/old-python-stuff
λ . env/bin/activate
(old-python-env)alin in ~/src/old-python-stuff
λ python --version && pip --version
Python 2.7.16
pip 20.1.1 from /Users/alin/src/old-python-stuff/env/lib/python2.7/site-packages/pip (python 2.7)

As you can see, the python inside my new virtualenv is the same as my system python (the one I used to install virtualenv).

Ok, now deactivate the virtualenv with deactivate and lets go into the future by installing python 3.8.3 using pyenv:

pyenv install 3.8.3

Once it’s installed, to confirm that it exists you should be able to run this:

alin in ~/src
λ pyenv versions
* system (set by /Users/alin/.python-version)
  3.8.3
alin in ~/src
λ pyenv local 3.8.3
alin in ~/src
λ python --version
Python 3.8.3

pyenv versions will list all the versions of python you installed using pyenv as well as your system python. python local lets you switch python versions in your current shell so that calls to python will invoke a specific version installed by pyenv.

Great, now lets repeat a similar virtualenv setup process with our second project, new-python-stuff.

alin in ~/src
λ cd new-python-stuff/
alin in ~/src/new-python-stuff
λ  virtualenv -p "/Users/alin/.pyenv/versions/3.8.3/bin/python3" env --prompt="(new-python-env)"
Running virtualenv with interpreter /Users/alin/.pyenv/versions/3.8.3/bin/python3
Already using interpreter /Users/alin/.pyenv/versions/3.8.3/bin/python3
Using base prefix '/Users/alin/.pyenv/versions/3.8.3'
New python executable in /Users/alin/src/new-python-stuff/env/bin/python3
Also creating executable in /Users/alin/src/new-python-stuff/env/bin/python
Installing setuptools, pip, wheel...
done.
alin in ~/src/new-python-stuff
λ . env/bin/activate
(new-python-env)alin in ~/src/new-python-stuff
λ python --version && pip --version
Python 3.8.3
pip 20.1.1 from /Users/alin/src/new-python-stuff/env/lib/python3.8/site-packages/pip (python 3.8)

virtualenv -p "/Users/alin/.pyenv/versions/3.8.3/bin/python3" was missing from our previous setup with old-python-stuff. This time, we’re telling virtualenv which python to use. Since pyenv installs all of its python interpreters to <HOME>/.pyenv/versions, you can supply it with a path to the interpreter you want to use.

And that’s it! Now you can manage your projects in both directories with isolation installation and different python versions!

Hope that helps.