Python Virtual Environments with Virtualenv

When working on your Python projects it's a really good idea to use virtual environments. Basically, a virtual environment is like having a folder for your project where you can specify what version of Python and what dependencies (other libraries) you'd like to use for that particular project.

If you don't use virtual environments then whenever you install dependencies for your projects they will be installed to your system's standard directory for Python. Having all the dependencies for all your projects installed in the same directory isn't a great way to manage things.

For example, say you have a Python Project A that relies on version 1.0.0 of a specific library. Then you have a second Project B which relies on version 2.0.0 of that library. So you upgrade the library to version 2.0.0 to get Project B working and by doing so you risk breaking Project A.

Another example, say you're developing a Python project that uses only a few dependencies. You want to share your project with others so you create a requirements.txt file that they can use to easily install all of the project's dependencies. So you use pip freeze > requirements.txt to create this file, but pip now includes all of the libraries that are in that standard directory for all of your projects. So using virtual environments helps to keep your projects separated and organised.

One of the most popular tools for managing virtual environments for Python is Virtualenv, which does a great job of helping you to set up isolated Python environments for your projects.

Installing Virtualenv

In order to install Virtualenv you'll need to make sure you have pip installed first. pip is Python's recommended tool for installing Python packages and you'll almost certainly be using it to install any packages for your projects. You can install pip by entering the following into your terminal.

$ sudo easy_install pip

Once you have pip installed then you can tell it to install Virtualenv with the following command:

$ pip install virtualenv

If you encounter any errors when installing Virtualenv you can refer to the troubleshooting section at the bottom of this post.

Creating a virtualenv

Now that you have Virtualenv installed you can use it to create virtual environments for your projects.

Python 3

To create a virtual environment that uses Python 3, open up your terminal and navigate to the directory where you would like to create your new environment, then enter the following into your command line:

 $ virtualenv -p python3 myproject

Replace myproject in the command above with whatever name you would like to set for your new environment. This should now create a folder directory that looks like this:

myproject
├── bin
├── include
├── lib
└── pip-selfcheck.json

The above folder and all of its content is now your shiny new virtual environment. The /bin directory is where your project code should live. This can be a single .py file or a folder containing a more comprehensive project.

Any dependency libraries that you install with pip will be placed in /lib/pythonX.X/site-packages/ this is useful to know in case you ever need to inspect the source code for any of them.

Note: Before installing any dependencies with pip install make sure you have activated the virtual environment, otherwise pip will install the library into your machine's standard Python directory, thus negating the entire point of having a virtual environment in the first place.

Python 2

You should try and avoid using Python 2 from now on unless it's absolutely necessary, as Python 2 will no longer be actively supported from the 1st of January 2020. However, if you do need to create a virtual environment that uses Python 2 you can use the following command:

$ virtualenv myproject

Activate a virtualenv

Inside the /bin directory of your virtual environment is a shell script called activate, to activate the virtual environment you just need to run this. The first command below navigates into the bin folder, the second runs the activate script.

$ cd myproject/bin
$ source activate

You should now see your terminal prompt change to something like this:

(myproject) $

Your virtual environment is now active and you can now proceed to install any project dependencies using the pip install command.

Deactivate a virtualenv

To deactivate the virtual environment, you can simply type deactivate into your terminal, from any directory, like this:

(myproject) $ deactivate
$ 

And that's it! The above should help you to keep your Python projects in a more organised state. If you have any questions, feel free to leave them in the comments.

Troubleshooting

Below are some suggestions for possible errors that you may encounter.

$ pip install virtualenv
ERROR: Could not not install packages due to an EnvironmentError: 
[Errno 13]Permission denied: '/Library/Python/2.7/site-packages/virtualenv.pyc'
Consider using the --user option or check the permissions.

The above error means that pip tried to add the virtualenv.pyc file to your machine's standard Python packages directory but did not have permission to add new files to that folder. To remedy this you could change the permissions of the folder, use the --user flag to specify a user that does have permissions, or lastly perform the action as a root user.

Solution 1: Change the folder permissions
In the above error, the directory that did not allow the file to be written to was /Library/Python/2.7/site-packages in this case we can change the permissions for the file by using the following command in your terminal:

$ cd
$ chmod -R 777 /Library/Python/2.7/site-packages

This will make that folder and all folders within it writeable, because of the -R (recursive) flag. You should then be able to run the pip install virtualenv command again. This is should be your first course of action as the solutions that follow start opening up permissions for scripts to have greater access to your system.

Solution 2: Use the --user flag
Next you could try installing the script as your user with the following command:

$ pip install --user virtualenv

Solution 3: Use the sudo command
This should be used as a last resort. Essentially, this will run the command as a root user so if there is any chance of your pip installation being compromised then it could run any sort of malicious action when you execute this command. It's unlikely but not impossible.

$ sudo pip install virtualenv

This may ask you to enter your password after which it should allow the installation to proceed successfully.