A Brief Introduction To Virtual Environments in Python

Python virtual environments are a powerful tool for managing project dependencies and ensuring a clean and isolated development environment. In this post, we will explore the key concepts of virtual environments and how to use them effectively in your Python projects.

What are Virtual Environments?

A virtual environment is a self-contained directory that contains a Python installation and a set of libraries. It allows you to create an isolated environment for each of your projects, ensuring that dependencies for one project do not interfere with dependencies for another project or with the system Python installation.

Why should you use a virtual environment?

It is highly recommended for one to use virtual environments in their projects because:

• Dependency Isolation: They allow you to isolate dependencies for each project. This means that you can install different versions of libraries for different projects without worrying about conflicts.

• Dependency Management: They make it easy to manage dependencies. You can install, upgrade, and remove packages using pip without affecting other projects or the system Python installation.

• Reproducibility: They ensure that your project's dependencies are reproducible. By specifying your project's dependencies in a requirements.txt file, you can recreate the environment on another machine or share it with collaborators.

• Clean Environment: They help keep your system Python installation clean. Installing packages globally can lead to conflicts and unexpected behavior, whereas virtual environments keep your system Python installation untouched.

So how do you create and use a virtual environment?

There are a couple of ways that you can create a virtual environment; in this post we will talk about three ways I constantly use:

venv python model

Venv is a built-in python model that was introduced in python 3.3, which allows you separate different installations for different projects by creating a light weight environments.

To create a virtual environment using the venv model, navigate to your project directory and run the following command on you terminal:

• Windows

  python -m venv <name_of_environment>

• macOS / Unix

  python3 -m venv <name_of_environment>

Let us break down the commands:

• python(3) : This is the Python interpreter executable. It's the command used to run Python scripts and modules.

• -m : This option stands for "module." It tells Python to run the specified module (venv in this case) as a script.

• venv : This is the module name. The venv module is used for creating virtual environments in Python. It provides tools for managing dependencies and isolating your project's environment.

• <name_of_environment> : This is the name you give to your virtual environment. It's a directory that will be created in your current working directory to contain all the files related to the virtual environment.

In order to use the virtual environment, you will need to activate it. To do this run the following commands

• Windows

  <name_of_environment>\Scripts\activate

• macOS / Unix

  source <name_of_environment>/bin/activate

You will see that the prompt on your terminal has changed with the name of your environment in parenthesis, after you have activated it the virtual environment.

You can also confirm the activation of the virtual environment by checking the location of the python interpreter. This is done by running which python on macOS / Unix and where python on windows. These commands will give you the file path of the interpreter which will include the virtual environments directory. The file path should end with <name_of_environment\Scripts\python for windows and <name_of_environment/bin/python for macOS / Unix

Use deactivate to deactivate your virtual environment when you are done or want to switch projects.

Virtualenv

virtualenv is a third-party python package that was created back in 2007 to address the need for isolated Python environments, allowing developers to manage dependencies separately for different projects. This was before the creation of the venv module. Since it is a third-party package, you have to install it using pip in order to use it.

pip install virtualenv

After installing virtualenv you can now create a python virtual environment using the following command:

virtualenv <name_of_environment>

Activating the virtual environment is similar to activating a virtual environment created by the venv model:

• Windows

  <name_of_environment>\Scripts\activate

• macOS / Unix

  source <name_of_environment>/bin/activate

To deactivate the virtual environment run deactivate on your terminal.

virtualenv offers more features and flexibility, including compatibility with Python 2 and various additional options. It can be faster when creating environments because it uses symlinks rather than copies for some files.

Pipenv

pipenv is quite different from venv and virtualenv. It is a python packaging tool that combines the functionalities of pip, virtualenv, and pip-tools to provide a comprehensive tool for managing project dependencies and virtual environments.

Just like virtualenv, pipenv is a third-party package and you need to first install it before using it.

pip install pipenv

Now that you have pipenv installed, navigate to the directory of your project and create a Pipfile by installing a package or specifying a Python version. This will also create a virtual environment if one does not exist.

pipenv creates 2 files when you create a virtual environment which you should know a little bit about.

1. pipfile

   This file manages all your projects dependencies.

    [[source]]
    url = "https://pypi.org/simple"
    verify_ssl = true
    name = "pypi"
   
    [requires]
    python_version = "3.8"
   
    [packages]
    requests = "*"
    flask = "==2.0.1"
   
    [dev-packages]
    pytest = "*"
   

   It uses the TOML (Tom's Obvious, Minimal Language) format, which is more human-readable and allows for complex specifications. Here’s a breakdown of its sections:

   1. Source Section:

      • Specifies the source (repository) from which to fetch the packages.

      • The default is usually PyPI (https://pypi.org/simple).

   2. Packages Section:

      • Lists the main dependencies for the project.

      • You can specify exact versions, version ranges, or simply the latest version.

   3. Dev-Packages Section:

      • Lists development dependencies, which are needed only for development and testing, not for production.

   4. Requires Section:

      • Specifies the required Python version for the project.

2. pipefile.lock

   The pipefile.lock compliments the pipefile by locking dependencies to specific versions. Here are some benefits of the pipefile.lock file:

   • Deterministic Builds: Ensures that the same versions of dependencies are installed every time the project is set up. This is crucial for reproducibility, ensuring that the project behaves the same way across different environments.

   • Dependency Resolution: Records the exact versions of all dependencies, including transitive dependencies (dependencies of dependencies), that were resolved when the Pipfile was last used to install or update packages.

   • Security: By locking dependencies to specific versions, it helps prevent issues caused by unexpected updates or changes in dependencies, reducing the risk of introducing bugs or vulnerabilities.

To activate the virtual environment run pipenv shell and just like venv and virtualenv your prompt will change with the name of your environment in parenthesis.

So What Are Some Between venv, virtualenv, and pipenv?

Here is a comparison of the three and when to use them:

venv

• Built-In: Included with Python 3.3 and later, no additional installation required.

• Lightweight: Simple and minimalistic, creating virtual environments with the necessary isolation.

• Standardized: Since it is part of the standard library, it is widely used and supported across various platforms and tools.

Use When:

• You need a straightforward and quick way to create a virtual environment.

• You are working on a Python 3 project and prefer to use built-in tools.

• You want minimal dependencies and a lightweight setup.

virtualenv

• Backward Compatibility: Supports Python 2 and earlier versions of Python 3, making it a good choice for legacy projects.

• Extended Features: Offers more features compared to venv, such as integration with virtualenvwrapper for enhanced environment management.

• Flexibility: Can be used alongside venv for additional functionality

Use When:

• You are working with Python 2 or need to maintain compatibility with older Python versions.

• You need advanced features and extended functionality not available in venv.

• You prefer or require tools like virtualenvwrapper for managing multiple virtual environments.

pipenv

• Dependency Management: Combines pip and virtualenv with dependency management using Pipfile and Pipfile.lock.

• Reproducibility: Ensures exact versions of dependencies are used, providing a more secure and reproducible environment.

• User-Friendly: Simplifies the creation and management of virtual environments and dependencies in a single tool.

Use When:

• You need to manage both your virtual environment and dependencies in a unified way.

• You want a clear separation between development and production dependencies.

• You require a more secure and reproducible environment with automatic dependency resolution.

Conclusion

By incorporating virtual environments into your workflow, you can develop Python projects more effectively and avoid many common pitfalls associated with dependency management.

Whether you choose venv, virtualenv, or pipenv, each tool offers unique benefits to help you maintain clean, isolated environments tailored to your specific needs.

Embracing virtual environments ensures that your projects remain stable, reproducible, and free from dependency conflicts, ultimately leading to more efficient and reliable development practices.