Managing dependencies in Python projects has always been a challenging task. From virtual environments to requirements files, the ecosystem offers multiple solutions, each with its own advantages and limitations. Enter Poetry, a modern dependency management and packaging tool that aims to simplify the entire workflow.

In this blog post, we’ll explore how Poetry addresses common pain points in Python project management and walk through its key features that can streamline your development process.

What is Poetry?

Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and manages them for you. Poetry replaces the need for separate tools like pip, virtualenv, and setuptools by providing a unified workflow for dependency management, environment isolation, and package building.

Some key benefits of Poetry include:

• Simplified dependency resolution and management
• Automatic virtual environment creation
• Consistent environment across development, testing, and production
• Streamlined packaging and publishing process
• Dependency group management for development, testing, and other specialized requirements

Installing Poetry

Getting started with Poetry is straightforward. You can install it using pip:

pip install poetry

For a more controlled installation, refer to the official installation guide, which offers various methods tailored to different operating systems.

Configuration Tip

For better IDE integration, it’s recommended to configure Poetry to create virtual environments within your project directory:

poetry config virtualenvs.in-project true

Creating a New Project

One of Poetry’s strengths is its ability to scaffold new projects with a proper structure. To create a new project:

poetry new my-project

This will generate a directory structure that follows Python best practices:

my-project/
├── pyproject.toml       # Project configuration
├── README.md           # Documentation
├── my_project/         # Source code
│   └── __init__.py
└── tests/              # Test files
    └── __init__.py

Customizing Project Creation

Poetry provides several options to customize your project structure:

• Use -src to add an src folder above the package directory
• Specify a different package name with -name

For example:

poetry new my-directory --name my-package

Managing Dependencies

Adding and removing dependencies in Poetry is streamlined and intuitive.

Adding Dependencies

To add a package to your project:

poetry add requests

This command installs the package and updates your pyproject.toml file automatically. It also generates a poetry.lock file, which ensures consistent installations across environments.

Removing Dependencies

Removing a package is equally simple:

poetry remove requests

Manual Dependency Management

You can also edit the pyproject.toml file directly to specify dependencies:

[tool.poetry.dependencies]
python = "^3.11"
requests = "^2.28.1"

After manually updating dependencies, run:

poetry install

Dependency Grouping

One of Poetry’s most powerful features is its ability to organize dependencies into logical groups.

poetry add mkdocs --group docs
poetry add black --group dev
poetry add pytest --group test

When installing dependencies, you can select which groups to include:

• -only: Installs only the specified groups
• -with: Installs the specified groups plus the main group
• -without: Installs everything except the specified groups

poetry install --with dev,test --without docs

Synchronizing Project State

Poetry offers several commands to keep your project in sync:

• poetry lock: Updates the lock file when you change pyproject.toml without installing packages. Add -no-update to preserve existing dependency versions where possible.
• poetry install --sync: Ensures your environment exactly matches the lock file by adding missing and removing extra packages.
• poetry update: Updates dependencies to their latest allowed versions, regenerates the lock file, and installs the updates. Works with -with, -without, and -only flags for targeted updates.
• poetry check --lock: Verifies consistency between pyproject.toml and poetry.lock without making changes. Perfect for CI/CD pipelines.

These commands help maintain consistency across environments and ensure reproducible builds.

Working with Your Project Environment

Poetry creates isolated environments for your projects, making it easy to run commands within them.

Running Commands

To execute a command in your project’s environment:

poetry run python app.py

Activating the Environment

For a more interactive experience, activate the virtual environment:

poetry shell

This allows you to run commands directly without the poetry run prefix. To exit the environment, simply type exit.

Testing with Poetry

Poetry integrates seamlessly with testing frameworks like pytest.

Setting Up Test Dependencies

Add your test dependencies to a dedicated group:

poetry add pytest --group test
poetry add pytest-cov --group test

Configuring Tests

Configure pytest in your pyproject.toml:

[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q"
testpaths = [
    "tests",
]

Running Tests

Execute tests with Poetry:

poetry run pytest
poetry run pytest --cov=my_project

You can also add shortcuts in your pyproject.toml:

[tool.poetry.scripts]
test = "pytest"
test-cov = "pytest --cov=my_project"

Then run them with:

poetry run test

CI/CD Integration

Poetry works well with CI/CD pipelines. For GitHub Actions, you can set up Poetry with:

- name: Set up Poetry
  uses: Gr1N/setup-poetry@v8

- name: Install dependencies
  run: |
    poetry install --with test

- name: Test with pytest
  run: |
    poetry run test

If you need a requirements.txt file for other systems:

poetry export -f requirements.txt --without-hashes > requirements.txt

Building and Publishing Packages

When your project is ready for distribution, Poetry simplifies the packaging and publishing process:

# Build your package
poetry build

# Configure PyPI credentials
poetry config pypi-token.pypi your-token

# Publish to PyPI
poetry publish

# Or combine building and publishing
poetry publish --build

Converting Existing Projects

If you have an existing project, you can add Poetry to it:

cd existing-project
poetry init

This starts an interactive process to create your pyproject.toml file based on your project’s needs.

Integrating with Other Tools

Poetry’s pyproject.toml file can also host configuration for other tools like Black, Ruff, and other Python utilities, creating a single source of truth for your project’s configuration.

Command Cheatsheet

Here’s a quick reference for common Poetry commands:

Category	Command	Description
Setup	poetry new project-name	Create a new project
	poetry init	Initialize in existing project
Environment	poetry env use python3.9	Specify Python version
	poetry shell	Activate virtual environment
Dependencies	poetry add package-name	Add a package
	poetry add --group dev package-name	Add a dev dependency
	poetry remove package-name	Remove a package
	poetry update	Update all packages
Project	poetry install	Install dependencies
	poetry build	Build your package
	poetry publish	Publish to PyPI
Information	poetry show	List installed packages
	poetry show --tree	Show dependency tree
Execution	poetry run python script.py	Run script in environment

Conclusion

Poetry offers a comprehensive solution to Python dependency management challenges. By integrating virtual environment management, dependency resolution, and packaging into a single tool, Poetry streamlines the development workflow and ensures consistency across environments.

Whether you’re starting a new project or improving an existing one, Poetry’s intuitive commands and powerful features make it an excellent choice for modern Python development.

For more information, check out the official Poetry documentation.