After spending a year troubleshooting and researching Python packaging tools and installers, I've put together this guide to share some best practices for managing Python environments and packages. By leveraging the strengths of conda, poetry, uv, and pip, you can create efficient workflows for both development and deployment.

Local Development

  • Use Conda to Create Environments
    • Conda is great for isolating environments on the same machine.
    • Create an environment: conda create -n local_env python=3.14 (creates an empty environment with Python 3.14).
    • Activate an environment: conda activate local_env.
      • Example: (local_env) user - $ - you are now in the local_env environment.
  • Use Poetry to Manage Packages
    • Poetry works with pyproject.toml to add and install packages.
    • Example: poetry add numpy to add a package.
    • This process creates a poetry.lock file that manages dependency resolution.
    • Keep using poetry add <package> to add packages.
    • Note: Poetry creates its own virtual environment, but it isn't used directly unless poetry run is invoked. This doesn't affect us since we're interested in package resolution.
    • Once ready, install the packages using pip install . or
      pip install -e . where poetry.lock and pyproject.toml are located.
    • This installs all packages into the current Conda environment (local_env).
    • You can repeat this process for other environments, making it easy to switch between projects (e.g., dev_env and prod_env).

CI / DevOps / Docker Builds

  • No Need for Conda
    • At this stage, we assume poetry.lock has all required packages resolved.
    • Use Poetry to Set Up Environments
      • poetry.lock and pyproject.toml are key for dependency management.
      • Use poetry install to install packages and poetry run to execute anything.
      • Poetry creates a local virtual environment (venv) that's only invoked with poetry run.
      • Alternatively, poetry shell can be used to activate the environment temporarily, though this is not recommended for production use.
    • Poetry Install Speed
      • Despite improvements, poetry install can still be slow (although it is 10x faster than pip install and 20x faster than conda install).
  • Use uv for Fast Installations
    • uv is a Rust-based package installer, and it's much faster.
    • Requires requirements.txt, which can be generated by Poetry:
      • poetry export --without-hashes -f requirements.txt
      • This takes less than 1 second to generate.
    • Run the following command to install dependencies:
      • uv pip install --no-deps -r <(poetry export --without-hashes -f requirements.txt)
    • uv uses pip under the hood and creates a virtual environment much faster than Poetry.
    • After using uv, run poetry install to update the Poetry source tree.
    • This allows you to use Poetry for a seamless developer experience while leveraging uv for fast installations.
  • Fastest Installation Method
    • For production environments, you can combine commands for speed:
      • uv pip install --no-deps -r <(poetry export --without-hashes --with dev -f requirements.txt)
      • Then update with: poetry install --only-root

Summary (TL;DR)

If Developing Locally/Natively

  • Use Conda to create environments.
  • Use Poetry to resolve and install packages.
  • Use Pip for installing packages or editable packages.

Everywhere Else (Production)

  • Use poetry.lock and pyproject.toml to craft your environment (10x faster than Conda).
  • Use uv to install dependencies (10x faster than poetry install, 30x faster than conda install).
  • Use Poetry to run or execute scripts (no Conda environment needed).
  • Optionally, uv run also exists.

Conclusion

The approach taken here shows an ambition to achieve a finely-tuned workflow that combines speed, reliability, and consistency. However, the layered complexity of combining conda, poetry, pip, and uv in this way could present a learning curve that not all teams are prepared to tackle. Many teams may find this setup over-engineered for their needs, especially if they do not require the minute optimizations that uv provides.

You’ve successfully subscribed to Sudhanva
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Success! Your email is updated.
Your link has expired
Success! Check your email for magic link to sign-in.