Glossary

package and environment manager: a package and environment manager is a tool that allows you to install packages, with different versions and correctly resolving and installing their dependencies (other packages that are needed by the package you want to install). The "environment" part refers to the fact that these tools install the packages in isolated environments that you can activate/deactivate, allowing you to have different environments with different software stack for different purposes in the same computer. Examples of a package and environment manager are conda and mamba.

conda channel: is essentially a repository of packages. In this documentation we make special mention to the defaults, main and anaconda channels (curated by Anaconda, and with some porprietary packages) and conda-forge channel (curated by the community, open source without the proprietary packages from Anaconda)

What is all the Anaconda fuzz about?

Anaconda, Miniconda and Miniforge are both tools for managing Python environments and packages, especially useful for scientific computing. Anaconda is a full-featured Python distribution while Miniconda and Miniforge are lighter versions. Miniforge is open-source and only sets Conda Forge channel by default!

Anaconda itself is proprietary and also includes proprietary packages that require a paid license for commercial and institutional use. These propritary packages are distributed via the defaults, main and anaconda channels together with other open-source packages. This effectively means that if scientists at AWI use Anaconda or any of the proprietary packages in these channels, AWI should be paying for Anaconda.

The good news is that all packages in conda-forge channel (the open-source community equivalent to the defaults channel) do not include proprietary packages. Other good news are that conda and mamba are open-source and free to use. So all we need to do is to make sure we don't use the channels defaults/main/anaconda or Anaconda itself to save AWI from paying an expensive premium. Luckily, miniforge installs conda, mamba and sets the conda-forge as the default channel in a very easy way.

So, in this guide we will help you to switch from Anaconda to miniforge, and from defaults/main/anaconda channels to conda-forge channel!

Step-by-step guide

In this guide we will help you to uninstall anaconda, export your conda environments, install miniforge and build again your environments without the use of the defaults channel. If you get stuck at any point please contact helpdesk@awi.de

WARNING!!!!!! Follow this guide at your own risk. We are not responsible for any data loss or any other issues that might arise from following this guide. Make sure you have backups of your data and environments before proceeding. If you don't understand some of the commands described here, please contact us so that we can help you.

1. Check whether you have anaconda installed in your system

Open a terminal and type:

conda info --base

This would display the path to your conda base directory (in my case /opt/homebrew/Caskroom/miniforge/base). If that path contains anaconda or anaconda3 or other variants, instead of miniforge, miniconda, ... you have Anaconda installed.

  • If you have Anaconda installed move on to step 3.
  • If you don't have Anaconda installed, but miniconda or others, jump to step 2.

2. Check for channels with proprietary packages

Open a terminal and type:

conda config --show channels

If you see defaults, main, or anaconda you are using channels that include proprietary packages.

  • If you didn't have any of the channels above and you did not have Anaconda installed you should be clean, you are done with this guide, congratulations!
  • If you had any of the channels above, move on to step 3.

3. Export your conda environments to yaml files, backup your environments and remove the environments

If you are here, you have potentially built some conda environments using channels that include proprietary packages, and consequently you might be using packages that are proprietary.

Later we will remove this environments, and rebuild them without the proprietary packages. But first, we will export the environments to yaml files, so you can rebuild them later.

  1. List your environments:

    conda env list

    and decide which ones would you like to export and which ones you'd like to delete forever.

  2. Export the environments you want to keep to yaml files:

    conda env export --name myenv --file myenv.yml

    where myenv is the name of the environment you want to export.

  3. Backup the environments by copying the path of the environment (given by conda env list) to a safe place, for example:

    cp -r /path/to/myenv ~/myenv_backup

  4. Backup the yaml files to a safe place, for example:

    mkdir ~/env_yaml_backups
cp myenv.yml ~/env_yaml_backups

    This is very important, because we are going to need to modify the yaml files later on and having a backup of them is crucial if something goes wrong.

  5. Remove the environments:

    conda env remove --name myenv

4. Remove defaults channel if you were using conda but you never installed Anaconda

If you are using miniforge, miniconda and you still have any of the channels defaults, main, or anaconda you should remove them now:

conda config --remove channels defaults
conda config --remove channels main
conda config --remove channels anaconda

5. Uninstall Anaconda

  • If you don't have Anaconda installed you can skip this step.

If you have Anaconda installed, you should uninstall it now. You can do this by running the following command:

conda install anaconda-clean
anaconda-clean --yes

Once anaconda-clean is done, you can remove the Anaconda directory installation directory by running:

rm -rf /path/to/anaconda

where /path/to/anaconda is the path to your Anaconda installation directory as obtained in step 1 (conda info --base) until the /anaconda part.

6. Install miniforge

We recommend installing miniforge to substitute Anaconda, as it is open-source and uses the conda-forge channel by default. There are other possibilities (as miniconda) but they might include channels with proprietary packages that you'll need to remove as in step 4.

To install miniforge follow the instructions in the miniforge repository.

7. Build your environments again using the yamls

Make sure your yaml files are backed up before you continue!

We need to modify the yaml files to remove the channels that include proprietary packages. Open the yaml files with a text editor and remove the lines that include the channels defaults, main, or anaconda.

You might find that many of the packages nested under dependencies: have a second = followed by a hash:

name: esm_tools
channels:
  - conda-forge
dependencies:
  - attrs=22.1.0=py39hecd8cb5_0
  - ...

In this case the hash is py39hecd8cb5_0. You should remove this hash from all the packages. This hashes refer to the old default channels and do not correspond to the ones in conda-forge channel, so if you try to rebuild the environment with these hashes you might get errors. After removing the hashes the yaml file should look like this:

name: esm_tools
channels:
  - conda-forge
dependencies:
    - attrs=22.1.0
    - ...

At the end of the yaml file you'll also find a prefix key. You should remove this key as well if the path contains anaconda, to install the environment in the default location, or specify a new location of your choice.

Now your yaml files are ready to be used to build the environments. To build the environments you can run:

conda env create --file myenv.yml

or

mamba env create --file myenv.yml

(mamba is a faster alternative to conda, but you need to install it first with conda install mamba).

If you get any errors at this point (very likely), is probably because 1) some of the packages in the environment were proprietary, 2) because the dependency trees in conda-forge might be different from those in the defaults channel, or 3) because some dependencies names are slightly different. In many cases, it will be sufficient with removing the package that is causing the error from the yaml file and try to build the environment again. If this is not enough, we recommend to manually delete all the packages you don't remember having installed manually and try building the environment again.

Congratulations! You have successfully switched from Anaconda to miniforge and from defaults channel to conda-forge channel! You can now enjoy the benefits of open-source software without the need to pay for proprietary packages.


  • No labels