Skip to content

Setup

In order to use the template, please follow instructions carefully:

1. Create a local copy of the template

You need to install cruft. In MM servers, it's already installed and there is a specific environment, so you need to activate it:

conda activate cruft

You have two options to run:

  • "https"
cruft create "https://github.com/Gradiant/ai-project-template.git"

In order to be able to clone the template using this method, you might need to perform some extra steps if you have enabled 2FA in your github account. One easy solution is to create a personal access token.

  • "ssh"
cruft create git@github.com:Gradiant/ai-project-template.git

In order to be able to clone the template using this method, you have to upload your public ssh key-pair to GitHub and link it with your GitHub account.

After either method, follow the prompts. A new folder will be created using your answers to the prompts (i.e. ai-project-{{my_new_project}}).

If you are in a MM server, you can now deactivate the cruft environment, since from this point on it will not be necessary:

conda deactivate

2. Setup the enviroment

Clone or create the conda environment

There are 3 main options for creating the new conda environment:

  • Use the ai-project-base environment as a donor for the new environment.
conda create --name ai-project-PROJECT_NAME --clone ai-project-base

This is the best option for creating a new environment, it only takes a few minutes and uses an already tested environment. The ai-project-base environment is always updated by the administrators in Gea1/2 in order to keep it synced with the latest master changes.

  • Create a conda environment from conda.yml using conda and activate it:
conda env create -f environment/conda.yaml
. activate [environment-name]

Using this method, the entire environment is recreated again using conda. This operation can take a few hours but conda manages to create an exact replica of the already created ai-project-base.

  • Create a conda environment from conda.yaml using mamba and activate it:
mamba env create -f environment/conda.yaml
. activate [environment-name]

Similar to the previous method, but mamba takes only 15~30 minutes to create the conda environment. Since mamba uses a different strategy for resolving the environment deps, the created environment could differ a little from the original ai-project-base.

Mamba commands

Remember that in order to use mamba, you have to be inside base environment.

Use cases of each method:

  1. Use conda clone procedure if you want an original copy of the ai-project environment, this is the most usual case.
  2. Use conda create if you add custom deps to the original conda.yaml file, it will build the original environment plus your additional custom deps.
  3. Use mamba if you don't care about random errors or incompatibilities between the installed packages for your custom environment. For example: toy environment, quick test, etc.

Using mamba

Use mamba at your own responsibility, methods 1 and 2 are tested and safe but we can't ensure that method 3 will produce a stable environment.

Post setup

Activate conda environment. Remember to have it activated from here on.

. activate [environment-name]

Initialise the git repository

git init

Install pre-commit hooks:

pre-commit install

Install the template as a python package inside the environment:

python setup.py develop

Test the enviroment:

dvc checkout

dvc repro

3. Create a repository on Github

Follow the official GitHub instructions

Remember that your repository name and permissions must follow the MM conventions (i.e. MM-PR-XYZ-ai-project-{{my_new_project}})

4. Publish the local template

First run (Don't skip commands):

# Assuming you are at `ai-project-{{my_new_project}}`
git add --all
# Skip Github Labels in First commit
git rm --cached .github/labels.yml
git commit -m "First commit"
git remote add origin https://github.com/Gradiant/ai-project-{{my_new_project}}.git
git branch -M main
git push -u origin main

And then:

# Add Github Labels afterwards to trigger Issue Label creatation
git add .github/labels.yml
git commit -m "Add github labels"
git push