Skip to content

A.I. Project Template Internals

Under the hood, the template uses many open source tools. In some cases, you will need to gain a better understanding of this tools in order to be able to adapt the template to your needs.

Conda

Conda is a virtual environment manager, a software that allows you to create, removing or packaging virtual environments as well as installing software. A virtual environment is just an isolated installation of Python and its libraries. If you need to know more (and you should know more) read this post before moving on.

DVC

DVC is configured to follow the Shared Development Server use case. Yo can find the configuration for your proyect in the .dvc directoy.

DVC is used for:

MLFlow

MLFlow is configured to use a common tracking server hosted in the intranet. The configuration details can be found here.

MLFlow is used for:

  • Package the training stage in a reusable enviroment using MLFlow Projects. The configuration can be found in the MLproject file.
  • Logging parameters, code versions, metrics, and output files for each experiment using MLFlow Tracking. We use the mmcv MLFlow callback for automatic logging and also some manual logging in stages/experiment/run_experiment_mlflow.py.

TensorBoard

TensorBoard is configured to tracking and visualizing the metrics, model graphs, and images generated during the training stage. TensorBoard is used for:

  • Logging metrics and compare them between different experiments. We use the [mmcv MLFlow callback] (https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/logger/tensorboard.py) for automatic logging.

Open MMLab Training frameworks

We use different open-mmlab repositories for each of the supported tasks.

You can refer to the documentation of each task's repository for more details about features and about the configuration files under config.

In addition, all of the task-specific repository use MMCV as the core library so it can be usefull to also use it's documentation as reference.