{ "cells": [ { "cell_type": "markdown", "id": "865749bb217bdd8b", "metadata": {}, "source": [ "# MAIA\n", "\n", "This tutorial has the purpose to showcase all the major applications and functionalities available in MAIA. In detail, we will cover all the essential Medical AI lifecycle stages, to provide a comprehensive overview of the platform.\n", "\n", "The tutorial is based on two different datasets:\n", "- [**Decathlon Spleen**](http://medicaldecathlon.com/): a dataset of 3D spleen CT scans from the Medical Segmentation Decathlon challenge. This NIFTI dataset is used to demonstrate the model preprocessing, training and evaluation functionalities in MAIA.\n", "- [**CT Lymph Nodes**](https://www.cancerimagingarchive.net/collection/ct-lymph-nodes/) from TCIA: a dataset of 3D lymph node CT scans from The Cancer Imaging Archive. This DICOM dataset is used to demonstrate the data management functionalities in MAIA, including DICOM upload,visualization, annotation and AI model inference (including Active Learning with MONAI Label).\n", "\n", "The tutorial will cover all the necessary steps to download the Decathlon Spleen dataset, preprocess it, train a [ResEnc nnU-Net](https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/resenc_presets.md) model as a MONAI Bundle, evaluate the model, and finally deploy it for inference on the CT Lymph Nodes dataset. The tutorial will also cover the necessary steps to upload the CT Lymph Nodes dataset, visualize it, annotate it, and perform AI model inference on it.\n", "\n", "In this tutorial, we will use [PyMAIA](https://pypi.org/project/pymaia-learn/), a Python package developed to handle nnUNet-based experiments. PyMAIA provides a set of functionalities to prepare the data, run the training, and evaluate the model. PyMAIA also provides a set of functionalities to convert the MONAI Bundle to a nnUNet Bundle, which can be used to deploy the model for inference. To read more about PyMAIA, please refer to the [documentation](https://pymaia.readthedocs.io/en/latest/).\n" ] }, { "cell_type": "markdown", "id": "2805b24f-5af5-4597-b091-29132ea586f0", "metadata": { "vscode": { "languageId": "markdown" } }, "source": [ "### Prerequisites\n", "\n", "To run this notebook, you need to have the following prerequisites:\n", "\n", "1. **Python Environment**: Ensure you have a Python environment set up. This notebook is designed to work with Python 3.8 or later.\n", "2. **Required Packages**: Install the necessary Python packages. You can install them using the following command:\n", " ```bash\n", " !{sys.executable} -m pip install odict plotly dtale \"monai[nibabel, skimage, scipy, pillow, tensorboard, gdown, ignite, torchvision, itk, tqdm, pandas, mlflow, matplotlib, pydicom]\" \"pydicom==2.4.4\" nnunetv2==2.5.1 dtale pymaia-learn fire\n", " ```\n", "3. **Datasets**: Download the required datasets:\n", " - [Decathlon Spleen](http://medicaldecathlon.com/)\n", " - [CT Lymph Nodes](https://www.cancerimagingarchive.net/collection/ct-lymph-nodes/)\n", "4. **3D Slicer**: Ensure you have 3D Slicer installed for visualization purposes. You can download it from [here](https://www.slicer.org/).\n", "5. **Remote Desktop Access**: Access to the remote desktop environment for interacting with 3D Slicer.\n", "\n", "Make sure to follow the instructions in the notebook to set up the environment and download the datasets." ] }, { "cell_type": "code", "execution_count": null, "id": "b5447525-e0e3-4298-87d8-2efe012d931a", "metadata": { "scrolled": true }, "outputs": [], "source": [ "import pip\n", "import sys\n", "\n", "!{sys.executable} -m pip install odict plotly nbformat==4.2.0 dtale \"monai[nibabel, skimage, scipy, pillow, tensorboard, gdown, ignite, torchvision, itk, tqdm, pandas, mlflow, matplotlib, pydicom]\" \"pydicom==2.4.4\" nnunetv2==2.5.1 dtale pymaia-learn fire" ] }, { "cell_type": "code", "execution_count": null, "id": "f5fbf6aeed0a7dd8", "metadata": { "scrolled": true }, "outputs": [], "source": [ "from monai.apps import DecathlonDataset\n", "import subprocess\n", "import json\n", "import os\n", "import pandas as pd\n", "import pathlib\n", "import mlflow\n", "from pathlib import Path\n", "import numpy as np\n", "import yaml\n", "import dtale\n", "import dtale.app as dtale_app\n", "\n", "from monai.bundle import ConfigParser\n", "\n", "from mlflow.models import ModelSignature\n", "from mlflow.types.schema import Schema, TensorSpec" ] }, { "cell_type": "markdown", "id": "90c47266462d9e08", "metadata": {}, "source": [ "## Download the Decathlon Spleen dataset" ] }, { "cell_type": "code", "execution_count": null, "id": "1bc6902b6e42cbc4", "metadata": {}, "outputs": [], "source": [ "DecathlonDataset(\"MAIA\",\"Task09_Spleen\",\"training\",download=True)" ] }, { "cell_type": "markdown", "id": "d34e7d474b6d8ed0", "metadata": {}, "source": [ "## Visualize Spleen CT and SEG in 3D Slicer" ] }, { "cell_type": "markdown", "id": "7b24e17f-4757-45ab-9d17-0de5a8ca23ad", "metadata": {}, "source": [ "You can navigate to [Remote Desktop](/user/demo@maia.se/proxy/80/desktop/demo@maia.se/) (password: `maia`) to interact with the Remote Desktop environment and 3D Slicer." ] }, { "cell_type": "code", "execution_count": null, "id": "de310610-a2b6-47d1-a4c3-0d25ce066128", "metadata": {}, "outputs": [], "source": [ "slicer_executable = \"/home/maia-user/Documents/Slicer-5.7.0-2024-06-08-linux-amd64/Slicer\"\n", "\n", "ct_volume = \"/home/maia-user/Tutorials/MAIA/Task09_Spleen/imagesTr/spleen_2.nii.gz\"\n", "\n", "seg_volume = \"/home/maia-user/Tutorials/MAIA/Task09_Spleen/labelsTr/spleen_2.nii.gz\"\n", "\n", "subprocess.run([\n", " slicer_executable,\n", " \"--python-code\", f\"slicer.util.loadVolume('{ct_volume}'); seg=slicer.util.loadSegmentation('{seg_volume}'); seg.CreateClosedSurfaceRepresentation()\"\n", "])" ] }, { "cell_type": "markdown", "id": "b28763d4-f46e-4274-9f0d-ad0314cd39b9", "metadata": {}, "source": [ "## Prepare nnUNet ResEnc Training\n", "\n", "Following the PyMAIA workflow steps for nnUnet experiments, we will prepare the data folder and run the plan and preprocessing steps.\n", "\n", "To start with the data preparation, we need to create a configuration file that contains the experiment name, seed, label suffix, modalities, label dictionary, number of folds, and file extension. The configuration file is saved as `Task09_Spleen_config.json`. We also need to create a dataset file that contains the training and test data paths. The dataset file is saved as `Task09_Spleen_Dataset.json`." ] }, { "cell_type": "code", "execution_count": null, "id": "5756a99c-ab41-45a5-afa6-badae32d24cf", "metadata": {}, "outputs": [], "source": [ "modality_conf = {\n", " \"CT\": {\n", " \"suffix\": \".nii.gz\"\n", " }\n", "}\n", "\n", "config_dict = {\n", " \"Experiment Name\": \"Task09_Spleen\",\n", " \"Seed\": 12345,\n", " \"label_suffix\":\".nii.gz\",\n", " \"Modalities\": {modality_conf[key][\"suffix\"]:key for key in modality_conf },\n", " \"label_dict\": \n", " {\n", " \"background\": 0,\n", " \"Spleen\": 1\n", " },\n", " \"n_folds\":5,\n", " \"FileExtension\": \".nii.gz\"\n", "\n", "}\n", "\n", "with open(\"MAIA/Task09_Spleen_config.json\",\"w\") as f:\n", " json.dump(config_dict,f)" ] }, { "cell_type": "code", "execution_count": null, "id": "5ae00ce8-946b-47b9-b750-e311c44aff11", "metadata": {}, "outputs": [], "source": [ "data_dir = \"/home/maia-user/Tutorials/MAIA/Task09_Spleen\"\n", "\n", "data_list = []\n", "\n", "image_suffix_list = [modality_conf[image_suffix]['suffix'] for image_suffix in modality_conf ]\n", "\n", "subjects = []\n", "for f in os.scandir(pathlib.Path(data_dir).joinpath(\"imagesTr\")):\n", " if f.is_file():\n", "\n", " for image_suffix in image_suffix_list:\n", " if f.name.endswith(image_suffix) and not f.name.startswith(\".\"):\n", " subjects.append(f.name[:-len(image_suffix)])\n", " \n", "subjects = np.unique(subjects)\n", "\n", "for subject in subjects:\n", " data = {}\n", " for modality in modality_conf:\n", " data[modality] = str(pathlib.Path(data_dir).joinpath(\"imagesTr\",subject+modality_conf[modality]['suffix']))\n", " data[\"label\"] = str(pathlib.Path(data_dir).joinpath(\"labelsTr\",subject+\".nii.gz\"))\n", " data_list.append(data)\n", " \n", "data ={\"train\":data_list,\"test\":[]}\n", "\n", "with open(\"MAIA/Task09_Spleen_Dataset.json\",\"w\") as f:\n", " json.dump(data,f)" ] }, { "cell_type": "markdown", "id": "95c94dbc-6281-4ed6-ace0-04d5f5228023", "metadata": {}, "source": [ "## Visualize Sample Case from Dataset\n", "\n", "To verify that the data location and format are correct, we can visualize a sample case from the dataset in 3D Slicer:" ] }, { "cell_type": "code", "execution_count": null, "id": "90b389bf-187b-46be-af5c-43a6640f291a", "metadata": {}, "outputs": [], "source": [ "sample_case = data[\"train\"][0]\n", "\n", "image_volume = sample_case[\"CT\"]\n", "label_volume = sample_case[\"label\"]\n", "slicer_executable = \"/home/maia-user/Documents/Slicer-5.7.0-2024-06-08-linux-amd64/Slicer\"\n", "subprocess.run([\n", " slicer_executable,\n", " \"--python-code\", f\"slicer.util.loadVolume('{image_volume}'); seg=slicer.util.loadSegmentation('{label_volume}'); seg.CreateClosedSurfaceRepresentation()\"\n", "])" ] }, { "cell_type": "markdown", "id": "d6dd8c87-a195-41e4-816b-008234589810", "metadata": {}, "source": [ "## Prepare nnUNet Experiment\n", "\n", "We will use the [`nnunet_prepare_data_folder`](https://pymaia.readthedocs.io/en/latest/apidocs/nnunet_prepare_data_folder.html) command to prepare the data folder for nnUNet training. The command requires the dataset file and the configuration file as input. The command will create the necessary folders and files for the nnUNet training, including the dataset.json file, the plans folder, and the preprocessing folder." ] }, { "cell_type": "code", "execution_count": null, "id": "4a00fccc-c375-4bcb-8a76-858d0aa83277", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "export ROOT_FOLDER=MAIA/Experiments\n", "export PATH=/home/maia-user/.conda/envs/MAIA/bin:$PATH\n", "\n", "nnunet_prepare_data_folder \\\n", "-i MAIA/Task09_Spleen_Dataset.json \\\n", "--task-name Task09_Spleen \\\n", "--task-ID 109 \\\n", "--test-split 0 \\\n", "--config-file MAIA/Task09_Spleen_config.json " ] }, { "cell_type": "markdown", "id": "8517a5ac-c687-4fc4-9471-cdbe3cd20cc0", "metadata": {}, "source": [ "## nnUNet Plan and Preprocessing\n", "\n", "After completing the Data Preparation step, we will run the [`nnunet_run_plan_and_preprocessing`](https://pymaia.readthedocs.io/en/latest/apidocs/nnunet_run_plan_and_preprocessing.html) command to generate the plans and preprocess the data. The command requires the configuration file as input. The command will generate the plans and preprocess the data according to the configuration file." ] }, { "cell_type": "code", "execution_count": null, "id": "a0d9de94-ea09-4975-98c1-8be7b3e6964c", "metadata": { "scrolled": true }, "outputs": [], "source": [ "%%bash\n", "\n", "export ROOT_FOLDER=MAIA/Experiments\n", "export PATH=/home/maia-user/.conda/envs/MAIA/bin:$PATH\n", "\n", "nnunet_run_plan_and_preprocessing \\\n", "--config-file MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen.json \\\n", "--n-workers 2 \\\n", "-pl nnUNetPlannerResEncL" ] }, { "cell_type": "markdown", "id": "1279fe049eea8712", "metadata": {}, "source": [ "## nnUNet Training\n", "\n", "We will train a nnUNet model using a MONAI Bundle specifically designed for nnUNet. To generate the MONAI Bundle configuration files, run the following cells:" ] }, { "cell_type": "code", "execution_count": null, "id": "19537b1e-75fe-4d5b-b23d-07e05bc03f15", "metadata": {}, "outputs": [], "source": [ "def create_config(config_folder, output_file):\n", " config_files = [f.path for f in os.scandir(config_folder) if f.path.endswith(\".yaml\")]\n", " config = {}\n", " for config_file in config_files:\n", " with open(config_file, 'r') as file:\n", " config.update(yaml.safe_load(file))\n", "\n", " if output_file.endswith(\".yaml\"):\n", " with open(output_file, 'w') as file:\n", " yaml.dump(config, file)\n", " if output_file.endswith(\".json\"):\n", " with open(output_file, 'w') as file:\n", " json.dump(config, file)\n", "\n", " return config\n" ] }, { "cell_type": "code", "execution_count": null, "id": "79a082d6-411e-4fb8-aae7-5acf7d818bd2", "metadata": {}, "outputs": [], "source": [ "train_config = create_config(\"nnUNetBundle/nnUNet/\",\"nnUNetBundle/configs/train.json\")\n", "train_config = create_config(\"nnUNetBundle/nnUNet/\",\"nnUNetBundle/configs/train.yaml\")" ] }, { "cell_type": "code", "execution_count": null, "id": "6ae30a71-4b21-4029-b120-d6a396668210", "metadata": {}, "outputs": [], "source": [ "evaluate_config = create_config(\"nnUNetBundle/nnUNet/evaluator/\",\"nnUNetBundle/configs/evaluate.json\")\n", "evaluate_config = create_config(\"nnUNetBundle/nnUNet/evaluator/\",\"nnUNetBundle/configs/evaluate.yaml\")" ] }, { "cell_type": "markdown", "id": "c3f88148889a8662", "metadata": {}, "source": [ "Additionally, since the original nnUNet Scheduler implementation is not compatible with a MONAI Bundle training, we will create a custom PolyLRScheduler class that can be used in the nnUNet training, overriding the original implementation.\n", "\n", "The incompatibility is derived from the missing `get_last_lr` method in the original implementation, which is used to log the learning rate in the MONAI Bundle training." ] }, { "cell_type": "code", "execution_count": null, "id": "3adec8cf-745d-4e5b-af52-4623cf7495e4", "metadata": {}, "outputs": [], "source": [ "%%writefile /home/maia-user/.conda/envs/MAIA/lib/python3.10/site-packages/nnunetv2/training/lr_scheduler/polylr.py\n", "\n", "from torch.optim.lr_scheduler import _LRScheduler\n", "\n", "\n", "class PolyLRScheduler(_LRScheduler):\n", " def __init__(self, optimizer, initial_lr: float, max_steps: int, exponent: float = 0.9, current_step: int = None):\n", " self.optimizer = optimizer\n", " self.initial_lr = initial_lr\n", " self.max_steps = max_steps\n", " self.exponent = exponent\n", " self.ctr = 0\n", " super().__init__(optimizer, current_step if current_step is not None else -1, False)\n", "\n", " def step(self, current_step=None):\n", " if current_step is None or current_step == -1:\n", " current_step = self.ctr\n", " self.ctr += 1\n", "\n", " new_lr = self.initial_lr * (1 - current_step / self.max_steps) ** self.exponent\n", " for param_group in self.optimizer.param_groups:\n", " param_group['lr'] = new_lr\n", "\n", " self._last_lr = [group['lr'] for group in self.optimizer.param_groups]\n", "\n", " def get_last_lr(self):\n", " return self._last_lr" ] }, { "cell_type": "markdown", "id": "aa0b6941225166f7", "metadata": {}, "source": [ "As an alternative to the MONAI Bundle training, we can run the nnUNet training using the [`nnunet_run_training`](https://pymaia.readthedocs.io/en/latest/apidocs/nnunet_run_training.html) command. The command requires the configuration file as input. The command will train the model according to the configuration file." ] }, { "cell_type": "code", "execution_count": null, "id": "c72e50fe-0b7d-449b-9248-c756c70ac085", "metadata": { "scrolled": true }, "outputs": [], "source": [ "%%bash\n", "\n", "export ROOT_FOLDER=./Experiments\n", "\n", "/home/maia-user/.conda/envs/MAIA/bin/python -m monai.bundle run \\/envs/MAIA/bin/nnunet_run_training --config-file MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen.json --n-workers 2 \\\n", "-p nnUNetResEncUNetLPlans \n", "#-tr nnUNetTrainerDemo" ] }, { "cell_type": "markdown", "id": "7d9789516041a4ee", "metadata": {}, "source": [ "The nnUNet MONAI Bundle is designed to log the training metrics to [MLFLow](/mlflow). Additionally, the training metrics are logged to [Tensorboard](/user/demo@maia.se/vscode/proxy/6006/), which can be accessed by running the following command:" ] }, { "cell_type": "code", "execution_count": null, "id": "ff88cf9a-cef3-4957-bec9-f0bf1510bdcc", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "tensorboard --logdir nnUNet_Bundle/eval" ] }, { "cell_type": "markdown", "id": "1f6f6d2b658afd90", "metadata": {}, "source": [ "To start the MONAI Bundle nnUNet training:" ] }, { "cell_type": "code", "execution_count": null, "id": "088bd69d-174d-481d-b10a-66c6430ae6d9", "metadata": { "scrolled": true }, "outputs": [], "source": [ "%%bash\n", "\n", "export BUNDLE=$HOME/Tutorials/nnUNetBundle\n", "export PYTHONPATH=$BUNDLE\n", "\n", "export nnUNet_def_n_proc=2\n", "export nnUNet_n_proc_DA=2\n", "\n", "/home/maia-user/.conda/envs/MAIA/bin/python -m monai.bundle run \\\n", "--bundle-root $BUNDLE \\\n", "--pymaia-config-file MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen.json \\\n", "--task-id 109 \\\n", "--token \"\" \\\n", "--nnunet_plans_identifier nnUNetResEncUNetLPlans \\\n", "--mlflow_experiment_name \"Task09_Spleen\" \\\n", "--tracking_uri $MLFLOW_TRACKING_URI \\\n", "--mlflow_run_name \"nnUNetResEncUNetL\" \\\n", "--nnunet_model_folder $HOME/Tutorials/MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres \\\n", "--config-file $BUNDLE/configs/train.yaml" ] }, { "cell_type": "markdown", "id": "3d25fd32-f8b2-4362-8b06-4c60af2067e2", "metadata": {}, "source": [ "## Validation\n", "\n", "After the training on fold 0 is completed, we can perform the corresponding validation using the [`nnunet_run_training`](https://pymaia.readthedocs.io/en/latest/apidocs/nnunet_run_training.html) command. The command requires the configuration file as input. The command will validate the model according to the configuration file.\n", "\n", "Since the validation is performed according to the native nnUNet scripts, we need to locally convert the trained model from the MONAI Bundle format back to the original nnUNet format. To do this, we will use the `convert_MONAI_to_nnUNet` function:" ] }, { "cell_type": "code", "execution_count": null, "id": "c23ba455-e1d4-447e-9d3b-fc98cfe7677f", "metadata": {}, "outputs": [], "source": [ "def convert_MONAI_to_nnUNet(nnunet_root_folder, nnunet_config, bundle_config):\n", " from PyMAIA.utils.file_utils import subfiles\n", " from nnunetv2.training.logging.nnunet_logger import nnUNetLogger\n", " from pathlib import Path\n", " import json \n", " import torch\n", " from odict import odict\n", " import os\n", " import shutil\n", " \n", " network_weights_prefix = \"_orig_mod.\"\n", " os.environ[\"ROOT_FOLDER\"] = nnunet_root_folder\n", "\n", " os.environ[\"RESULTS_FOLDER\"] = str(\n", " Path(os.environ[\"ROOT_FOLDER\"]).joinpath(\n", " nnunet_config[\"Experiment Name\"], nnunet_config[\"Experiment Name\"] + \"_results\"\n", " )\n", " )\n", "\n", " nnunet_trainer = \"nnUNetTrainer\"\n", " nnunet_plans = \"nnUNetPlans\"\n", "\n", " if \"nnunet_trainer\" in nnunet_config:\n", " nnunet_trainer = nnunet_config[\"nnunet_trainer\"]\n", "\n", " if \"nnunet_plans\" in nnunet_config:\n", " nnunet_plans = nnunet_config[\"nnunet_plans\"]\n", "\n", " nnunet_model_folder = Path(os.environ[\"RESULTS_FOLDER\"]).joinpath(\n", " \"Dataset\" + nnunet_config[\"task_ID\"] + \"_\" + nnunet_config[\n", " \"Experiment Name\"],\n", " f\"{nnunet_trainer}__{nnunet_plans}__3d_fullres\")\n", "\n", " bundle_name = bundle_config[\"Bundle_Name\"]\n", "\n", " nnunet_checkpoint = torch.load(f\"{bundle_name}/models/nnunet_checkpoint.pth\")\n", " latest_checkpoints = subfiles(Path(bundle_name).joinpath(\"models\"),prefix=\"checkpoint_epoch\",sort=True,join=False)\n", " epochs = []\n", " for latest_checkpoint in latest_checkpoints:\n", " epochs.append(int(latest_checkpoint[len(\"checkpoint_epoch=\"):-len(\".pt\")]))\n", "\n", " epochs.sort()\n", " final_epoch = epochs[-1]\n", " monai_last_checkpoint = torch.load(f\"{bundle_name}/models/checkpoint_epoch={final_epoch}.pt\")\n", "\n", " best_checkpoints = subfiles(Path(bundle_name).joinpath(\"models\"), prefix=\"checkpoint_key_metric\", sort=True,\n", " join=False)\n", " key_metrics = []\n", " for best_checkpoint in best_checkpoints:\n", " key_metrics.append(str(best_checkpoint[len(\"checkpoint_key_metric=\"):-len(\".pt\")]))\n", "\n", " key_metrics.sort()\n", " best_key_metric = key_metrics[-1]\n", " monai_best_checkpoint = torch.load(f\"{bundle_name}/models/checkpoint_key_metric={best_key_metric}.pt\")\n", "\n", " nnunet_checkpoint['optimizer_state'] = monai_last_checkpoint['optimizer_state']\n", "\n", "\n", "\n", " nnunet_checkpoint['network_weights'] = odict()\n", "\n", " for key in monai_last_checkpoint['network_weights']:\n", " nnunet_checkpoint['network_weights'][key] = monai_last_checkpoint['network_weights'][key]\n", "\n", " nnunet_checkpoint['current_epoch'] = final_epoch\n", " nnunet_checkpoint['logging'] = nnUNetLogger().get_checkpoint()\n", " nnunet_checkpoint['_best_ema'] = 0\n", " nnunet_checkpoint['grad_scaler_state'] = None\n", "\n", "\n", "\n", " torch.save(nnunet_checkpoint, Path(nnunet_model_folder).joinpath(\"fold_0\",\"checkpoint_final.pth\"))\n", "\n", " nnunet_checkpoint['network_weights'] = odict()\n", "\n", " nnunet_checkpoint['optimizer_state'] = monai_best_checkpoint['optimizer_state']\n", "\n", " for key in monai_best_checkpoint['network_weights']:\n", " nnunet_checkpoint['network_weights'][key] = \\\n", " monai_best_checkpoint['network_weights'][key]\n", "\n", " torch.save(nnunet_checkpoint, Path(nnunet_model_folder).joinpath(\"fold_0\", \"checkpoint_best.pth\"))\n", "\n", " shutil.move(f\"{bundle_name}/models/checkpoint_epoch={final_epoch}.pt\",f\"{bundle_name}/models/model.pt\")\n", " shutil.move(f\"{bundle_name}/models/checkpoint_key_metric={best_key_metric}.pt\",f\"{bundle_name}/models/best_model.pt\")\n", " #shutil.copy(Path(nnunet_model_folder).joinpath(\"fold_0\", \"checkpoint_final.pth\"), f\"{bundle_name}/models/model.pt\")\n", " " ] }, { "cell_type": "code", "execution_count": null, "id": "12eee921-680f-4b47-8f00-ed5c86fcb40c", "metadata": {}, "outputs": [], "source": [ "nnunet_root_folder = \"MAIA/Experiments\"\n", "\n", "\n", "\n", "nnunet_config = {\n", " \"Experiment Name\": \"Task09_Spleen\",\n", " \"task_ID\": \"109\",\n", " \"nnunet_plans\":\"nnUNetResEncUNetLPlans\"\n", "}\n", "\n", "bundle_config = {\n", " \"Bundle_Name\": \"nnUNetBundle\"\n", "}\n" ] }, { "cell_type": "code", "execution_count": null, "id": "dc16eb1d-7784-460c-896b-e2ae1f9786a8", "metadata": {}, "outputs": [], "source": [ "convert_MONAI_to_nnUNet(nnunet_root_folder,nnunet_config, bundle_config)" ] }, { "cell_type": "markdown", "id": "633b7fe0b78c6271", "metadata": {}, "source": [ "After converting the MONAI Bundle to the nnUNet format, we can run the validation:" ] }, { "cell_type": "code", "execution_count": null, "id": "0f2f94f3-ea01-4f1e-99d5-321bcc856203", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "export ROOT_FOLDER=MAIA/Experiments\n", "\n", "\n", "/home/maia-user/.conda/envs/MAIA/bin/nnunet_run_training --config-file MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen.json --n-workers 2 \\\n", "-p nnUNetResEncUNetLPlans --run-fold 0 --run-validation-only yes\n" ] }, { "cell_type": "markdown", "id": "37887f39c3655ad5", "metadata": {}, "source": [ "To visualize in the 3D Slicer the fold-0 validation predictions, together with the ground truth and the CT volume, we can use the following code:" ] }, { "cell_type": "code", "execution_count": null, "id": "9843766b-ae37-4e53-aade-b3d2da5755d9", "metadata": {}, "outputs": [], "source": [ "slicer_executable = \"/home/maia-user/Documents/Slicer-5.7.0-2024-06-08-linux-amd64/Slicer\"\n", "\n", "ct_volume = \"/home/maia-user/Tutorials/MAIA/Task09_Spleen/imagesTr/spleen_10.nii.gz\"\n", "\n", "seg_volume = \"/home/maia-user/Tutorials/MAIA/Task09_Spleen/labelsTr/spleen_10.nii.gz\"\n", "pred_volume = \"/home/maia-user/Tutorials/MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/fold_0/validation/spleen_10.nii.gz\"\n", "\n", "subprocess.run([\n", " slicer_executable,\n", " \"--python-code\", f\"slicer.util.loadVolume('{ct_volume}'); pred=slicer.util.loadSegmentation('{pred_volume}'); seg=slicer.util.loadSegmentation('{seg_volume}'); seg.CreateClosedSurfaceRepresentation()\"\n", "])" ] }, { "cell_type": "markdown", "id": "cc4c49bb5aaa8f16", "metadata": {}, "source": [ "### Validation Metrics\n", "\n", "The validation metrics are stored in a JSON file named `summary.json` in the validation folder. We can load the file and visualize the metrics with [DTale](https://github.com/man-group/dtale) using the following code:" ] }, { "cell_type": "code", "execution_count": 53, "id": "5c2eb82a-331f-4616-bdd9-0030f960cd4e", "metadata": {}, "outputs": [], "source": [ "summary_file= \"MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/fold_0/validation/summary.json\"" ] }, { "cell_type": "code", "execution_count": 54, "id": "bdf643ae-bf75-4d18-81c0-e7d6e968d303", "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'json' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[54], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mopen\u001b[39m(summary_file) \u001b[38;5;28;01mas\u001b[39;00m f:\n\u001b[0;32m----> 2\u001b[0m summary \u001b[38;5;241m=\u001b[39m \u001b[43mjson\u001b[49m\u001b[38;5;241m.\u001b[39mload(f)\n", "\u001b[0;31mNameError\u001b[0m: name 'json' is not defined" ] } ], "source": [ "with open(summary_file) as f:\n", " summary = json.load(f)" ] }, { "cell_type": "code", "execution_count": null, "id": "feda6346-f6a1-4282-8dc6-ba6f5e56a5d7", "metadata": {}, "outputs": [], "source": [ "df = []\n", "\n", "label_to_name = {v: k for k, v in config_dict[\"label_dict\"].items()}\n", "\n", "for case in summary['metric_per_case']:\n", " for label_id in case['metrics']:\n", " for metric in case['metrics'][label_id]:\n", " df.append({\n", " \"Case\": Path(case['reference_file']).name[:-len(config_dict[\"label_suffix\"])],\n", " \"Label\": label_to_name[int(label_id)],\n", " \"Metric\": metric,\n", " \"Value\": case['metrics'][label_id][metric]\n", " })" ] }, { "cell_type": "code", "execution_count": null, "id": "e3e77bb2-6f72-4b44-acb3-c10685baf887", "metadata": {}, "outputs": [], "source": [ "df = pd.DataFrame(df)" ] }, { "cell_type": "code", "execution_count": null, "id": "444b24cf-182d-4183-8fb3-bf9dda62b68c", "metadata": {}, "outputs": [], "source": [ "dtale_app.JUPYTER_SERVER_PROXY = True\n", "\n", "d = dtale.show(df,host=\"0.0.0.0\",)" ] }, { "cell_type": "code", "execution_count": null, "id": "6fa3f1c1-647e-47a2-ace4-7c5ad8dce784", "metadata": {}, "outputs": [], "source": [ "from IPython.display import Markdown\n", "from IPython.core.magic import register_cell_magic\n", "import os\n", "\n", "\n", "DTALE_URL = d._main_url\n", "@register_cell_magic\n", "def markdown(line, cell):\n", " return Markdown(cell.format(**globals()))" ] }, { "cell_type": "code", "execution_count": null, "id": "bbdefb9b-9359-42e8-aa45-830e41aced6f", "metadata": {}, "outputs": [], "source": [ "%%markdown\n", "\n", "[DTale]({DTALE_URL})" ] }, { "cell_type": "markdown", "id": "a28e1768177a6b7c", "metadata": {}, "source": [ "The DTale charts can be recreated and visualized in the notebook using the following code:" ] }, { "cell_type": "code", "execution_count": null, "id": "660e7045-c349-40b4-a8be-71fd544a9690", "metadata": { "scrolled": true }, "outputs": [], "source": [ "# DISCLAIMER: 'df' refers to the data you passed in when calling 'dtale.show'\n", "\n", "import pandas as pd\n", "\n", "if isinstance(df, (pd.DatetimeIndex, pd.MultiIndex)):\n", "\tdf = df.to_frame(index=False)\n", "\n", "# remove any pre-existing indices for ease of use in the D-Tale code, but this is not required\n", "df = df.reset_index().drop('index', axis=1, errors='ignore')\n", "df.columns = [str(c) for c in df.columns] # update columns to strings in case they are numbers\n", "\n", "df = df.query(\"\"\"`Metric` == 'Dice'\"\"\")\n", "\n", "chart_data = pd.concat([\n", "\tdf['Case'],\n", "\tdf['Value'],\n", "], axis=1)\n", "chart_data = chart_data.sort_values(['Case'])\n", "chart_data = chart_data.rename(columns={'Case': 'x'})\n", "chart_data = chart_data.dropna()\n", "\n", "import plotly.graph_objs as go\n", "\n", "charts = []\n", "charts.append(go.Bar(\n", "\tx=chart_data['x'],\n", "\ty=chart_data['Value']\n", "))\n", "figure = go.Figure(data=charts, layout=go.Layout({\n", " 'barmode': 'group',\n", " 'legend': {'orientation': 'h', 'y': -0.3},\n", " 'title': {'text': 'Validation Fold 0, Dice score'},\n", " 'xaxis': {'title': {'text': 'Case'}},\n", " 'yaxis': {'title': {'text': 'Dice'}, 'type': 'linear'}\n", "}))\n", "\n", "# If you're having trouble viewing your chart in your notebook try passing your 'chart' into this snippet:\n", "#\n", "from plotly.offline import iplot, init_notebook_mode\n", "#\n", "init_notebook_mode(connected=True)\n", "for chart in charts:\n", " chart.pop('id', None) # for some reason iplot does not like 'id'\n", "iplot(figure)\n", "\n", "figure.write_html(\"Fold_0_Val_Dice.html\")" ] }, { "cell_type": "code", "execution_count": null, "id": "c328ca1d-1e92-4b95-8a8b-0c75d8ae76dd", "metadata": {}, "outputs": [], "source": [ "df.groupby([\"Metric\"]).describe()" ] }, { "cell_type": "code", "execution_count": null, "id": "5afd83c8-edb8-49ae-a9c7-03c231bd3363", "metadata": {}, "outputs": [], "source": [ "df.groupby([\"Metric\"]).describe()['Value']['mean'].values[0]" ] }, { "cell_type": "markdown", "id": "88c055b67677ddff", "metadata": {}, "source": [ "Finally, we can upload the validation metrics and the plots to MLFlow using the following code:" ] }, { "cell_type": "code", "execution_count": null, "id": "67c7a4a7-162b-4a47-b1cf-e51b9101b4ed", "metadata": {}, "outputs": [], "source": [ "mlflow.set_tracking_uri(os.environ[\"MLFLOW_TRACKING_URI\"])\n", "mlflow.set_experiment(\"Task09_Spleen\")\n", "\n", "with mlflow.start_run(run_id=\"58d802747d7d493092d21287cc16af2d\"):\n", " mean_dice = df.groupby([\"Metric\"]).describe()['Value']['mean'].values[0]\n", " \n", " mlflow.log_metric(\"Val_Dice_Fold_0\", mean_dice)\n", " mlflow.log_artifact(\"Fold_0_Val_Dice.html\")\n", " " ] }, { "cell_type": "markdown", "id": "acdba2c483d1906f", "metadata": {}, "source": [ "In the final step of the validation phase, we export the trained model, saving the nnUNet Bundle as a zip file (`Task09_Spleen_nnUNet.zip`). The zip file contains the model, the configuration files, and the environment files." ] }, { "cell_type": "code", "execution_count": null, "id": "b53c9cfa-8eba-46d5-bb48-8c82e793f9f3", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "export ROOT_FOLDER=MAIA/Experiments\n", "export PATH=/home/maia-user/.conda/envs/MAIA/bin:$PATH\n", "\n", "touch MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/fold_0/progress.png\n", "\n", "/home/maia-user/.conda/envs/MAIA/bin/nnunet_run_training --config-file MAIA/Experiments/Task09_Spleen/Task09_Spleen_results/Dataset109_Task09_Spleen.json --n-workers 2 \\\n", "-p nnUNetResEncUNetLPlans --run-fold -1 --output-model-file Task09_Spleen_nnUNet.zip --post-processing-folds 0" ] }, { "cell_type": "markdown", "id": "4610c3f7b3b01f30", "metadata": {}, "source": [ "## Package MONAI Bundle\n", "\n", "After completing the training and validation of the nnUNet model, we can package the model as a MONAI Bundle, to be used for inference (i.e. Active Learning with MONAI Label). We first copy the trained model into the bundle folder and remove the validation results:" ] }, { "cell_type": "code", "execution_count": null, "id": "bda3a636-c26f-4906-b91f-1c76b87ef03b", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "#OPTIONAL STEP, NOT REQUIRED ANYMORE\n", "\n", "unzip Task09_Spleen_nnUNet.zip -d nnUNetBundle/models\n", "\n", "rm -r nnUNetBundle/models/Dataset109_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/fold_0/validation\n", "rm -r nnUNetBundle/models/Dataset109_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/crossval_results_folds_0" ] }, { "cell_type": "markdown", "id": "81b69b5551a08604", "metadata": {}, "source": [ "We then test it by running the model on a sample case from the dataset:" ] }, { "cell_type": "code", "execution_count": null, "id": "5fd98141-f1e7-45e7-806b-1493c723dbe6", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "rm -r MAIA/MONAI_Bundle\n", "mkdir -p MAIA/MONAI_Bundle/input\n", "mkdir -p MAIA/MONAI_Bundle/output\n", "mkdir -p MAIA/MONAI_Bundle/input/spleen_1\n", "\n", "cp MAIA/Task09_Spleen/imagesTs/spleen_1.nii.gz MAIA/MONAI_Bundle/input/spleen_1" ] }, { "cell_type": "code", "execution_count": null, "id": "8b4828ab-9650-4c36-b797-ac6d94b30feb", "metadata": { "scrolled": true }, "outputs": [], "source": [ "%%bash\n", "\n", "/home/maia-user/.conda/envs/MAIA/bin/python -m monai.bundle run \\\n", " --config-file nnUNetBundle/configs/inference.yaml \\\n", " --bundle-root nnUNetBundle \\\n", " --data-dir MAIA/MONAI_Bundle/input \\\n", " --output-dir MAIA/MONAI_Bundle/output \\\n", " --logging-file nnUNetBundle/configs/logging.conf" ] }, { "cell_type": "markdown", "id": "6218be4f305153db", "metadata": {}, "source": [ "To visualize the prediction in 3D Slicer:" ] }, { "cell_type": "code", "execution_count": null, "id": "3d2a5db0-1d8b-4130-a4a5-3e947b342da8", "metadata": {}, "outputs": [], "source": [ "slicer_executable = \"/home/maia-user/Documents/Slicer-5.7.0-2024-06-08-linux-amd64/Slicer\"\n", "\n", "ct_volume = \"/home/maia-user/Tutorials/MAIA/MONAI_Bundle/input/spleen_1/spleen_1.nii.gz\"\n", "\n", "pred_volume = \"/home/maia-user/Tutorials/MAIA/MONAI_Bundle/output/spleen_1/spleen_1_prediction.nii.gz\"\n", "\n", "subprocess.run([\n", " slicer_executable,\n", " \"--python-code\", f\"slicer.util.loadVolume('{ct_volume}'); pred=slicer.util.loadSegmentation('{pred_volume}'); pred.CreateClosedSurfaceRepresentation()\"\n", "])" ] }, { "cell_type": "markdown", "id": "1c7f21258d74bfa7", "metadata": {}, "source": [ "We finally export the python environment and the requirements for the MONAI Bundle:" ] }, { "cell_type": "code", "execution_count": null, "id": "f844b1f8-7960-4cf9-97e3-884db90e18d7", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "/opt/conda/bin/conda env export -n MAIA > nnUNetBundle/environment.yml\n", "/home/maia-user/.conda/envs/MAIA/bin/python -m pip freeze > nnUNetBundle/requirements.txt" ] }, { "cell_type": "markdown", "id": "133f4eed", "metadata": {}, "source": [ "We can then create the `metadata` file for the MONAI Bundle:" ] }, { "cell_type": "code", "execution_count": null, "id": "82e18f5f", "metadata": {}, "outputs": [], "source": [ "import monai\n", "import torch\n", "import nnunetv2\n", "import PyMAIA\n", "import pydicom\n", "import numpy\n", "import ignite\n", "#import lightning\n", "from importlib.metadata import version\n", "\n", "\n", "MONAI_VERSION = monai.__version__\n", "PYTORCH_VERSION = torch.__version__\n", "NNUNET_VERSION = version(\"nnunetv2\")\n", "PYMAIA_VERSION = PyMAIA.__version__\n", "PYDICOM_VERSION = pydicom.__version__\n", "NUMPY_VERSION = numpy.__version__\n", "IGNITE_VERSION = ignite.__version__\n", "#LIGHTNING_VERSION = lightning.__version__\n", "\n", "patch_size = net.predictor.configuration_manager.patch_size" ] }, { "cell_type": "code", "execution_count": null, "id": "26c22777", "metadata": {}, "outputs": [], "source": [ "#nnUNetBundle/configs/metadata.json\n", "\n", "metadata = {\n", " \"version\": \"0.1.0\",\n", " \"monai_version\": MONAI_VERSION,\n", " \"pytorch_version\": PYTORCH_VERSION,\n", " \"numpy_version\": NUMPY_VERSION,\n", " \"optional_packages_version\": {\n", " \"nnunetv2\": NNUNET_VERSION,\n", " \"pytorch-ignite\": IGNITE_VERSION,\n", " \"pymaia_learn\": PYMAIA_VERSION,\n", " \"pydicom\": PYDICOM_VERSION,\n", " #\"lightning\": lightning.__version__\n", " },\n", " \"task\": \"Spleen Segmentation with ResEnc nnUNet [L]\",\n", " \"description\": \"A nnUNet MONAI Bundle for Spleen Segmentation in CT, used for nnUNet-based Experiments within the MONAI Bundle framework.\",\n", " \"authors\": \"Simone Bendazzoli\",\n", " \"copyright\": \"Copyright 2024, Apache Software Foundation\",\n", " \"data_source\": \"Decathlon Dataset, Task 09\",\n", " \"data_type\": \"nifti\",\n", " \"image_classes\": \"1 single channel volumes: CT\",\n", " \"label_classes\": \"single channel data: 0 is background, 1 is spleen\",\n", " \"pred_classes\": \"single channel data: 0 is background, 1 is spleen\",\n", " \"network_data_format\":\n", " {\n", " \"inputs\": {\n", " \"image\": {\n", " \"type\": \"image\",\n", " \"format\": \"magnitude\",\n", " \"modality\": \"CT\",\n", " \"num_channels\": 1,\n", " \"spatial_shape\": patch_size,\n", " \"dtype\": \"float32\",\n", " \"value_range\": [0, 1],\n", " \"is_patch_data\": True,\n", " \"channel_def\": {\"0\": \"ct\"}\n", " }\n", " },\n", " \"outputs\":{\n", " \"pred\": {\n", " \"type\": \"image\",\n", " \"format\": \"segmentation\",\n", " \"num_channels\": 1,\n", " \"spatial_shape\": patch_size,\n", " \"dtype\": \"float32\",\n", " \"value_range\": [0,1],\n", " \"is_patch_data\": True,\n", " \"channel_def\": {\"0\": \"Background\", \"1\": \"Spleen\"}\n", " }\n", " }\n", " },\n", " \"references\": [\n", " \"Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 203-211.\"\n", " ]\n", "}\n", "\n", "with open(\"nnUNetBundle/configs/metadata.json\",\"w\") as f:\n", " json.dump(metadata,f,indent=4)" ] }, { "cell_type": "code", "execution_count": 51, "id": "b99a717f-c9c7-4949-9910-8d51ab4b04eb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting nnUNetBundle/requirements.txt\n" ] } ], "source": [ "%%writefile nnUNetBundle/requirements.txt\n", "nnunetv2==2.5.1\n", "odict\n", "pymaia-learn\n", "fire" ] }, { "cell_type": "markdown", "id": "dac8ae2ca7b3fc5c", "metadata": {}, "source": [ "Optionally, before packaging the MONAI Bundle, we need to set the `model_folder` parameter in the `inference.yaml` file.\n", "\n", "This step is only needed if the native nnUNet Inference is used.\n", "\n", "```yaml\n", "model_folder: /models/Dataset109_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres\n", "```" ] }, { "cell_type": "code", "execution_count": 52, "id": "e8b38cca-e5c1-4f56-bd44-e03097653e5f", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " adding: Task09_Spleen_Bundle/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/src/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/src/inferer.py (deflated 74%)\n", " adding: Task09_Spleen_Bundle/src/__init__.py (stored 0%)\n", " adding: Task09_Spleen_Bundle/src/__pycache__/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/src/__pycache__/__init__.cpython-310.pyc (deflated 22%)\n", " adding: Task09_Spleen_Bundle/src/__pycache__/nnUNet_Trainer.cpython-310.pyc (deflated 44%)\n", " adding: Task09_Spleen_Bundle/src/__pycache__/utils.cpython-310.pyc (deflated 40%)\n", " adding: Task09_Spleen_Bundle/src/__pycache__/dataset.cpython-310.pyc (deflated 31%)\n", " adding: Task09_Spleen_Bundle/src/__pycache__/inferer.cpython-310.pyc (deflated 47%)\n", " adding: Task09_Spleen_Bundle/src/nnUNet_Trainer.py (deflated 76%)\n", " adding: Task09_Spleen_Bundle/src/utils.py (deflated 68%)\n", " adding: Task09_Spleen_Bundle/src/dataset.py (deflated 43%)\n", " adding: Task09_Spleen_Bundle/configs/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/configs/evaluate.yaml (deflated 62%)\n", " adding: Task09_Spleen_Bundle/configs/train.yaml (deflated 74%)\n", " adding: Task09_Spleen_Bundle/configs/.ipynb_checkpoints/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/configs/.ipynb_checkpoints/inference-checkpoint.json (deflated 71%)\n", " adding: Task09_Spleen_Bundle/configs/train.json (deflated 74%)\n", " adding: Task09_Spleen_Bundle/configs/inference.yaml (deflated 63%)\n", " adding: Task09_Spleen_Bundle/configs/metadata.json (deflated 68%)\n", " adding: Task09_Spleen_Bundle/configs/evaluate.json (deflated 62%)\n", " adding: Task09_Spleen_Bundle/environment.yml (deflated 63%)\n", " adding: Task09_Spleen_Bundle/.ipynb_checkpoints/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/.ipynb_checkpoints/environment-checkpoint.yml (deflated 63%)\n", " adding: Task09_Spleen_Bundle/.ipynb_checkpoints/requirements-checkpoint.txt (deflated 56%)\n", " adding: Task09_Spleen_Bundle/nnUNet/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/nnUNet/evaluator/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/nnUNet/evaluator/evaluator.yaml (deflated 61%)\n", " adding: Task09_Spleen_Bundle/nnUNet/run.yaml (deflated 46%)\n", " adding: Task09_Spleen_Bundle/nnUNet/train.yaml (deflated 70%)\n", " adding: Task09_Spleen_Bundle/nnUNet/imports.yaml (deflated 46%)\n", " adding: Task09_Spleen_Bundle/nnUNet/validate.yaml (deflated 63%)\n", " adding: Task09_Spleen_Bundle/nnUNet/params.yaml (deflated 38%)\n", " adding: Task09_Spleen_Bundle/nnUNet/global.yaml (deflated 42%)\n", " adding: Task09_Spleen_Bundle/nnUNet/train_handlers.yaml (deflated 57%)\n", " adding: Task09_Spleen_Bundle/models/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/models/dataset.json (deflated 91%)\n", " adding: Task09_Spleen_Bundle/models/.ipynb_checkpoints/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/models/best_model.pt (deflated 7%)\n", " adding: Task09_Spleen_Bundle/models/plans.json (deflated 89%)\n", " adding: Task09_Spleen_Bundle/models/nnunet_checkpoint.pth (deflated 66%)\n", " adding: Task09_Spleen_Bundle/models/model.pt (deflated 7%)\n", " adding: Task09_Spleen_Bundle/LICENSE (stored 0%)\n", " adding: Task09_Spleen_Bundle/eval/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/eval/events.out.tfevents.1732567232.jupyter-demo-40maia-2ese.17102.0 (deflated 70%)\n", " adding: Task09_Spleen_Bundle/eval/events.out.tfevents.1732567232.jupyter-demo-40maia-2ese.17102.1 (deflated 66%)\n", " adding: Task09_Spleen_Bundle/requirements.txt (stored 0%)\n", " adding: Task09_Spleen_Bundle/docs/ (stored 0%)\n", " adding: Task09_Spleen_Bundle/docs/README.md (deflated 49%)\n" ] } ], "source": [ "%%bash\n", "\n", "rm -r Task09_Spleen_Bundle\n", "rm -r Task09_Spleen_Bundle.zip\n", "cp -r nnUNetBundle Task09_Spleen_Bundle\n", "zip -r Task09_Spleen_Bundle.zip Task09_Spleen_Bundle" ] }, { "cell_type": "markdown", "id": "d0d360d4-124e-43de-9eee-bb353374ae68", "metadata": {}, "source": [ "## MLFlow Model Upload\n", "\n", "To store the model and be able to deploy it for inference in future use cases, we can upload it to MLFlow. We will use the MLFlow Python API to log the model:" ] }, { "cell_type": "code", "execution_count": null, "id": "b18898f8-eff4-4916-9b96-d52cf2170951", "metadata": {}, "outputs": [], "source": [ "import sys\n", "import os\n", "import yaml\n", "from monai.bundle import ConfigParser\n", "import torch\n", "import numpy as np\n", "import mlflow\n", "from mlflow.models import ModelSignature\n", "from mlflow.types.schema import Schema, TensorSpec\n", "\n", "sys.path.append(\"Task09_Spleen_Bundle\")" ] }, { "cell_type": "code", "execution_count": null, "id": "fbe37c42-7ae8-4233-a049-a28b63ae7d8b", "metadata": {}, "outputs": [], "source": [ "config_files = [f.path for f in os.scandir(\"Task09_Spleen_Bundle/configs\") if f.path.endswith(\"inference.yaml\")]\n", "\n", "config = {}\n", "for config_file in config_files:\n", " with open(config_file, 'r') as file:\n", " config.update(yaml.safe_load(file))\n", "\n", "config[\"bundle_root\"] = \"Task09_Spleen_Bundle\"\n", "#config[\"model_folder\"] = \"Dataset109_Spleen/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres\"\n", "parser = ConfigParser(config,globals={\"os\": \"os\",\n", " \"pathlib\":\"pathlib\",\n", " \"json\":\"json\",\n", " \"ignite\":\"ignite\"\n", " })\n", "\n", "parser.parse(True)" ] }, { "cell_type": "code", "execution_count": null, "id": "5954a3d2-1023-4434-bf7e-8ac778c158b0", "metadata": { "scrolled": true }, "outputs": [], "source": [ "net = parser.get_parsed_content(\"network_def\",instantiate=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "7a0d65dd-d938-4594-9a7b-80ae986f585a", "metadata": {}, "outputs": [], "source": [ "net.network_weights.load_state_dict(torch.load(\"nnUNet_Bundle/models/model.pt\")['network_weights'])" ] }, { "cell_type": "code", "execution_count": null, "id": "ca526ca8-6cf2-41a6-ad9f-15a2e77f55e3", "metadata": {}, "outputs": [], "source": [ "os.environ[\"MLFLOW_TRACKING_URI\"] = \"http://mlflow-v1-mkg:5000\"" ] }, { "cell_type": "code", "execution_count": null, "id": "ed432d0a-9684-4b6e-bc0a-ab1f72cf95ec", "metadata": { "scrolled": true }, "outputs": [], "source": [ "mlflow.set_experiment(\"Task09_Spleen\")\n", "mlflow.end_run()\n", "\n", "\n", "\n", "input_schema = Schema(\n", " [\n", " TensorSpec(np.dtype(np.float32), (1, *net.predictor.configuration_manager.patch_size),name=\"ct\")\n", " \n", " ]\n", " \n", ")\n", "output_schema = Schema([TensorSpec(np.dtype(np.float32), (1, *net.predictor.configuration_manager.patch_size),name=\"Spleen\")])\n", "\n", "signature = ModelSignature(inputs=input_schema, outputs=output_schema)\n", "\n", "with mlflow.start_run(run_id='58d802747d7d493092d21287cc16af2d'):\n", " mlflow.pytorch.log_model(\n", " net,\n", " \"Task09_Spleen\",\n", " signature=signature,\n", " conda_env = \"/home/maia-user/Tutorials/nnUNet_Bundle/environment.yml\",\n", " registered_model_name = \"Task09_Spleen\",\n", " extra_files = [\n", " \"/home/maia-user/Tutorials/Task09_Spleen_Bundle.zip\",\n", " \"/home/maia-user/Tutorials/nnUNet_Bundle/environment.yml\",\n", " \"/home/maia-user/Tutorials/nnUNet_Bundle/requirements.txt\"\n", " ]\n", " )" ] }, { "cell_type": "markdown", "id": "72e66c1a-4702-4d39-b498-9ada8f1e0c7b", "metadata": {}, "source": [ "## Deploy Model for Inference in MAIA\n", "\n", "To deploy the Spleen segmentation model as a service in MAIA, run the following command to install it as an Helm chart:" ] }, { "cell_type": "code", "execution_count": null, "id": "5b5e9da8-301b-410a-8456-7b2bf5f84874", "metadata": {}, "outputs": [], "source": [ "from kfp import kubernetes\n", "from kfp import client\n", "from kfp import dsl\n", "from kfp import compiler\n", "\n", "from pathlib import Path" ] }, { "cell_type": "code", "execution_count": null, "id": "632be399-5634-468b-9fa9-ba2dac750d4b", "metadata": {}, "outputs": [], "source": [ "Path(\"/home/maia-user/shared\").joinpath(\"mlflow-models\",\"Task09_Spleen\",\"extra_files\").mkdir(parents=True, exist_ok=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "4323b367-142e-4f0f-aa1f-b43c618df3c3", "metadata": {}, "outputs": [], "source": [ "import shutil\n", "\n", "shutil.copy(\"/home/maia-user/Tutorials/Task09_Spleen_Bundle.zip\",Path(\"/home/maia-user/shared\").joinpath(\"mlflow-models\",\"Task09_Spleen\",\"extra_files\"))" ] }, { "cell_type": "code", "execution_count": null, "id": "1d49c732-69e8-4147-b06b-68f60570de12", "metadata": {}, "outputs": [], "source": [ "@dsl.component(base_image='kthcloud/maia-workspace-admin:1.5')\n", "def helm_install_monai_label_ohif(cluster_api: str,namespace: str, user_id: str, id_token: str):\n", "\n", " import subprocess\n", " from pathlib import Path\n", " import os\n", " import yaml\n", " def generate_kubeconfig(id_token,user_id,namespace,cluster_api):\n", " kube_config = {'apiVersion': 'v1', 'kind': 'Config', 'preferences': {},\n", " 'current-context': 'MAIA/{}'.format(user_id), 'contexts': [\n", " {'name': 'MAIA/{}'.format(user_id),\n", " 'context': {'user': user_id, 'cluster': 'MAIA', 'namespace': namespace}}],\n", " 'clusters': [\n", " {'name': 'MAIA', 'cluster': {'certificate-authority-data': \"\",\n", " 'server': cluster_api,\n", " \n", " \"insecure-skip-tls-verify\": True}}],\n", " \"users\": [{'name': user_id,\n", " 'user': {'token': id_token}}]}\n", " return kube_config\n", " \n", " kubeconfig_dict = generate_kubeconfig(id_token,user_id,namespace,cluster_api)\n", " \n", " with open(Path(\".\").joinpath(\"kubeconfig\"), \"w\") as f:\n", " yaml.dump(kubeconfig_dict, f)\n", " os.environ[\"KUBECONFIG\"] = \"kubeconfig\"\n", " subprocess.run([\"helm\",\"repo\",\"add\",\"maia\",\"https://kthcloud.github.io/MAIA/\"])\n", " subprocess.run([\"helm\",\"repo\",\"update\"])\n", " subprocess.run([\"helm\", \"install\", \n", " \"spleen-segmentation\",\n", " \"-n\", namespace,\n", " \"maia/monai-label-ohif-maia\",\n", " \"--set\", \"hostname=monai-demo.maia.cloud.cbh.kth.se\",\n", " \"--set\", \"pvc.pvc_type=nfs\",\n", " \"--set\", \"bundle_model_name=Task09_Spleen_Bundle\",\n", " \"--set\", \"mlflow_pvc_name=shared\",\n", " \"--set\", \"mlflow_model_path=/workspace/mlflow/mlflow-models/Task09_Spleen\",\n", " ])" ] }, { "cell_type": "code", "execution_count": null, "id": "8a325edd-85ec-4c1a-9eb5-70921f3b87ce", "metadata": {}, "outputs": [], "source": [ "@dsl.pipeline\n", "def install_monai_label_ohif_pipeline(cluster_api: str,namespace: str, user_id: str, id_token: str):\n", " task1 = helm_install_monai_label_ohif(cluster_api=cluster_api,namespace=namespace,user_id=user_id,id_token=id_token)\n", "\n", " kubernetes.set_image_pull_secrets(task1, secret_names=[\"harbor-maia-docker-registry-secret\"])" ] }, { "cell_type": "code", "execution_count": null, "id": "0918eace-9f2e-4131-9751-986eae31d7f9", "metadata": {}, "outputs": [], "source": [ "compiler.Compiler().compile(install_monai_label_ohif_pipeline, package_path='Install_MONAI_Label_OHIF_pipeline.yaml')" ] }, { "cell_type": "code", "execution_count": null, "id": "fe63f1f5-9ab0-46f2-9a07-8c52b22ef519", "metadata": {}, "outputs": [], "source": [ "token = \"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ2T3FLaTZ2MGNHYXhxVzFOaFZiOFh5ekZqbDJERFRkUEhHTWgtTXVEbXNNIn0.eyJleHAiOjE3MzU2NDg0MTksImlhdCI6MTczMzA1NjQxOSwiYXV0aF90aW1lIjoxNzMzMDU2NDE5LCJqdGkiOiI1YWVkNDJhYi1mZTAxLTRlNGYtYTFmOS1hMWExNWYzYmFiODQiLCJpc3MiOiJodHRwczovL2lhbS5jbG91ZC5jYmgua3RoLnNlL3JlYWxtcy9jbG91ZCIsImF1ZCI6Im1haWEiLCJzdWIiOiI4YTdjYTk4Ny05M2E0LTRlOTgtODY3Ny00NWI5NmRiYzAzYzUiLCJ0eXAiOiJJRCIsImF6cCI6Im1haWEiLCJzaWQiOiIyMDhlODI4MC04MzI1LTQ5MjYtYTQ4MC01NzA0ZWJlY2FiMDMiLCJhdF9oYXNoIjoiWTdIM2NwY0FKckwtUG13Q3FpdVlxdyIsImFjciI6IjEiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwibmFtZSI6IkRlbW8gTUFJQSIsImdyb3VwcyI6WyJNQUlBOk1PTkFJX0RlbW9fMiIsIk1BSUE6bW9uYWktZGVtbyIsIk1BSUE6dXNlcnMiXSwicHJlZmVycmVkX3VzZXJuYW1lIjoiZGVtb0BtYWlhLnNlIiwiZ2l2ZW5fbmFtZSI6IkRlbW8iLCJmYW1pbHlfbmFtZSI6Ik1BSUEiLCJlbWFpbCI6ImRlbW9AbWFpYS5zZSJ9.WJck1-YyVf12eQFjCW2kSFMjVuQJsou_5XPI9luGD5CgTj2MvguYm9vlpZivaDN_abaRzKz-5301Y2TJhQaLt-O2Nfs3rFo0ddcKxnZu1uAZzBxh1KhdtjKjbNfMIPaWLz7D8m664C4tOgHv6jk-sTiqBwE0R3aTSuFRzd7FgMxdn00e23bzIUnJYqSgwXKRkTCUivw2ZopcMjBj6ONFb3l_kHsBA-xo26CLtUtHIUlqndJgfMoa05RXkUavbLzS09vD6vZxVET-ZV3-NX7mMRbm3d7CHtE0cLATxv4tj8oO3KiirKFmtdnLqoFNC5tsgenqkcUhKal31EMOMYVnGNL_Ijbza_Jp40xCm3o-s2vyV3B0W-SQV53cj3jxztMQa1IC3PAQ_UxRiKxIUnc_YWzdNvt0cWoLz33OozOJvvkIJq6EwfbeL1kxHlflzZmo6qnl3MayW9-FMAtne4iE5r_d0IMaawMr0YyE8QxcKhOxckde5f0nrNZtjCfeykGAZwt99l-DPZqTDFoiokDWr7qVYvcU9QHQmI73Sfb8tj78KkMHgSOHs1Bsv2I5htBfOd9GyNhjmPN_b-9yiy-TcUs-Sx1oYR5ur8MMVcJi-ZNxv9egAhrgtQYnSA_XtJ7n_8SukrpeQTSMbGlF4RRYDBS1_Mte9bWW6KJZY8CwTBY\"" ] }, { "cell_type": "code", "execution_count": null, "id": "62b2db2f-49b2-4faf-be78-4b847474b0f8", "metadata": {}, "outputs": [], "source": [ "kfp_client = client.Client(host=\"http://ml-pipeline-ui\")\n", "kfp_client.create_run_from_pipeline_package('Install_MONAI_Label_OHIF_pipeline.yaml',\n", " arguments={\n", " \"cluster_api\":\"https://kubernetes.default:443\",\n", " \n", " \"namespace\":\"monai-demo\",\n", " \"user_id\":\"monai-demo@kth.se\",\n", " \"id_token\":token\n", " })" ] }, { "cell_type": "markdown", "id": "848f8e9f-e328-42f4-9ec1-4d79540b5930", "metadata": {}, "source": [ "After deploy the Helm Chart, you can get access to [MONAI Label](https://monai-demo.maia.cloud.cbh.kth.se/spleen-segmentation-monai-label/info/) and the linked [Orthanc server](https://monai-demo.maia.cloud.cbh.kth.se/spleen-segmentation/). \n", "\n", "To perform Active Learning with MONAI Label from the OHIF Viewer: [OHIF Viewer](https://monai-demo.maia.cloud.cbh.kth.se/spleen-segmentation-monai-label/ohif/)" ] } ], "metadata": { "kernelspec": { "display_name": "MAIA", "language": "python", "name": "maia" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.15" } }, "nbformat": 4, "nbformat_minor": 5 }