Infer using a fine-tuned model

You can use any classifier (model) to predict labels on unannotated patches.

Initialize ClassifierContainer()

To initialize your ClassifierContainer() for inference, you will need to define:

  • model - The model (classifier) you would like to use.

  • labels_map - A dictionary mapping your labels to their indices (e.g. {0: "no_railspace", 1: "railspace"}). This labels map should be the same as that used when training/fine-tuning the classifier.

There are a number of options for the model argument:

1. To load a locally-saved model, use ``torch.load()`` to load your file and then pass this as the ``model`` argument.

If you have already trained a model using MapReader, your outputs, by default, should be saved in directory called models. Within this directory will be checkpoint_X.pkl and model_checkpoint_X.pkl files. Your models are saved in the model_checkpoint_X.pkl files.

e.g. To load one of these files:

#EXAMPLE
import torch

my_model = torch.load("./models/model_checkpoint_6.pkl")
labels_map = {0: "no_railspace", 1: "railspace"}

my_classifier = ClassifierContainer(my_model, labels_map)

Advanced usage

The checkpoint_X.pkl files contain all the information, except for your models (which is saved in the model_checkpoint_X.pkl files), you had previously loaded in to your ClassifierContainer(). If you have already trained a model using MapReader, you can use these files to reload your previously used ClassifierContainer().

To do this, set the model, dataloaders and label_map arguments to None and pass load_path="./models/your_checkpoint_file.pkl" when initializing your ClassifierContainer():

#EXAMPLE
my_classifier = ClassifierContainer(None, None, None, load_path="./models/checkpoint_6.pkl")

This will also load the corresponding model file (in this case “./models/model_checkpoint_6.pkl”).

If you use this option, your optimizer, scheduler and criterion will be loaded from last time.

2. To load a hugging face model, choose your model, follow the “Use in Transformers” or “Use in timm” instructions to load it and then pass this as the ``model`` argument.

e.g. This model is based on our *gold standard* dataset. It can be loaded using the transformers library:

#EXAMPLE
from transformers import AutoFeatureExtractor, AutoModelForImageClassification

extractor = AutoFeatureExtractor.from_pretrained("davanstrien/autotrain-mapreader-5000-40830105612")
my_model = AutoModelForImageClassification.from_pretrained("davanstrien/autotrain-mapreader-5000-40830105612")
labels_map = {0: "no_railspace", 1: "railspace"}

my_classifier = ClassifierContainer(my_model, labels_map)

Note

You will need to install the transformers library to do this (pip install transformers).

e.g. This model is an example of one which uses the timm library. It can be loaded as follows:

#EXAMPLE
import timm

my_model = timm.create_model("hf_hub:timm/resnest101e.in1k", pretrained=True, num_classes=len(annotated_images.labels_map))

my_classifier = ClassifierContainer(my_model, annotated_images.labels_map, dataloaders)

Note

You will need to install the timm library to do this (pip install timm).

Create dataset and add to my_classifier

You will then need to create a new dataset containing your unannotated patches. This can be done by loading a dataframe containing the paths to your patches:

from mapreader import PatchDataset

infer = PatchDataset("./patch_df.csv", delimiter=",", transform="test")

Note

You can create this .csv file using the .convert_image(save=True) method on your MapImages object (follow instructions in the Load user guidance).

The transform argument is used to specify which image transforms to use on your patch images. See this section for more information on transforms.

You should then add this dataset to your ClassifierContainer() (my_classifier):

my_classifier.load_dataset(infer, set_name="infer")

This will create a DataLoader from your dataset and add it to your ClassifierContainer()'s dataloaders attribute.

By default, the .load_dataset() method will create a dataloader with batch size of 16 and will not use a sampler. You can change these by specifying the batch_size and sampler arguments respectively. See this section for more information on samplers.

Infer

After loading your dataset, you can then simply run the .inference() method to infer the labels on the patches in your dataset:

my_classifier.inference(set_name="infer")

As with the “test” dataset, to see a sample of your predictions, use:

my_classifier.show_inference_sample_results(label="railspace", set_name="infer")

Save predictions

To save your predictions, use the .save_predictions() method. e.g. to save your predictions on the “infer” dataset:

my_classifier.save_predictions(set_name="infer")

Add predictions to metadata and save

To add your predictions to your patch metadata (saved in patch_df.csv), you will need to load your predictions as metadata in the MapImages object.

To do this, you will need to create a new MapImages object and load in your patches and parent images:

from mapreader import load_patches

my_maps = load_patches(patch_paths = "./path/to/patches/*png", parent_paths="./path/to/parents/*png")

You can then add your predictions to the metadata using the .add_metadata() method:

my_maps.add_metadata("path_to_predictions_patch_df.csv", tree_level='patch') # add dataframe as metadata

For example, to load the predictions for the “infer” dataset:

#EXAMPLE
my_maps.add_metadata("./infer_predictions_patch_df.csv", tree_level='patch')

From here, you can use the .show_parent() method to visualize your predictions on the parent images as shown in the Load user guide:

my_maps.add_shape()

parent_list = my_maps.list_parents()
my_maps.show_parent(parent_list[0], column_to_plot="conf", vmin=0, vmax=1, alpha=0.5, patch_border=False)

Refer to the Load user guidance for further details on how these methods work.