mapreader.classify.classifier

Classes

ClassifierContainer

A class to store and train a PyTorch model.

Module Contents

class mapreader.classify.classifier.ClassifierContainer(model=None, labels_map=None, dataloaders=None, device='default', input_size=(224, 224), is_inception=False, load_path=None, force_device=False, huggingface=False, **kwargs)

A class to store and train a PyTorch model.

Parameters:

  • model (str, nn.Module or None) –

    The PyTorch model to add to the object.

  • labels_map (Dict or None) – A dictionary containing the mapping of each label index to its label, with indices as keys and labels as values (i.e. idx: label). Can only be None if load_path is specified as labels_map will be loaded from file.

  • dataloaders (Dict or None) – A dictionary containing set names as keys and dataloaders as values (i.e. set_name: dataloader).

  • device (str, optional) – The device to be used for training and storing models. Can be set to “default”, “cpu”, “cuda:0”, etc. By default, “default”.

  • input_size (int, optional) – The expected input size of the model. Default is (224,224).

  • is_inception (bool, optional) – Whether the model is an Inception-style model. Default is False.

  • load_path (str, optional) – The path to an .obj file containing a

  • force_device (bool, optional) – Whether to force the use of a specific device. If set to True, the default device is used. Defaults to False.

  • kwargs (Dict) – Keyword arguments to pass to the _initialize_model() method (if passing model as a string).

  • huggingface (bool)

device

The device being used for training and storing models.

Type:

torch.device

dataloaders

A dictionary to store dataloaders for the model.

Type:

dict

labels_map

A dictionary mapping label indices to their labels.

Type:

dict

dataset_sizes

A dictionary to store sizes of datasets for the model.

Type:

dict

model

The model.

Type:

torch.nn.Module

input_size

The size of the input to the model.

Type:

Tuple of int

is_inception

A flag indicating if the model is an Inception model.

Type:

bool

optimizer

The optimizer being used for training the model.

Type:

None or torch.optim.Optimizer

scheduler

The learning rate scheduler being used for training the model.

Type:

None or torch.optim.lr_scheduler._LRScheduler

loss_fn

The loss function to use for training the model.

Type:

None or nn.modules.loss._Loss

metrics

A dictionary to store the metrics computed during training.

Type:

dict

last_epoch

The last epoch number completed during training.

Type:

int

best_loss

The best validation loss achieved during training.

Type:

torch.Tensor

best_epoch

The epoch in which the best validation loss was achieved during training.

Type:

int

tmp_save_filename

A temporary file name to save checkpoints during training and validation.

Type:

str

generate_layerwise_lrs(min_lr, max_lr, spacing='linspace')

Calculates layer-wise learning rates for a given set of model parameters.

Parameters:

  • min_lr (float) – The minimum learning rate to be used.

  • max_lr (float) – The maximum learning rate to be used.

  • spacing (str, optional) – The type of sequence to use for spacing the specified interval learning rates. Can be either "linspace" or "geomspace", where “linspace” uses evenly spaced learning rates over a specified interval and “geomspace” uses learning rates spaced evenly on a log scale (a geometric progression). By default "linspace".

Returns:

A list of dictionaries containing the parameters and learning rates for each layer.

Return type:

list of dicts

initialize_optimizer(optim_type='adam', params2optimize='default', optim_param_dict=None)

Initializes an optimizer for the model and adds it to the classifier object.

Parameters:

  • optim_type (str, optional) – The type of optimizer to use. Can be set to "adam" (default), "adamw", or "sgd".

  • params2optimize (str or iterable, optional) – The parameters to optimize. If set to "default", all model parameters that require gradients will be optimized. Default is "default".

  • optim_param_dict (dict, optional) – The parameters to pass to the optimizer constructor as a dictionary, by default {"lr": 1e-3}.

Notes

Note that the first argument of an optimizer is parameters to optimize, e.g. params2optimize = model_ft.parameters():

  • model_ft.parameters(): all parameters are being optimized

  • model_ft.fc.parameters(): only parameters of final layer are being optimized

Here, we use:

filter(lambda p: p.requires_grad, self.model.parameters())
add_optimizer(optimizer)

Add an optimizer to the classifier object.

Parameters:

optimizer (torch.optim.Optimizer) – The optimizer to add to the classifier object.

Return type:

None

initialize_scheduler(scheduler_type='steplr', scheduler_param_dict=None)

Initializes a learning rate scheduler for the optimizer and adds it to the classifier object. Only StepLR is implemented - otherwise use torch.optim.lr_scheduler directly and the add_scheduler method.

Parameters:

  • scheduler_type (str, optional) – The type of learning rate scheduler to use. Default is "steplr".

  • scheduler_param_dict (dict, optional) – The parameters to pass to the scheduler constructor, by default {"step_size": 10, "gamma": 0.1}.

Raises:

ValueError – If the specified scheduler_type is not implemented.

add_scheduler(scheduler)

Add a scheduler to the classifier object. Note that during training, scheduler.step() is called after each epoch - i.e. do not use schedulers that should be stepped after each batch!

Parameters:

scheduler (torch.optim.lr_scheduler._LRScheduler) – The scheduler to add to the classifier object.

Raises:

ValueError – If no optimizer has been set. Use initialize_optimizer or add_optimizer to set an optimizer first.

Return type:

None

add_loss_fn(loss_fn='cross entropy')

Add a loss function to the classifier object.

Parameters:

loss_fn (str or torch.nn.modules.loss._Loss) – The loss function to add to the classifier object. Accepted string values are “cross entropy” or “ce” (cross-entropy), “bce” (binary cross-entropy) and “mse” (mean squared error).

Returns:

The function only modifies the loss_fn attribute of the classifier and does not return anything.

Return type:

None

model_summary(input_size=None, trainable_col=False, **kwargs)

Print a summary of the model.

Parameters:

  • input_size (tuple or list, optional) – The size of the input data. If None, input size is taken from “train” dataloader (self.dataloaders["train"]).

  • trainable_col (bool, optional) – If True, adds a column showing which parameters are trainable. Defaults to False.

  • **kwargs (Dict) – Keyword arguments to pass to torchinfo.summary() (see https://github.com/TylerYep/torchinfo).

Return type:

None

Notes

Other ways to check params:

sum(p.numel() for p in myclassifier.model.parameters())

sum(p.numel() for p in myclassifier.model.parameters()
    if p.requires_grad)

And:

for name, param in self.model.named_parameters():
    n = name.split(".")[0].split("_")[0]
    print(name, param.requires_grad)
freeze_layers(layers_to_freeze)

Freezes the specified layers in the neural network by setting requires_grad attribute to False for their parameters.

Parameters:

layers_to_freeze (list of str) – List of names of the layers to freeze. If a layer name ends with an asterisk ("*"), then all parameters whose name contains the layer name (excluding the asterisk) are frozen. Otherwise, only the parameters with an exact match to the layer name are frozen.

Returns:

The function only modifies the requires_grad attribute of the specified parameters and does not return anything.

Return type:

None

Notes

e.g. [“layer1*”, “layer2*”] will freeze all parameters whose name contains “layer1” and “layer2” (excluding the asterisk). e.g. [“layer1”, “layer2”] will freeze all parameters with an exact match to “layer1” and “layer2”.

unfreeze_layers(layers_to_unfreeze)

Unfreezes the specified layers in the neural network by setting requires_grad attribute to True for their parameters.

Parameters:

layers_to_unfreeze (list of str) – List of names of the layers to unfreeze. If a layer name ends with an asterisk ("*"), then all parameters whose name contains the layer name (excluding the asterisk) are unfrozen. Otherwise, only the parameters with an exact match to the layer name are unfrozen.

Returns:

The function only modifies the requires_grad attribute of the specified parameters and does not return anything.

Return type:

None

Notes

e.g. [“layer1*”, “layer2*”] will unfreeze all parameters whose name contains “layer1” and “layer2” (excluding the asterisk). e.g. [“layer1”, “layer2”] will unfreeze all parameters with an exact match to “layer1” and “layer2”.

only_keep_layers(only_keep_layers_list)

Only keep the specified layers (only_keep_layers_list) for gradient computation during the backpropagation.

Parameters:

only_keep_layers_list (list) – List of layer names to keep. All other layers will have their gradient computation turned off.

Returns:

The function only modifies the requires_grad attribute of the specified parameters and does not return anything.

Return type:

None

inference(set_name='infer', verbose=False, print_info_batch_freq=5)

Run inference on a specified dataset (set_name).

Parameters:

  • set_name (str, optional) – The name of the dataset to run inference on, by default "infer".

  • verbose (bool, optional) – Whether to print verbose outputs, by default False.

  • print_info_batch_freq (int, optional) – The frequency of printouts, by default 5.

Return type:

None

Notes

This method calls the train() method with the num_epochs set to 1 and all the other parameters specified in the function arguments.

train_component_summary()

Print a summary of the optimizer, loss function, and trainable model components.

Returns:

None

Return type:

None

train(phases=None, num_epochs=25, save_model_dir='models', verbose=False, tensorboard_path=None, tmp_file_save_freq=2, remove_after_load=True, print_info_batch_freq=5)

Train the model on the specified phases for a given number of epochs.

Wrapper function for train_core() method to capture exceptions (KeyboardInterrupt is the only supported exception currently).

Parameters:

  • phases (list of str, optional) – The phases to run through during each training iteration. Default is ["train", "val"].

  • num_epochs (int, optional) – The number of epochs to train the model for. Default is 25.

  • save_model_dir (str or None, optional) – The directory to save the model in. Default is "models". If set to None, the model is not saved.

  • verbose (int, optional) – Whether to print verbose outputs, by default False.

  • tensorboard_path (str or None, optional) – The path to the directory to save TensorBoard logs in. If set to None, no TensorBoard logs are saved. Default is None.

  • tmp_file_save_freq (int, optional) – The frequency (in epochs) to save a temporary file of the model. Default is 2. If set to 0 or None, no temporary file is saved.

  • remove_after_load (bool, optional) – Whether to remove the temporary file after loading it. Default is True.

  • print_info_batch_freq (int, optional) – The frequency (in batches) to print training information. Default is 5. If set to 0 or None, no training information is printed.

Returns:

The function saves the model to the save_model_dir directory, and optionally to a temporary file. If interrupted with a KeyboardInterrupt, the function tries to load the temporary file. If no temporary file is found, it continues without loading.

Return type:

None

Notes

Refer to the documentation of train_core() for more information.

train_core(phases=None, num_epochs=25, save_model_dir='models', verbose=False, tensorboard_path=None, tmp_file_save_freq=2, print_info_batch_freq=5)

Trains/fine-tunes a classifier for the specified number of epochs on the given phases using the specified hyperparameters.

Parameters:

  • phases (list of str, optional) – The phases to run through during each training iteration. Default is ["train", "val"].

  • num_epochs (int, optional) – The number of epochs to train the model for. Default is 25.

  • save_model_dir (str or None, optional) – The directory to save the model in. Default is "models". If set to None, the model is not saved.

  • verbose (bool, optional) – Whether to print verbose outputs, by default False.

  • tensorboard_path (str or None, optional) – The path to the directory to save TensorBoard logs in. If set to None, no TensorBoard logs are saved. Default is None.

  • tmp_file_save_freq (int, optional) – The frequency (in epochs) to save a temporary file of the model. Default is 2. If set to 0 or None, no temporary file is saved.

  • print_info_batch_freq (int, optional) – The frequency (in batches) to print training information. Default is 5. If set to 0 or None, no training information is printed.

Raises:

  • ValueError –

    If the loss function is not set. Use the add_loss_fn() method to set the loss function.

    If the optimizer is not set and the phase is “train”. Use the initialize_optimizer() or add_optimizer() method to set the optimizer.

  • KeyError – If the specified phase cannot be found in the keys of the object’s dataloaders dictionary property.

Return type:

None

calculate_add_metrics(y_true, y_pred, y_score, phase, epoch=-1, tboard_writer=None)

Calculate and add metrics to the classifier’s metrics dictionary.

Parameters:

  • y_true (1d array-like of shape (n_samples,)) – True binary labels or multiclass labels. Can be considered ground truth or (correct) target values.

  • y_pred (1d array-like of shape (n_samples,)) – Predicted binary labels or multiclass labels. The estimated targets as returned by a classifier.

  • y_score (array-like of shape (n_samples, n_classes)) – Predicted probabilities for each class.

  • phase (str) – Name of the current phase, typically "train" or "val". See train function.

  • epoch (int, optional) – Current epoch number. Default is -1.

  • tboard_writer (object, optional) – TensorBoard SummaryWriter object to write the metrics. Default is None.

Return type:

None

Notes

This method uses both the sklearn.metrics.precision_recall_fscore_support() and sklearn.metrics.roc_auc_score() functions from scikit-learn to calculate the metrics for each average type ("micro", "macro" and "weighted"). The results are then added to the metrics dictionary. It also writes the metrics to the TensorBoard SummaryWriter, if tboard_writer is not None.

list_metrics(phases='all')

Prints the available metrics for the specified phases.

Parameters:

phases (str | list[str], optional) – The phases to find metrics for, by default “all”

Return type:

None

plot_metric(metrics, phases='all', colors=None, figsize=(10, 5), plt_yrange=None, plt_xrange=None)

Plot the metrics of the classifier object.

Parameters:

  • metrics (str or list of str) – A string of list of strings containing metric names to be plotted on the y-axis.

  • phases (str or list of str, optional) – The phases for which the metric is to be plotted. Defaults to "all".

  • colors (list of str, optional) – Colors to be used for the lines of each metric. Length must be at least the length of the number of phases being plotted (phases). If None, will use the default matplotlib colors. Defaults to None.

  • figsize (tuple of int, optional) – The size of the figure in inches. Defaults to (10, 5).

  • plt_yrange (tuple of float, optional) – The range of values for the y-axis. Defaults to None.

  • plt_xrange (tuple of float, optional) – The range of values for the x-axis. Defaults to None.

print_batch_info(set_name='train')

Print information about a dataset’s batches, samples, and batch-size.

Parameters:

set_name (str, optional) – Name of the dataset to display batch information for (default is "train").

Return type:

None

show_inference_sample_results(label, num_samples=6, set_name='test', min_conf=None, max_conf=None, figsize=(15, 15))

Shows a sample of the results of the inference with current model.

Parameters:

  • label (str, optional) – The label for which to display results.

  • num_samples (int, optional) – The number of sample results to display. Defaults to 6.

  • set_name (str, optional) – The name of the dataset split to use for inference. Defaults to "test".

  • min_conf (float, optional) – The minimum confidence score for a sample result to be displayed. Samples with lower confidence scores will be skipped. Defaults to None.

  • max_conf (float, optional) – The maximum confidence score for a sample result to be displayed. Samples with higher confidence scores will be skipped. Defaults to None.

  • figsize (tuple[int, int], optional) – Figure size (width, height) in inches, displaying the sample results. Defaults to (15, 15).

Return type:

None

save(save_path='default.obj', force=False)

Save the object to a file.

Parameters:

  • save_path (str, optional) – The path to the file to write. If the file already exists and force is not True, a FileExistsError is raised. Defaults to "default.obj".

  • force (bool, optional) – Whether to overwrite the file if it already exists. Defaults to False.

Raises:

FileExistsError – If the file already exists and force is not True.

Return type:

None

Notes

The object is saved in two parts. First, a serialized copy of the object’s dictionary is written to the specified file using the joblib.dump function. The object’s model attribute is excluded from this dictionary and saved separately using the torch.save function, with a filename derived from the original save_path.

save_predictions(set_name, save_path=None, delimiter=',')
Parameters:

  • set_name (str)

  • save_path (str | None)

  • delimiter (str)

load_dataset(dataset, set_name, batch_size=16, sampler=None, shuffle=False, num_workers=0, **kwargs)

Creates a DataLoader from a PatchDataset and adds it to the dataloaders dictionary.

Parameters:

  • dataset (PatchDataset) – The dataset to add

  • set_name (str) – The name to use for the dataset

  • batch_size (Optional[int], optional) – The batch size to use when creating the DataLoader, by default 16

  • sampler (Optional[Union[Sampler, None]], optional) – The sampler to use when creating the DataLoader, by default None

  • shuffle (Optional[bool], optional) – Whether to shuffle the PatchDataset, by default False

  • num_workers (Optional[int], optional) – The number of worker threads to use for loading data, by default 0.

Return type:

None

load(load_path, force_device=False)

This function loads the state of a class instance from a saved file using the joblib library. It also loads a PyTorch model from a separate file and maps it to the device used to load the class instance.

Parameters:

  • load_path (str) – Path to the saved file to load.

  • force_device (bool or str, optional) – Whether to force the use of a specific device, or the name of the device to use. If set to True, the default device is used. Defaults to False.

Raises:

FileNotFoundError – If the specified file does not exist.

Return type:

None

cprint(type_info, bc_color, text)

Print colored text with additional information.

Parameters:

  • type_info (str) – The type of message to display.

  • bc_color (str) – The color to use for the message text.

  • text (str) – The text to display.

Returns:

The colored message is displayed on the standard output stream.

Return type:

None

update_progress(progress, text='', barLength=30)

Update the progress bar.

Parameters:

  • progress (float or int) – The progress value to display, between 0 and 1. If an integer is provided, it will be converted to a float. If a value outside the range [0, 1] is provided, it will be clamped to the nearest valid value.

  • text (str, optional) – Additional text to display after the progress bar, defaults to "".

  • barLength (int, optional) – The length of the progress bar in characters, defaults to 30.

Raises:

TypeError – If progress is not a floating point value or an integer.

Returns:

The progress bar is displayed on the standard output stream.

Return type:

None