Classification of plant phenotype

This is an example of a non-geospatial use of the MapReader pipeline

Load

https://mapreader.readthedocs.io/en/latest/User-guide/Load.html

Load images

[37]:

from mapreader import loader

path2images = "./dataset/open_access_plant/*.png"
my_files = loader(path2images)

100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 77.27it/s]

[38]:

# len() shows the total number of images currently read (or sliced, see below)
print(f"Number of images: {len(my_files)}")

Number of images: 2

[39]:

print(my_files)

#images: 2

#parents: 2
2014-06-06_plant001_rgb.png
2014-07-17_plant047_rgb.png

#patches: 0

[40]:

my_files.show_sample(num_samples=2, tree_level='parent')

../_images/Worked-examples_plant_pipeline_6_0.png

Patchify images

[41]:

my_files.patchify_all(patch_size=50, # in pixels
                square_cuts=False)

[INFO] Saving patches in directory named "patches_50_pixel".

100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 30.68it/s]

[42]:

my_files.show_sample(num_samples=6, tree_level="patch")

../_images/Worked-examples_plant_pipeline_9_0.png

[43]:

parent_list=my_files.list_parents()
patch_list=my_files.list_patches()

[44]:

my_files.show_parent(parent_list[0],
                patch_border=True,
                plot_parent=True,
                figsize=(15, 15))

100%|█████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.06it/s]

[44]:

[<Figure size 1500x1500 with 1 Axes>]

../_images/Worked-examples_plant_pipeline_11_2.png

Calculate pixel intensities

[45]:

my_files.calc_pixel_stats()

100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 15.66it/s]

[46]:

parent_df, patch_df = my_files.convert_images(save=True)

[INFO] Saved parent dataframe as "parent_df.csv"
[INFO] Saved patch dataframe as "patch_df.csv"

[47]:

parent_df.head()

[47]:

	parent_id	image_path	shape	patches
image_id
2014-06-06_plant001_rgb.png	None	/Users/rwood/LwM/MapReader/worked_examples/non...	(530, 500, 4)	[patch-0-0-50-50-#2014-06-06_plant001_rgb.png#...
2014-07-17_plant047_rgb.png	None	/Users/rwood/LwM/MapReader/worked_examples/non...	(530, 500, 4)	[patch-0-0-50-50-#2014-07-17_plant047_rgb.png#...

[48]:

patch_df.head()

[48]:

	parent_id	image_path	shape	pixel_bounds	mean_pixel_R	mean_pixel_G	mean_pixel_B	mean_pixel_A	std_pixel_R	std_pixel_G	std_pixel_B	std_pixel_A
image_id
patch-0-0-50-50-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 0, 50, 50)	0.478751	0.453997	0.447296	1.0	0.046421	0.024224	0.040283	0.0
patch-0-50-50-100-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 50, 50, 100)	0.811368	0.413255	0.238436	1.0	0.131187	0.100754	0.112988	0.0
patch-0-100-50-150-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 100, 50, 150)	0.682626	0.314795	0.156773	1.0	0.085176	0.064740	0.082339	0.0
patch-0-150-50-200-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 150, 50, 200)	0.630331	0.300660	0.159903	1.0	0.083151	0.062500	0.083706	0.0
patch-0-200-50-250-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 200, 50, 250)	0.605075	0.351104	0.238678	1.0	0.056804	0.056446	0.065008	0.0

[49]:

patch_df.mean()

[49]:

mean_pixel_R    0.512725
mean_pixel_G    0.388799
mean_pixel_B    0.208943
mean_pixel_A    1.000000
std_pixel_R     0.093617
std_pixel_G     0.084910
std_pixel_B     0.070554
std_pixel_A     0.000000
dtype: float64

[ ]:

my_files.show_parent(parent_list[0] ,
                column_to_plot="mean_pixel_G",
                patch_border=False,
                plot_parent=True,
                vmin=0.1,
                vmax=0.6,
                figsize=(20, 20),
                alpha=0.7)

100%|█████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 24.86it/s]

[<Figure size 2000x2000 with 2 Axes>]

Annotate

https://mapreader.readthedocs.io/en/latest/User-guide/Annotate.html

Remember to first edit ./annotation_tasks.yaml file! It should look something like this:

# ---------------------------------------
# Define an annotation task
# This includes:
# 1. a name (e.g., building_simple or rail_space, see below)
# 2. a list of labels to be used for this task
# ---------------------------------------
tasks:
  phenotype_test:
    labels: ["No", "Plant"]

# ---------------------------------------
# paths
# You need to specify:
# 1. a name (e.g., task_test_one_inch_maps_001, see below)
# 2. patch_paths: path to all the patches to be annotated
# 3. parent_paths: path to the original/parent maps/images (which were patchified)
# 4. annot_dir: directory in which the outputs will be stored
# ---------------------------------------
paths:
  task_test_phenotype:
    patch_paths: "./patches_50_pixel/*png"
    parent_paths: "./dataset/open_access_plant/*png"
    annot_dir: "./annotations_phenotype_open_access"

Set up inputs

[37]:

from mapreader.annotate.utils import prepare_annotation, save_annotation

[38]:

userID = "kasra"
annotation_tasks_file = "./annotation_tasks_open.yaml"
task = "phenotype_test"
annotation_set = "task_test_phenotype"

# sortby="mean" sorts the patches according to the mean pixel intensities
# xoffset and yoffset specify the border size around a patch to be used as the context image (in pixel)
annotate = prepare_annotation(userID,
                            task,
                            annotation_tasks_file=annotation_tasks_file,
                            annotation_set=annotation_set,
                            sortby='mean',
                            min_alpha_channel=0.01,
                            xoffset=50,
                            yoffset=50,
                            context_image=True)

0it [00:00, ?it/s]
100%|███████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 1795.12it/s]
100%|██████████████████████████████████████████████████████████████████████| 220/220 [00:00<00:00, 20526.01it/s]
100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 32.61it/s]

Number of already annotated images: 124
Number of images to be annotated (total): 96
[INFO] len(df) = 96, .sample method is deactivated.
Number of images to annotate (current batch): 96

Annotate images and save annotations

[39]:

annotate

[40]:

save_annotation(annotate,
                userID,
                task,
                annotation_tasks_file=annotation_tasks_file,
                annotation_set=annotation_set)

[INFO] Save 15 new annotations to ./annotations_phenotype_open_access/phenotype_test_#kasra#.csv
[INFO] 15 labels were not already stored
[INFO] Total number of saved annotations: 139

Classify - Train

https://mapreader.readthedocs.io/en/latest/User-guide/Classify.html

Read in annotations

[1]:

from mapreader import AnnotationsLoader

[2]:

annotated_images = AnnotationsLoader()

annotated_images.load("./annotations_phenotype_open_access/phenotype_test_#kasra#.csv")

[INFO] Reading "./annotations_phenotype_open_access/phenotype_test_#kasra#.csv"
[INFO] Number of annotations:   139

[INFO] Number of instances of each label (from column "label"):
        - 1:      92
        - 2:      47

[3]:

annotated_images.annotations

[3]:

	image_id	label	image_path	label_index
0	patch-300-350-350-400-#2014-06-06_plant001_rgb...	1	/Users/rwood/LwM/MapReader/worked_examples/non...	0
1	patch-250-400-300-450-#2014-06-06_plant001_rgb...	1	/Users/rwood/LwM/MapReader/worked_examples/non...	0
2	patch-300-250-350-300-#2014-06-06_plant001_rgb...	2	/Users/rwood/LwM/MapReader/worked_examples/non...	1
3	patch-200-400-250-450-#2014-06-06_plant001_rgb...	1	/Users/rwood/LwM/MapReader/worked_examples/non...	0
4	patch-300-150-350-200-#2014-06-06_plant001_rgb...	1	/Users/rwood/LwM/MapReader/worked_examples/non...	0
...	...	...	...	...
134	patch-150-300-200-350-#2014-07-17_plant047_rgb...	2	/Users/rwood/LwM/MapReader/worked_examples/non...	1
135	patch-350-300-400-350-#2014-07-17_plant047_rgb...	2	/Users/rwood/LwM/MapReader/worked_examples/non...	1
136	patch-200-300-250-350-#2014-06-06_plant001_rgb...	2	/Users/rwood/LwM/MapReader/worked_examples/non...	1
137	patch-200-250-250-300-#2014-07-17_plant047_rgb...	2	/Users/rwood/LwM/MapReader/worked_examples/non...	1
138	patch-400-250-450-300-#2014-06-06_plant001_rgb...	1	/Users/rwood/LwM/MapReader/worked_examples/non...	0

139 rows × 4 columns

[4]:

print(annotated_images)

[INFO] Number of annotations:   139

[INFO] Number of instances of each label (from column "label"):
        - 1:      92
        - 2:      47

[5]:

annotated_images.annotations["image_id"][5]

[5]:

'patch-300-300-350-350-#2014-06-06_plant001_rgb.png#.png'

[6]:

# show sample images for one labe (label_to_show)
annotated_images.show_sample(label_to_show="2")

../_images/Worked-examples_plant_pipeline_34_0.png

[7]:

# show an image based on its index
annotated_images.show_patch("patch-300-300-350-350-#2014-06-06_plant001_rgb.png#.png")

../_images/Worked-examples_plant_pipeline_35_0.png

[8]:

annotated_images.review_labels()

[INFO] Type "exit", "end" or "stop" to exit.
[INFO] Showing 0-24 out of 139.

../_images/Worked-examples_plant_pipeline_36_1.png

[INFO] IDs of current patches: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]

Enter IDs, comma separated (or press enter to continue): exit
[INFO] Exited.

Prepare datasets and dataloaders

[15]:

annotated_images.create_datasets(frac_train=0.7, frac_val=0.2, frac_test=0.1)

[INFO] Number of annotations in each set:
        - Train:        97
        - Validate:     28
        - Test:         14

[16]:

annotated_images.labels_map

[16]:

{0: '1', 1: '2'}

[17]:

dataloaders = annotated_images.create_dataloaders(batch_size=16)

[INFO] Using default sampler.

Set up `ClassifierContainer`

Load a pretrained PyTorch model, dataloaders and labels map

[18]:

from mapreader import ClassifierContainer

[19]:

my_classifier = ClassifierContainer(model ="resnet18",
                                    dataloaders=dataloaders,
                                    labels_map={0: 'No', 1: 'plant'})

[INFO] Device is set to cpu
[INFO] Loaded "train" with 97 items.
[INFO] Loaded "val" with 28 items.
[INFO] Loaded "test" with 14 items.
[INFO] Initializing model.

[20]:

my_classifier.labels_map

[20]:

{0: 'No', 1: 'plant'}

[21]:

#show sample of transformed images
my_classifier.show_sample(set_name='train', batch_number=1)

[INFO] dataset: train
        - items:        97
        - batch size:   16
        - batches:      7

../_images/Worked-examples_plant_pipeline_46_1.png

[22]:

my_classifier.model_summary()

===================================================================================================================
Layer (type:depth-idx)                   Output Shape              Output Shape              Param #
===================================================================================================================
ResNet                                   [16, 2]                   [16, 2]                   --
├─Conv2d: 1-1                            [16, 64, 112, 112]        [16, 64, 112, 112]        9,408
├─BatchNorm2d: 1-2                       [16, 64, 112, 112]        [16, 64, 112, 112]        128
├─ReLU: 1-3                              [16, 64, 112, 112]        [16, 64, 112, 112]        --
├─MaxPool2d: 1-4                         [16, 64, 56, 56]          [16, 64, 56, 56]          --
├─Sequential: 1-5                        [16, 64, 56, 56]          [16, 64, 56, 56]          --
│    └─BasicBlock: 2-1                   [16, 64, 56, 56]          [16, 64, 56, 56]          --
│    │    └─Conv2d: 3-1                  [16, 64, 56, 56]          [16, 64, 56, 56]          36,864
│    │    └─BatchNorm2d: 3-2             [16, 64, 56, 56]          [16, 64, 56, 56]          128
│    │    └─ReLU: 3-3                    [16, 64, 56, 56]          [16, 64, 56, 56]          --
│    │    └─Conv2d: 3-4                  [16, 64, 56, 56]          [16, 64, 56, 56]          36,864
│    │    └─BatchNorm2d: 3-5             [16, 64, 56, 56]          [16, 64, 56, 56]          128
│    │    └─ReLU: 3-6                    [16, 64, 56, 56]          [16, 64, 56, 56]          --
│    └─BasicBlock: 2-2                   [16, 64, 56, 56]          [16, 64, 56, 56]          --
│    │    └─Conv2d: 3-7                  [16, 64, 56, 56]          [16, 64, 56, 56]          36,864
│    │    └─BatchNorm2d: 3-8             [16, 64, 56, 56]          [16, 64, 56, 56]          128
│    │    └─ReLU: 3-9                    [16, 64, 56, 56]          [16, 64, 56, 56]          --
│    │    └─Conv2d: 3-10                 [16, 64, 56, 56]          [16, 64, 56, 56]          36,864
│    │    └─BatchNorm2d: 3-11            [16, 64, 56, 56]          [16, 64, 56, 56]          128
│    │    └─ReLU: 3-12                   [16, 64, 56, 56]          [16, 64, 56, 56]          --
├─Sequential: 1-6                        [16, 128, 28, 28]         [16, 128, 28, 28]         --
│    └─BasicBlock: 2-3                   [16, 128, 28, 28]         [16, 128, 28, 28]         --
│    │    └─Conv2d: 3-13                 [16, 128, 28, 28]         [16, 128, 28, 28]         73,728
│    │    └─BatchNorm2d: 3-14            [16, 128, 28, 28]         [16, 128, 28, 28]         256
│    │    └─ReLU: 3-15                   [16, 128, 28, 28]         [16, 128, 28, 28]         --
│    │    └─Conv2d: 3-16                 [16, 128, 28, 28]         [16, 128, 28, 28]         147,456
│    │    └─BatchNorm2d: 3-17            [16, 128, 28, 28]         [16, 128, 28, 28]         256
│    │    └─Sequential: 3-18             [16, 128, 28, 28]         [16, 128, 28, 28]         8,448
│    │    └─ReLU: 3-19                   [16, 128, 28, 28]         [16, 128, 28, 28]         --
│    └─BasicBlock: 2-4                   [16, 128, 28, 28]         [16, 128, 28, 28]         --
│    │    └─Conv2d: 3-20                 [16, 128, 28, 28]         [16, 128, 28, 28]         147,456
│    │    └─BatchNorm2d: 3-21            [16, 128, 28, 28]         [16, 128, 28, 28]         256
│    │    └─ReLU: 3-22                   [16, 128, 28, 28]         [16, 128, 28, 28]         --
│    │    └─Conv2d: 3-23                 [16, 128, 28, 28]         [16, 128, 28, 28]         147,456
│    │    └─BatchNorm2d: 3-24            [16, 128, 28, 28]         [16, 128, 28, 28]         256
│    │    └─ReLU: 3-25                   [16, 128, 28, 28]         [16, 128, 28, 28]         --
├─Sequential: 1-7                        [16, 256, 14, 14]         [16, 256, 14, 14]         --
│    └─BasicBlock: 2-5                   [16, 256, 14, 14]         [16, 256, 14, 14]         --
│    │    └─Conv2d: 3-26                 [16, 256, 14, 14]         [16, 256, 14, 14]         294,912
│    │    └─BatchNorm2d: 3-27            [16, 256, 14, 14]         [16, 256, 14, 14]         512
│    │    └─ReLU: 3-28                   [16, 256, 14, 14]         [16, 256, 14, 14]         --
│    │    └─Conv2d: 3-29                 [16, 256, 14, 14]         [16, 256, 14, 14]         589,824
│    │    └─BatchNorm2d: 3-30            [16, 256, 14, 14]         [16, 256, 14, 14]         512
│    │    └─Sequential: 3-31             [16, 256, 14, 14]         [16, 256, 14, 14]         33,280
│    │    └─ReLU: 3-32                   [16, 256, 14, 14]         [16, 256, 14, 14]         --
│    └─BasicBlock: 2-6                   [16, 256, 14, 14]         [16, 256, 14, 14]         --
│    │    └─Conv2d: 3-33                 [16, 256, 14, 14]         [16, 256, 14, 14]         589,824
│    │    └─BatchNorm2d: 3-34            [16, 256, 14, 14]         [16, 256, 14, 14]         512
│    │    └─ReLU: 3-35                   [16, 256, 14, 14]         [16, 256, 14, 14]         --
│    │    └─Conv2d: 3-36                 [16, 256, 14, 14]         [16, 256, 14, 14]         589,824
│    │    └─BatchNorm2d: 3-37            [16, 256, 14, 14]         [16, 256, 14, 14]         512
│    │    └─ReLU: 3-38                   [16, 256, 14, 14]         [16, 256, 14, 14]         --
├─Sequential: 1-8                        [16, 512, 7, 7]           [16, 512, 7, 7]           --
│    └─BasicBlock: 2-7                   [16, 512, 7, 7]           [16, 512, 7, 7]           --
│    │    └─Conv2d: 3-39                 [16, 512, 7, 7]           [16, 512, 7, 7]           1,179,648
│    │    └─BatchNorm2d: 3-40            [16, 512, 7, 7]           [16, 512, 7, 7]           1,024
│    │    └─ReLU: 3-41                   [16, 512, 7, 7]           [16, 512, 7, 7]           --
│    │    └─Conv2d: 3-42                 [16, 512, 7, 7]           [16, 512, 7, 7]           2,359,296
│    │    └─BatchNorm2d: 3-43            [16, 512, 7, 7]           [16, 512, 7, 7]           1,024
│    │    └─Sequential: 3-44             [16, 512, 7, 7]           [16, 512, 7, 7]           132,096
│    │    └─ReLU: 3-45                   [16, 512, 7, 7]           [16, 512, 7, 7]           --
│    └─BasicBlock: 2-8                   [16, 512, 7, 7]           [16, 512, 7, 7]           --
│    │    └─Conv2d: 3-46                 [16, 512, 7, 7]           [16, 512, 7, 7]           2,359,296
│    │    └─BatchNorm2d: 3-47            [16, 512, 7, 7]           [16, 512, 7, 7]           1,024
│    │    └─ReLU: 3-48                   [16, 512, 7, 7]           [16, 512, 7, 7]           --
│    │    └─Conv2d: 3-49                 [16, 512, 7, 7]           [16, 512, 7, 7]           2,359,296
│    │    └─BatchNorm2d: 3-50            [16, 512, 7, 7]           [16, 512, 7, 7]           1,024
│    │    └─ReLU: 3-51                   [16, 512, 7, 7]           [16, 512, 7, 7]           --
├─AdaptiveAvgPool2d: 1-9                 [16, 512, 1, 1]           [16, 512, 1, 1]           --
├─Linear: 1-10                           [16, 2]                   [16, 2]                   1,026
===================================================================================================================
Total params: 11,177,538
Trainable params: 11,177,538
Non-trainable params: 0
Total mult-adds (G): 29.02
===================================================================================================================
Input size (MB): 9.63
Forward/backward pass size (MB): 635.83
Params size (MB): 44.71
Estimated Total Size (MB): 690.17
===================================================================================================================

Set up criterion, optimizer and scheduler

[28]:

my_classifier.add_criterion()

[INFO] Using "CrossEntropyLoss()" as criterion.

[29]:

my_classifier.initialize_optimizer()

[30]:

my_classifier.initialize_scheduler()

Train/fine-tune a model

[31]:

my_classifier.train(num_epochs=10,
                   tmp_file_save_freq=2,
                   remove_after_load=False,
                   print_info_batch_freq=5)

[INFO] Each training step will pass: ['train', 'val'].
2023-06-05 11:09:08 599-JY5FK6 [INFO] train    -- 1/10 --      16/97 ( 16.5% ) -- Loss: 0.771
2023-06-05 11:09:13 599-JY5FK6 [INFO] train    -- 1/10 --      96/97 ( 99.0% ) -- Loss: 0.939
2023-06-05 11:09:13 599-JY5FK6 [INFO] train    -- 1/10 -- Loss: 0.489; F_macro: 82.41; R_macro: 83.35
2023-06-05 11:09:14 599-JY5FK6 [INFO] val      -- 1/10 --      16/28 ( 57.1% ) -- Loss: 3.502
2023-06-05 11:09:14 599-JY5FK6 [INFO] val      -- 1/10 -- Loss: 2.994; F_macro: 78.12; R_macro: 84.21

2023-06-05 11:09:15 599-JY5FK6 [INFO] train    -- 2/10 --      16/97 ( 16.5% ) -- Loss: 1.931
2023-06-05 11:09:20 599-JY5FK6 [INFO] train    -- 2/10 --      96/97 ( 99.0% ) -- Loss: 0.023
2023-06-05 11:09:20 599-JY5FK6 [INFO] train    -- 2/10 -- Loss: 0.515; F_macro: 90.66; R_macro: 90.74
2023-06-05 11:09:21 599-JY5FK6 [INFO] val      -- 2/10 --      16/28 ( 57.1% ) -- Loss: 12.868
2023-06-05 11:09:21 599-JY5FK6 [INFO] val      -- 2/10 -- Loss: 12.950; F_macro: 24.32; R_macro: 50.00

[INFO] Checkpoint file saved to "./tmp_checkpoints/tmp_4647429002_checkpoint.pkl".
2023-06-05 11:09:22 599-JY5FK6 [INFO] train    -- 3/10 --      16/97 ( 16.5% ) -- Loss: 0.037
2023-06-05 11:09:27 599-JY5FK6 [INFO] train    -- 3/10 --      96/97 ( 99.0% ) -- Loss: 0.157
2023-06-05 11:09:27 599-JY5FK6 [INFO] train    -- 3/10 -- Loss: 0.143; F_macro: 94.84; R_macro: 95.04
2023-06-05 11:09:28 599-JY5FK6 [INFO] val      -- 3/10 --      16/28 ( 57.1% ) -- Loss: 11.105
2023-06-05 11:09:28 599-JY5FK6 [INFO] val      -- 3/10 -- Loss: 11.837; F_macro: 53.03; R_macro: 65.79

2023-06-05 11:09:29 599-JY5FK6 [INFO] train    -- 4/10 --      16/97 ( 16.5% ) -- Loss: 0.514
2023-06-05 11:09:34 599-JY5FK6 [INFO] train    -- 4/10 --      96/97 ( 99.0% ) -- Loss: 0.203
2023-06-05 11:09:34 599-JY5FK6 [INFO] train    -- 4/10 -- Loss: 0.338; F_macro: 90.58; R_macro: 90.48
2023-06-05 11:09:35 599-JY5FK6 [INFO] val      -- 4/10 --      16/28 ( 57.1% ) -- Loss: 0.689
2023-06-05 11:09:35 599-JY5FK6 [INFO] val      -- 4/10 -- Loss: 0.435; F_macro: 92.22; R_macro: 94.74

[INFO] Checkpoint file saved to "./tmp_checkpoints/tmp_4647429002_checkpoint.pkl".
2023-06-05 11:09:36 599-JY5FK6 [INFO] train    -- 5/10 --      16/97 ( 16.5% ) -- Loss: 0.030
2023-06-05 11:09:41 599-JY5FK6 [INFO] train    -- 5/10 --      96/97 ( 99.0% ) -- Loss: 0.114
2023-06-05 11:09:41 599-JY5FK6 [INFO] train    -- 5/10 -- Loss: 0.256; F_macro: 89.56; R_macro: 89.58
2023-06-05 11:09:42 599-JY5FK6 [INFO] val      -- 5/10 --      16/28 ( 57.1% ) -- Loss: 1.025
2023-06-05 11:09:42 599-JY5FK6 [INFO] val      -- 5/10 -- Loss: 0.681; F_macro: 74.71; R_macro: 81.58

2023-06-05 11:09:43 599-JY5FK6 [INFO] train    -- 6/10 --      16/97 ( 16.5% ) -- Loss: 0.040
2023-06-05 11:09:48 599-JY5FK6 [INFO] train    -- 6/10 --      96/97 ( 99.0% ) -- Loss: 0.066
2023-06-05 11:09:48 599-JY5FK6 [INFO] train    -- 6/10 -- Loss: 0.160; F_macro: 93.78; R_macro: 94.21
2023-06-05 11:09:49 599-JY5FK6 [INFO] val      -- 6/10 --      16/28 ( 57.1% ) -- Loss: 0.746
2023-06-05 11:09:49 599-JY5FK6 [INFO] val      -- 6/10 -- Loss: 0.448; F_macro: 88.05; R_macro: 89.18

[INFO] Checkpoint file saved to "./tmp_checkpoints/tmp_4647429002_checkpoint.pkl".
2023-06-05 11:09:50 599-JY5FK6 [INFO] train    -- 7/10 --      16/97 ( 16.5% ) -- Loss: 0.084
2023-06-05 11:09:55 599-JY5FK6 [INFO] train    -- 7/10 --      96/97 ( 99.0% ) -- Loss: 0.034
2023-06-05 11:09:56 599-JY5FK6 [INFO] train    -- 7/10 -- Loss: 0.048; F_macro: 100.00; R_macro: 100.00
2023-06-05 11:09:56 599-JY5FK6 [INFO] val      -- 7/10 --      16/28 ( 57.1% ) -- Loss: 0.501
2023-06-05 11:09:56 599-JY5FK6 [INFO] val      -- 7/10 -- Loss: 0.289; F_macro: 96.02; R_macro: 97.37

2023-06-05 11:09:57 599-JY5FK6 [INFO] train    -- 8/10 --      16/97 ( 16.5% ) -- Loss: 0.118
2023-06-05 11:10:03 599-JY5FK6 [INFO] train    -- 8/10 --      96/97 ( 99.0% ) -- Loss: 0.022
2023-06-05 11:10:03 599-JY5FK6 [INFO] train    -- 8/10 -- Loss: 0.055; F_macro: 97.90; R_macro: 97.90
2023-06-05 11:10:03 599-JY5FK6 [INFO] val      -- 8/10 --      16/28 ( 57.1% ) -- Loss: 0.133
2023-06-05 11:10:03 599-JY5FK6 [INFO] val      -- 8/10 -- Loss: 0.085; F_macro: 96.02; R_macro: 97.37

[INFO] Checkpoint file saved to "./tmp_checkpoints/tmp_4647429002_checkpoint.pkl".
2023-06-05 11:10:05 599-JY5FK6 [INFO] train    -- 9/10 --      16/97 ( 16.5% ) -- Loss: 0.013
2023-06-05 11:10:10 599-JY5FK6 [INFO] train    -- 9/10 --      96/97 ( 99.0% ) -- Loss: 0.009
2023-06-05 11:10:10 599-JY5FK6 [INFO] train    -- 9/10 -- Loss: 0.027; F_macro: 98.97; R_macro: 99.00
2023-06-05 11:10:10 599-JY5FK6 [INFO] val      -- 9/10 --      16/28 ( 57.1% ) -- Loss: 0.044
2023-06-05 11:10:10 599-JY5FK6 [INFO] val      -- 9/10 -- Loss: 0.069; F_macro: 100.00; R_macro: 100.00

2023-06-05 11:10:12 599-JY5FK6 [INFO] train    -- 10/10 --      16/97 ( 16.5% ) -- Loss: 0.009
2023-06-05 11:10:17 599-JY5FK6 [INFO] train    -- 10/10 --      96/97 ( 99.0% ) -- Loss: 0.341
2023-06-05 11:10:17 599-JY5FK6 [INFO] train    -- 10/10 -- Loss: 0.097; F_macro: 96.90; R_macro: 97.17
2023-06-05 11:10:17 599-JY5FK6 [INFO] val      -- 10/10 --      16/28 ( 57.1% ) -- Loss: 0.028
2023-06-05 11:10:18 599-JY5FK6 [INFO] val      -- 10/10 -- Loss: 0.034; F_macro: 100.00; R_macro: 100.00

[INFO] Checkpoint file saved to "./tmp_checkpoints/tmp_4647429002_checkpoint.pkl".
[INFO] Total time: 1m 11s
[INFO] Model at epoch 10 has least valid loss (0.0335) so will be saved.
[INFO] Path: /Users/rwood/LwM/MapReader/worked_examples/non-geospatial/classification_plant_phenotype/models/checkpoint_10.pkl

[32]:

list(my_classifier.metrics.keys())

[32]:

['epoch_loss_train',
 'epoch_prec_micro_train',
 'epoch_recall_micro_train',
 'epoch_fscore_micro_train',
 'epoch_supp_micro_train',
 'epoch_rocauc_micro_train',
 'epoch_prec_macro_train',
 'epoch_recall_macro_train',
 'epoch_fscore_macro_train',
 'epoch_supp_macro_train',
 'epoch_rocauc_macro_train',
 'epoch_prec_weighted_train',
 'epoch_recall_weighted_train',
 'epoch_fscore_weighted_train',
 'epoch_supp_weighted_train',
 'epoch_rocauc_weighted_train',
 'epoch_prec_0_train',
 'epoch_recall_0_train',
 'epoch_fscore_0_train',
 'epoch_supp_0_train',
 'epoch_prec_1_train',
 'epoch_recall_1_train',
 'epoch_fscore_1_train',
 'epoch_supp_1_train',
 'epoch_loss_val',
 'epoch_prec_micro_val',
 'epoch_recall_micro_val',
 'epoch_fscore_micro_val',
 'epoch_supp_micro_val',
 'epoch_rocauc_micro_val',
 'epoch_prec_macro_val',
 'epoch_recall_macro_val',
 'epoch_fscore_macro_val',
 'epoch_supp_macro_val',
 'epoch_rocauc_macro_val',
 'epoch_prec_weighted_val',
 'epoch_recall_weighted_val',
 'epoch_fscore_weighted_val',
 'epoch_supp_weighted_val',
 'epoch_rocauc_weighted_val',
 'epoch_prec_0_val',
 'epoch_recall_0_val',
 'epoch_fscore_0_val',
 'epoch_supp_0_val',
 'epoch_prec_1_val',
 'epoch_recall_1_val',
 'epoch_fscore_1_val',
 'epoch_supp_1_val']

[33]:

my_classifier.plot_metric(y_axis=["epoch_loss_train", "epoch_loss_val"],
                         y_label="Loss",
                         legends=["Train", "Valid"],
                         colors=["k", "tab:red"])

../_images/Worked-examples_plant_pipeline_55_0.png

[34]:

my_classifier.plot_metric(y_axis=["epoch_rocauc_macro_train", "epoch_rocauc_macro_val"],
                         y_label="ROC AUC",
                         legends=["Train", "Valid"],
                         colors=["k", "tab:red"])

../_images/Worked-examples_plant_pipeline_56_0.png

Classify - Infer

https://mapreader.readthedocs.io/en/latest/User-guide/Classify.html

Create dataset with all patches and add to `ClassifierContainer`

[53]:

#create dataframe from MapImages object
parent_df, patch_df = my_files.convert_images()
patch_df.head()

[53]:

	parent_id	image_path	shape	pixel_bounds	mean_pixel_R	mean_pixel_G	mean_pixel_B	mean_pixel_A	std_pixel_R	std_pixel_G	std_pixel_B	std_pixel_A
image_id
patch-0-0-50-50-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 0, 50, 50)	0.478751	0.453997	0.447296	1.0	0.046421	0.024224	0.040283	0.0
patch-0-50-50-100-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 50, 50, 100)	0.811368	0.413255	0.238436	1.0	0.131187	0.100754	0.112988	0.0
patch-0-100-50-150-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 100, 50, 150)	0.682626	0.314795	0.156773	1.0	0.085176	0.064740	0.082339	0.0
patch-0-150-50-200-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 150, 50, 200)	0.630331	0.300660	0.159903	1.0	0.083151	0.062500	0.083706	0.0
patch-0-200-50-250-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 200, 50, 250)	0.605075	0.351104	0.238678	1.0	0.056804	0.056446	0.065008	0.0

[54]:

from mapreader import PatchDataset

[55]:

data = PatchDataset(patch_df, transform="val")

[56]:

my_classifier.load_dataset(data, set_name="all_plants")

Infer labels

[57]:

my_classifier.inference("all_plants")

[INFO] Each training step will pass: ['all_plants'].
2023-06-05 11:10:58 599-JY5FK6 [INFO] all_plants -- 11/11 --     16/220 (  7.3% ) -- 
2023-06-05 11:11:01 599-JY5FK6 [INFO] all_plants -- 11/11 --     96/220 ( 43.6% ) -- 
2023-06-05 11:11:04 599-JY5FK6 [INFO] all_plants -- 11/11 --    176/220 ( 80.0% ) -- 
[INFO] Total time: 0m 7s

[59]:

my_classifier.show_inference_sample_results(label="plant", set_name="all_plants", min_conf=99)

../_images/Worked-examples_plant_pipeline_65_0.png

Add predictions to dataframe

[60]:

predictions_df = data.patch_df

[61]:

import numpy as np
import pandas as pd

predictions_df['predicted_label'] = my_classifier.pred_label
predictions_df['pred'] = my_classifier.pred_label_indices
predictions_df['conf'] = np.array(my_classifier.pred_conf).max(axis=1)

predictions_df.head()

[61]:

	parent_id	image_path	shape	pixel_bounds	mean_pixel_R	mean_pixel_G	mean_pixel_B	mean_pixel_A	std_pixel_R	std_pixel_G	std_pixel_B	std_pixel_A	predicted_label	pred	conf
image_id
patch-0-0-50-50-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 0, 50, 50)	0.478751	0.453997	0.447296	1.0	0.046421	0.024224	0.040283	0.0	No	0	0.999786
patch-0-50-50-100-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 50, 50, 100)	0.811368	0.413255	0.238436	1.0	0.131187	0.100754	0.112988	0.0	No	0	0.957349
patch-0-100-50-150-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 100, 50, 150)	0.682626	0.314795	0.156773	1.0	0.085176	0.064740	0.082339	0.0	No	0	0.957769
patch-0-150-50-200-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 150, 50, 200)	0.630331	0.300660	0.159903	1.0	0.083151	0.062500	0.083706	0.0	No	0	0.971755
patch-0-200-50-250-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 200, 50, 250)	0.605075	0.351104	0.238678	1.0	0.056804	0.056446	0.065008	0.0	No	0	0.996740

[62]:

predictions_df.to_csv("./predictions_df.csv", sep=",", index_label="image_id")

Add outputs to `MapImages` as metadata

[63]:

my_files.add_metadata(predictions_df, tree_level="patch")

[65]:

parent_df, patch_df = my_files.convert_images()
patch_df.head()

[65]:

	parent_id	image_path	shape	pixel_bounds	mean_pixel_R	mean_pixel_G	mean_pixel_B	mean_pixel_A	std_pixel_R	std_pixel_G	std_pixel_B	std_pixel_A	predicted_label	pred	conf
image_id
patch-0-0-50-50-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 0, 50, 50)	0.478751	0.453997	0.447296	1.0	0.046421	0.024224	0.040283	0.0	No	0	0.999786
patch-0-50-50-100-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 50, 50, 100)	0.811368	0.413255	0.238436	1.0	0.131187	0.100754	0.112988	0.0	No	0	0.957349
patch-0-100-50-150-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 100, 50, 150)	0.682626	0.314795	0.156773	1.0	0.085176	0.064740	0.082339	0.0	No	0	0.957769
patch-0-150-50-200-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 150, 50, 200)	0.630331	0.300660	0.159903	1.0	0.083151	0.062500	0.083706	0.0	No	0	0.971755
patch-0-200-50-250-#2014-06-06_plant001_rgb.png#.png	2014-06-06_plant001_rgb.png	/Users/rwood/LwM/MapReader/worked_examples/non...	(50, 50, 4)	(0, 200, 50, 250)	0.605075	0.351104	0.238678	1.0	0.056804	0.056446	0.065008	0.0	No	0	0.996740

[66]:

parent_list=my_files.list_parents()

[68]:

my_files.show_parent(parent_list[0],
                column_to_plot="pred",
                patch_border=True,
                plot_parent=True,
                figsize=(15, 15),
                alpha=0.5)

100%|█████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 12.34it/s]

[68]:

[<Figure size 1500x1500 with 2 Axes>]

../_images/Worked-examples_plant_pipeline_74_2.png

[69]:

my_files.show_parent(parent_list[0],
                column_to_plot="conf",
                patch_border=True,
                plot_parent=True,
                figsize=(15, 15),
                alpha=0.5)

100%|█████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 12.60it/s]

[69]:

[<Figure size 1500x1500 with 2 Axes>]

../_images/Worked-examples_plant_pipeline_75_2.png

[ ]:

Classification of plant phenotype

Load

Load images

Patchify images

Calculate pixel intensities

Annotate

Set up inputs

Annotate images and save annotations

Classify - Train

Read in annotations

Prepare datasets and dataloaders

Set up ClassifierContainer

Load a pretrained PyTorch model, dataloaders and labels map

Set up criterion, optimizer and scheduler

Train/fine-tune a model

Classify - Infer

Create dataset with all patches and add to ClassifierContainer

Infer labels

Add predictions to dataframe

Add outputs to MapImages as metadata

Set up `ClassifierContainer`

Create dataset with all patches and add to `ClassifierContainer`

Add outputs to `MapImages` as metadata