Example 2: Stickleback morphometrics - landmarks

Functional morphology of organisms is often measured by placing landmarks at specific points that show structural, functional or developmental significance. In this example phenopype is used to place morphometric landmarks across the anterior half of a stickleback (Gasterosteus aculeatus) stained with alizarin red.

First we place landmarks in low throughput mode to learn how the landmark-function works, then we look at a high throughput landkmark example with a project directory and a global scale.

f7a70c8808c749ac854c0ff4473bf9a5

Input - Stained threespine stickleback, photographed in a glycerol bath from a camera stand

b1aadb5561e7478e972fd07a3b6ee2fb

Results - 22 landmarks are placed using the landmark tool from the phenopype.measurements module

Low throughput

[1]:
import phenopype as pp

filepath = r"images/stickle1.jpg"

ct = pp.load_image(filepath, cont=True) ## load image as container
[2]:
## just place a few test-landmarks

pp.measurement.landmarks(ct,
                         point_size=15,
                         point_colour="green",
                         label_size=1, overwrite=True)

## sets landmarks. the landmarks get stored inside the container. if you have already placed landmarks
## to the same container and set "overwrite=False", you wont be able to place them again until you set
## the overwrite flag to "True"
- setting landmarks

To create an image with the selected points, we first need te select the background image (or “canvas”) with select_canvas and then call the draw_landmarks function. These explicit steps are necessary when using the low throughput, but not the high throughput routine (see Tutorial 2); likewise the save_landmarks function.

[3]:
pp.visualization.select_canvas(ct, canvas="raw")
pp.visualization.draw_landmarks(ct, point_size=15,
                                point_colour="green",
                                label_size=1)
pp.show_image(ct.canvas)
ct.df_landmarks
- raw image
[3]:
filename width height size_ratio_original landmark x y
0 stickle1.jpg 2400 1600 1 1 1240 379
1 stickle1.jpg 2400 1600 1 2 1180 456
2 stickle1.jpg 2400 1600 1 3 1262 574
3 stickle1.jpg 2400 1600 1 4 1394 773
[4]:
## draw landmarks on canvas. appearance needs to be specified here too
pp.export.save_landmarks(ct, dirpath=r"../_temp/output/ex2") ## save landmarks as csv to folder
pp.export.save_canvas(ct, dirpath=r"../_temp/output/ex2") ## also save canvas for quality control
- landmarks saved under ../_temp/output/ex2\landmarks.csv (overwritten).
- canvas saved under ../_temp/output/ex2\canvas.jpg (overwritten).

Adding a scale

Now we will do the same thing again, but this time we use a reference image to create a scale-template so we can adjust our landmark coordinate space. This is important if for example the distance between the camera and your sample changes.

Adding a scale

After loading the reference image, we measure the distance on the millimeter scale (click on two points inside the image), enter the distance (e.g. 10 mm), and then we create a template by dragging a rectangle around the whole (!) reference card. Finish each step with “enter”.

[5]:
import phenopype as pp

ref_path = r"images/stickleback_side.jpg"
filepath = r"images/stickle1.jpg"

ref_image = pp.load_image(ref_path) ## load image as container
ref_ratio, df_masks, template_img = pp.preprocessing.create_scale(ref_image, template=True)
- measure pixel-to-mm-ratio
Scale set
- add column length
Template selected
- scale pixel-to-mm-ratio already measured (overwrite=False)

In the next step, we load the sample image again. The find_scale uses a classic machine learning algorithm to find the scale inside our already processed image.

[6]:
ct = pp.load_image(filepath, cont=True)
pp.preprocessing.find_scale(ct,
                            template=template_img,
                            px_mm_ratio_ref=ref_ratio,
                            equalize=False)
---------------------------------------------------
Reference card found with 235 keypoint matches:
template image has 36 pixel per mm.
current image has 34.8 pixel per mm.
= 96.748 % of template image.
---------------------------------------------------

This was successful - we see that the reference card found in the image has only ~ 96 % if the size of the card we marked in the template image. This means that one mm in our current image is 34.9 pixels, whereas it was 36 pixels in the original image. This information is automatically passed on to all produced data that is produced after detecting the scale.

Enter some data

For high data quality it is important to verify the ID of the specimen in the current picture. Often, the picture name contains the ID, but typically an label is placed inside the image. Using the enter_data tool we open the image and an entry prompt that will create a column with a name of our chosing inside all exported results:

[7]:
pp.preprocessing.enter_data(ct, columns="ID")
- add column ID

Now we draw the perimeter around the detected scale. Because the detected scale has coordinates like a mask we need the draw_masks function to draw it.

[8]:
pp.visualization.select_canvas(ct, canvas="raw")
pp.visualization.draw_masks(ct)
- raw image
 - show mask: scale.

Now we place our landmarks again. The resulting csv now contains a column for the pixel-to-mm-ratio from the scale we detected.

[9]:
pp.measurement.landmarks(ct, point_size=15, point_colour="green", label_size=1, overwrite=True)
pp.visualization.draw_landmarks(ct, point_size=15, point_colour="green", label_size=1)
- setting landmarks

Finally, we look at the results and export them.

[10]:
pp.show_image(ct.canvas)
ct.df_landmarks
[10]:
filename width height size_ratio_original current_px_mm_ratio ID landmark x y
0 stickle1.jpg 2400 1600 1 34.8 142501 1 1543 430
1 stickle1.jpg 2400 1600 1 34.8 142501 2 1490 468
2 stickle1.jpg 2400 1600 1 34.8 142501 3 1202 559
3 stickle1.jpg 2400 1600 1 34.8 142501 4 1087 545
[11]:
pp.export.save_landmarks(ct, dirpath=r"../_temp/output/ex2")
pp.export.save_canvas(ct, dirpath=r"../_temp/output/ex2")
- landmarks saved under ../_temp/output/ex2\landmarks.csv (overwritten).
- canvas saved under ../_temp/output/ex2\canvas.jpg (overwritten).

High throughput

Now we will use the pype method to place landmarks - first on a single file, and then from within a Phenopype project.

Landmarks in high throughput

High throughput method - the pype method opens the image and a text editor with the pype configuration file. Any change to the configuration file, in this case the point size for landmarks, will be immediately applied to the image.

[1]:
import phenopype as pp
import os

The pype can be used outside of a penopype project, supplying arrays or paths to images. Run the pype function, go to the output dirpath, and check the collected results.

[3]:
filepath = r"images/stickle1.jpg" # works
image = pp.load_image(filepath) # also works

pp.pype(filepath, name="lm1",
        config_preset="landmarks_plain",
        dirpath=r"../_temp/output/ex2")
## both "image" and filepath work
## dirpath specifies a directory where all results are saved
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
../_temp/output/ex2\pype_config_lm1.yaml


------------+++ new pype iteration 2020:05:17 13:30:42 +++--------------


Nothing loaded.
MEASUREMENT
landmarks
- setting landmarks
VISUALIZATION
- modifed image
- autoselect canvas
draw_landmarks
EXPORT
save_landmarks
- landmarks saved under ../_temp/output/ex2\landmarks_lm1.csv.
AUTOSAVE
save_canvas
- canvas saved under ../_temp/output/ex2\canvas_lm1.jpg.


TERMINATE
[3]:
<phenopype.main.pype at 0x23e23ed3608>

However, to increase throughput we will use the pype function on files organized within a phenopype project (see Tutorial 2 and Tutorial 3 for more general information on high throughput workflow). We start with providing some paths, and including select images into the project (i.e. all stickleback images). Make sure you

[4]:
## relative from now (phenopype-master/tutorials)
project_root = r"../_temp/output/ex2_project"

## relative from project root directory (phenopype-master/_temp/project)
image_dir = "images"
reference_image = "images/stickleback_side.jpg"
[5]:
myproj = pp.project(root_dir=project_root)
--------------------------------------------
Phenopype will create a new project at
E:\git_repos\phenopype\_temp\output\ex2_project

Proceed? (y/n)
y

project attributes written to E:\git_repos\phenopype\_temp\output\ex2_project\attributes.yaml
--------------------------------------------

First we add the image files in the directory, but only “stickle1”, “stickle2”, and “stickle3”. Then we add the appropriate configuration file. As for the other examples I have created a preset (“ex2”) with appropriate settings, which is passed to the pype using config_preset="ex2".

[6]:
myproj.add_files(image_dir=image_dir, include="stickle", exclude=["side","top"])
myproj.add_config(name = "lm2", config_preset="ex2")
print(pp.presets.ex2)
--------------------------------------------
phenopype will search for files at

E:\git_repos\phenopype\tutorials\images

using the following settings:

filetypes: ['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png'], include: stickle, exclude: ['side', 'top'], raw_mode: copy, search_mode: dir, unique_mode: path

Found image stickle1.JPG - phenopype-project folder 0__stickle1 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found
Found image stickle2.JPG - phenopype-project folder 0__stickle2 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found
Found image stickle3.JPG - phenopype-project folder 0__stickle3 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found

Found 3 files
--------------------------------------------
pype config generated from ex2.
pype_lm2.yaml created for 0__stickle1
pype_lm2.yaml created for 0__stickle2
pype_lm2.yaml created for 0__stickle3

preprocessing:
- find_scale
- enter_data
measurement:
- landmarks:
    point_size: 12
    point_colour: green
    label_size: 2
    label_width: 2
visualization:
- draw_masks
- draw_landmarks:
    point_size: 12
    point_colour: green
    label_size: 2
    label_width: 2
export:
- save_landmarks
- save_masks
- save_data_entry

Now we again set scale, but this time we pass on the information to all images included in the project. Afterwards, we save the project (to the root folder).

[7]:
myproj.add_scale(reference_image = reference_image, overwrite=True)
pp.project.save(myproj)
- scale template saved under E:\git_repos\phenopype\_temp\output\ex2_project\scale_template.jpg.
- measure pixel-to-mm-ratio
Scale set
- add column length
Template selected
added scale information to 0__stickle1
added scale information to 0__stickle2
added scale information to 0__stickle3
Project data saved under E:\git_repos\phenopype\_temp\output\ex2_project\project.data.
[8]:
## if you have already run above cell you can load your project using "project.load":
# myproj = pp.project.load(r"../_temp/project/project.data") ## run this if you

After creating the project files, we can run the pype routine with a simple loop on myproj.dirpaths, which is a list of all project directories that contain the copied raw images and the config files we generated before. Interrupt the loop with Esc. To resume to the point where you left, add the skip argument so directory with processed files are not run again.

[9]:
for dirpath in myproj.dirpaths:
    out = pp.pype(dirpath, name="lm2", skip=True)
E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle1\pype_config_lm2.yaml


------------+++ new pype iteration 2020:05:17 13:32:52 +++--------------


AUTOLOAD
- template scale information loaded from attributes.yaml
- template loaded from root directory
PREPROCESSING
find_scale
---------------------------------------------------
Reference card found with 237 keypoint matches:
template image has 36 pixel per mm.
current image has 34.9 pixel per mm.
= 97.009 % of template image.
---------------------------------------------------
enter_data
- add column ID
MEASUREMENT
landmarks
- setting landmarks
VISUALIZATION
- modifed image
- autoselect canvas
draw_masks
 - show mask: scale.
draw_landmarks
EXPORT
save_landmarks
- landmarks saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle1\landmarks_lm2.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle1\masks_lm2.csv.
save_data_entry
- add column ID
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle1\canvas_lm2.jpg.
save_scale
- save scale to attributes


TERMINATE
E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle2\pype_config_lm2.yaml


------------+++ new pype iteration 2020:05:17 13:33:02 +++--------------


AUTOLOAD
- template scale information loaded from attributes.yaml
- template loaded from root directory
PREPROCESSING
find_scale
---------------------------------------------------
Reference card found with 215 keypoint matches:
template image has 36 pixel per mm.
current image has 34.8 pixel per mm.
= 96.67 % of template image.
---------------------------------------------------
enter_data
- add column ID
MEASUREMENT
landmarks
- setting landmarks
VISUALIZATION
- modifed image
- autoselect canvas
draw_masks
 - show mask: scale.
draw_landmarks
EXPORT
save_landmarks
- landmarks saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle2\landmarks_lm2.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle2\masks_lm2.csv.
save_data_entry
- add column ID
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle2\canvas_lm2.jpg.
save_scale
- save scale to attributes


TERMINATE
E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle3\pype_config_lm2.yaml


------------+++ new pype iteration 2020:05:17 13:33:11 +++--------------


AUTOLOAD
- template scale information loaded from attributes.yaml
- template loaded from root directory
PREPROCESSING
find_scale
---------------------------------------------------
Reference card found with 240 keypoint matches:
template image has 36 pixel per mm.
current image has 34.8 pixel per mm.
= 96.78 % of template image.
---------------------------------------------------
enter_data
- add column ID
MEASUREMENT
landmarks
- setting landmarks
VISUALIZATION
- modifed image
- autoselect canvas
draw_masks
 - show mask: scale.
draw_landmarks
EXPORT
save_landmarks
- landmarks saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle3\landmarks_lm2.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle3\masks_lm2.csv.
save_data_entry
- add column ID
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex2_project\data/0__stickle3\canvas_lm2.jpg.
save_scale
- save scale to attributes


TERMINATE
[ ]: