Example 8: Extracting shape features from fossil teeth of cichlids

This example demonstrates how basic and complex features of tooth-shapes are extracted from images containing single and multiple microscope images. Two separate phenopype projects are used for better organization (single tooth and multiple tooth per images). All images are kindly provided by Ole Seehausen.

Cichlid jaw Fig. 1 - Teeth of haplochromine cichlids, still attached to the jaw.

Single tooth case

These are single-tooth images where the background is brighter than the teeth, and also quite noisy. The challenges here lie in preventing unwanted objects from being detected (e.g. the checkered paper). In this case, the easiest way is to draw a rectangle mask around the object. Furthermore, the scale bar needs to be measured in every image because it varies in size between images. This is because the microscope software returns non-standardized scale bars that vary between images.

Before - single tooth case

After - single tooth case

Fig. 2 - single tooth, removed from the jaw.

High throughput

First we need to specify two directory: the root directory of the project (for example, nested within your project folder - see https://mluerig.github.io/phenopype/api.html#projects) and the path to the image folder.

[1]:
import phenopype as pp
import os

root = r"../_temp/output/ex8_single" ## this is the path to a (new) folder inside your project directory
images = r"images"

Project creation follows the usual procedure (see Tutorial 2, Tutorial 3, or Example 2 and Example 6): first we create the project at the root folder location, then we add the images (with “link” so the files don’t get copied - read about the differences between the two modes in the API), add pype-configuration files (the preset for this part is “ex8_1”), and save the project object.

[2]:
proj = pp.project(root_dir=root)
proj.add_files(image_dir=images, search_mode="recursive", raw_mode="link", include=["cichlid"], exclude=["multi"])
proj.add_config(name = "v1", overwrite=True, config_preset="ex8_1")
pp.project.save(proj)
--------------------------------------------
Phenopype will create a new project at
E:\git_repos\phenopype\_temp\output\ex8_single

Proceed? (y/n)
y

project attributes written to E:\git_repos\phenopype\_temp\output\ex8_single\attributes.yaml
--------------------------------------------
--------------------------------------------
phenopype will search for files at

E:\git_repos\phenopype\tutorials\images

using the following settings:

filetypes: ['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png'], include: ['cichlid'], exclude: ['multi'], raw_mode: link, search_mode: recursive, unique_mode: path

Found image cichlid1.jpg - phenopype-project folder 0__cichlid1 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found
Found image cichlid2.jpg - phenopype-project folder 0__cichlid2 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found
Found image cichlid3.jpg - phenopype-project folder 0__cichlid3 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found

Found 3 files
--------------------------------------------
pype config generated from ex8_1.
pype_v1.yaml created for 0__cichlid1
pype_v1.yaml created for 0__cichlid2
pype_v1.yaml created for 0__cichlid3
Project data saved under E:\git_repos\phenopype\_temp\output\ex8_single\project.data.

Alternatively, if you have already created a project, load it:

[3]:
proj = pp.project.load(path=root)
--------------------------------------------
Project loaded from
E:\git_repos\phenopype\_temp\output\ex8_single
--------------------------------------------

When using "link" with adding files, we need to switch to the phenopype root folder so that the relative paths are recognized:

[4]:
os.chdir(root)

Now we can process the loaded images in a simple for loop. This is the central step in the analysis, so refer to the above mentioned tutorials and examples if you are unsure how this procedure works. Briefly said, phenopype will process all the information provided with the config files, show you the result, and wait for your input. If you are happy with the result, then hit Enter after the the script has run (as indicated in the Python console), or change the parameters (e.g. the thresholding value or the contour size cutoff) until your results are satisfying. Here is an example-animation (using a different image, but the principle is the same) for two of the involved tools: create_mask and create_scale (See Tutorial 3 and Example 1):

Example: create scale

Mask creation - Drag a rectangle across the image and hit enter. The dection algorithims will then only operate within this ROI.

Example: mask tool

Scale measurement - Drag a line along the scale bar and hit enter. Then enter the measured distance in mm and hit enter. This will include pixel-to-mm information, which we want for every image. In this example, you don’t need to do the third step (template/mask creation).

[5]:
for dirpath in proj.dirpaths:
    p1 = pp.pype(dirpath,
                 name="v1",      ## note that this is the name we gave our configuration file.
                                 ## with this you can specify multiple configurations side by side, e.g. if you
                                 ## want to re-run some of the analysis, but don't want overwrite your past attempts
                 skip=True       ## this will skip image-folders that already have results in them
                )
E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid1\pype_config_v1.yaml


------------+++ new pype iteration 2020:07:01 19:55:56 +++--------------


Nothing loaded.
PREPROCESSING
create_mask
- create mask
create_scale
- measure pixel-to-mm-ratio
Scale set
- add column length
SEGMENTATION
blur
threshold
- include mask "mask1" pixels
morphology
find_contours
MEASUREMENT
shape_features
VISUALIZATION
select_canvas
- red channel
draw_contours
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid1\canvas_v1.jpg.
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid1\contours_v1.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid1\masks_v1.csv.
save_scale
- save scale to attributes
save_shapes
- shapes saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid1\shapes_v1.csv.


TERMINATE
E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid2\pype_config_v1.yaml


------------+++ new pype iteration 2020:07:01 19:56:10 +++--------------


Nothing loaded.
PREPROCESSING
create_mask
- create mask
create_scale
- measure pixel-to-mm-ratio
Scale set
- add column length
SEGMENTATION
blur
threshold
- include mask "mask1" pixels
morphology
find_contours
MEASUREMENT
shape_features
VISUALIZATION
select_canvas
- red channel
draw_contours
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid2\canvas_v1.jpg.
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid2\contours_v1.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid2\masks_v1.csv.
save_scale
- save scale to attributes
save_shapes
- shapes saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid2\shapes_v1.csv.


TERMINATE
E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid3\pype_config_v1.yaml


------------+++ new pype iteration 2020:07:01 19:56:26 +++--------------


Nothing loaded.
PREPROCESSING
create_mask
- create mask
create_scale
- measure pixel-to-mm-ratio
Scale set
- add column length
SEGMENTATION
blur
threshold
- include mask "mask1" pixels
morphology
find_contours
MEASUREMENT
shape_features
VISUALIZATION
select_canvas
- red channel
draw_contours
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid3\canvas_v1.jpg.
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid3\contours_v1.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid3\masks_v1.csv.
save_scale
- save scale to attributes
save_shapes
- shapes saved under E:\git_repos\phenopype\_temp\output\ex8_single\data\0__cichlid3\shapes_v1.csv.


TERMINATE

After we completed the loop, use the collect_results method of proj to save all results to a folder in the root directory.

[6]:
proj.collect_results(name="v1",files="shapes", folder="shape_features", overwrite=True)
Created E:\git_repos\phenopype\_temp\output\ex8_single\shape_features
Collected shapes_v1.csv from 0__cichlid1
0__cichlid1_shapes_v1.csv saved under E:\git_repos\phenopype\_temp\output\ex8_single\shape_features\0__cichlid1_shapes_v1.csv.
Collected shapes_v1.csv from 0__cichlid2
0__cichlid2_shapes_v1.csv saved under E:\git_repos\phenopype\_temp\output\ex8_single\shape_features\0__cichlid2_shapes_v1.csv.
Collected shapes_v1.csv from 0__cichlid3
0__cichlid3_shapes_v1.csv saved under E:\git_repos\phenopype\_temp\output\ex8_single\shape_features\0__cichlid3_shapes_v1.csv.

Multiple teeth

Now we will implement the same procedure for the images with multiple teeth, which require only a slightly altered configuration.

Before

After

High throughput

Before running this part, restart the Kernel to reset the working-directory position.

Again, we need to specify the paths, and create the project.

[1]:
import phenopype as pp
import os

root = r"../_temp/output/ex8_multiple" ## this is the path to a (new) folder inside your project directory
images = r"images"
[2]:
proj = pp.project(root_dir=root)
proj.add_files(image_dir=images, search_mode="recursive", raw_mode="link", include=["cichlid_multi"])
proj.add_config(name = "v1", overwrite=True, config_preset="ex8_2")
pp.project.save(proj)
--------------------------------------------
Phenopype will create a new project at
E:\git_repos\phenopype\_temp\output\ex8_multiple

Proceed? (y/n)
y
Warning - project root_dir already exists - overwrite? (y/n)y

"E:\git_repos\phenopype\_temp\output\ex8_multiple" created (overwritten)

project attributes written to E:\git_repos\phenopype\_temp\output\ex8_multiple\attributes.yaml
--------------------------------------------
--------------------------------------------
phenopype will search for files at

E:\git_repos\phenopype\tutorials\images

using the following settings:

filetypes: ['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png'], include: ['cichlid_multi'], exclude: [], raw_mode: link, search_mode: recursive, unique_mode: path

Found image cichlid_multi1.jpg - phenopype-project folder 0__cichlid_multi1 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found
Found image cichlid_multi2.jpg - phenopype-project folder 0__cichlid_multi2 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found
Found image cichlid_multi3.jpg - phenopype-project folder 0__cichlid_multi3 created
dirpath defaulted to file directory - E:\git_repos\phenopype\tutorials\images
Directory to save files set at - E:\git_repos\phenopype\tutorials\images
no meta-data found

Found 3 files
--------------------------------------------
pype config generated from ex8_2.
pype_v1.yaml created for 0__cichlid_multi1
pype_v1.yaml created for 0__cichlid_multi2
pype_v1.yaml created for 0__cichlid_multi3
Project data saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\project.data.
[3]:
os.chdir(root)

Note that here you need to perform an extra step after specifying the scale: since we did not use a mask that excludes unwanted noise (as in the single tooth part above), we need to manually exclude the little scale bar in the bottom corner. Therefore, in the example preset "ex8_2", the mask argument is already specified. After typing in the distance in mm, you need to drag a rectangle across the mask - (no need to stop the cursor at the edges of the window, just drag it beyond).

Now you can loop through all directories stored within the project object to detect the snails.

[4]:
for i in proj.dirpaths:
    p = pp.pype(i, name="v1")
E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi1\pype_config_v1.yaml


------------+++ new pype iteration 2020:07:01 19:57:19 +++--------------


Nothing loaded.
PREPROCESSING
create_scale
- measure pixel-to-mm-ratio
Scale set
- add column length
Template selected
SEGMENTATION
blur
threshold
- exclude mask "scale" pixels
morphology
find_contours
MEASUREMENT
shape_features
VISUALIZATION
select_canvas
- red channel
draw_contours
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi1\canvas_v1.jpg.
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi1\contours_v1.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi1\masks_v1.csv.
save_scale
- save scale to attributes
save_shapes
- shapes saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi1\shapes_v1.csv.


TERMINATE
E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi2\pype_config_v1.yaml


------------+++ new pype iteration 2020:07:01 19:57:34 +++--------------


Nothing loaded.
PREPROCESSING
create_scale
- measure pixel-to-mm-ratio
Scale set
- add column length
Template selected
SEGMENTATION
blur
threshold
- exclude mask "scale" pixels
morphology
find_contours
MEASUREMENT
shape_features
VISUALIZATION
select_canvas
- red channel
draw_contours
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi2\canvas_v1.jpg.
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi2\contours_v1.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi2\masks_v1.csv.
save_scale
- save scale to attributes
save_shapes
- shapes saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi2\shapes_v1.csv.


TERMINATE
E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi3\pype_config_v1.yaml


------------+++ new pype iteration 2020:07:01 19:57:45 +++--------------


Nothing loaded.
PREPROCESSING
create_scale
- measure pixel-to-mm-ratio
Scale set
- add column length
Template selected
SEGMENTATION
blur
threshold
- exclude mask "scale" pixels
morphology
find_contours
MEASUREMENT
shape_features
VISUALIZATION
select_canvas
- red channel
draw_contours
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi3\canvas_v1.jpg.
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi3\contours_v1.csv.
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi3\masks_v1.csv.
save_scale
- save scale to attributes
save_shapes
- shapes saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\data/0__cichlid_multi3\shapes_v1.csv.


TERMINATE

In the end, use the collect_results method of proj to save all results to a folder in the root directory.

[5]:
proj.collect_results(name="v1",files="shapes", folder="shape_features", overwrite=True)
Created E:\git_repos\phenopype\_temp\output\ex8_multiple\shape_features
Collected shapes_v1.csv from 0__cichlid_multi1
0__cichlid_multi1_shapes_v1.csv saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\shape_features\0__cichlid_multi1_shapes_v1.csv.
Collected shapes_v1.csv from 0__cichlid_multi2
0__cichlid_multi2_shapes_v1.csv saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\shape_features\0__cichlid_multi2_shapes_v1.csv.
Collected shapes_v1.csv from 0__cichlid_multi3
0__cichlid_multi3_shapes_v1.csv saved under E:\git_repos\phenopype\_temp\output\ex8_multiple\shape_features\0__cichlid_multi3_shapes_v1.csv.
[ ]: